By django


2019-04-15 10:19:55 8 Comments

Objective: I am coming from javascript background. I am try to parse a json. json.loads is supposed to convert stringfied values into their relevant type.

How can it be done with python 3? Purpose is eval all values with relevant type.

Scenerio: I am reading csv in python when reading csv, values are converted to strings I removed csv code becuase it was not relevant !!!

Code:

import json
x = '{ "name":"John", "age":30, "dev":"true", "trig":"1.0E-10", "res":"0.1"}'
y = json.loads(x)
print(y)

Current Output:

{
  "name": "John",
  "age": "30", 
  "dev": "true", 
  "trig": "1.0E-10", 
  "res": "0.1"
}

Expected output:

{
  "name": "John",
  "age": 30,       // int 
  "dev": true,     //  bool
  "trig": 1.0E-10, // real number
  "res": 0.1       // float
}

2 comments

@thebjorn 2019-04-15 10:49:15

Your fundamental problem is that your json data contains strings, not values (e.g. "dev":"true" instead of "dev":true). Parsing the string in javascript will hit the same problems you're seeing in Python:

(dev) go|c:\srv\tmp> node
> x = '{ "name":"John", "age":30, "dev":"true", "trig":"1.0E-10", "res":"0.1"}'
'{ "name":"John", "age":30, "dev":"true", "trig":"1.0E-10", "res":"0.1"}'
> JSON.parse(x)
{ name: 'John', age: 30, dev: 'true', trig: '1.0E-10', res: '0.1' }
> JSON.parse(x).dev
'true'
> typeof JSON.parse(x).dev
'string'

The real solution here is to fix whatever is creating such malformed json.

You can hack your way around it in Python by e.g.:

import ast, json

x = '{ "name":"John", "age":30, "dev":"true", "trig":"1.0E-10", "res":"0.1"}'

def evalfn(pairs):
    res = {}
    for key, val in pairs:
        if val in {'true','false'}:
            res[key] = val == 'true'
            continue
        try:
            res[key] = ast.literal_eval(val)
        except Exception as e:
            res[key] = val
    return res

y = json.loads(x, object_pairs_hook=evalfn)
print y

which will print

{u'trig': 1e-10, u'res': 0.1, u'age': 30, u'name': u'John', u'dev': True}

@django 2019-04-15 10:53:09

Thankx. It works. But what should be standard practive when reading csv files , as csv reader converts every value to string

@thebjorn 2019-04-15 10:59:47

That sounds like something you should ask in a new question, but normally you'll know the data types for each column when you're parsing csv by hand (it's still a very manual process). All the "cool kids" seem to be using pandas for such things these days though, and it has a much more sophisticated csv parser (pandas.pydata.org/pandas-docs/stable/reference/api/…)

@django 2019-04-15 11:21:10

Thankx yes i know a bit about pandas, but i was more curious about vanilla way. Your solution is very helpful. Much appreciated

@Filip MÅ‚ynarski 2019-04-15 10:31:42

First you need to load your json from file

file = open('data.json', 'r')
content = file.read()
file.close()

Then we can go over each value and check whether we can convert it to int or float or if its either 'true' or 'false', if so we update specific value of our dictionary.

import json

loaded_json = json.loads(content)

def is_type(x, t):
    try:
        t(x)
        return True
    except:
        return False

for k, v in loaded_json.items():
    if is_type(v, int):
        loaded_json[k] = int(v)
    elif is_type(v, float):
        loaded_json[k] = float(v)
    elif v == 'true':
        loaded_json[k] = True
    elif v == 'false':
        loaded_json[k] = False

for k, v in sorted(loaded_json.items()):
    print(k, v, '//', type(v))

Output:

age 30 // <class 'int'>
dev True // <class 'bool'>
name John // <class 'str'>
res 0.1 // <class 'float'>
trig 1e-10 // <class 'float'>

@django 2019-04-15 10:34:15

Thankx , I have 2 questions, How can i do it with json_loads hook 2nd - should json_loads not suport it by defualt, how can i use parse_int option mentioned in docs

@Filip MÅ‚ynarski 2019-04-15 10:44:10

Since your values are in quotation marks " they are being interpreted as strings, if your values wouldn't have them they would be loaded as their corresponding types. I never used object_hook or parse_int arguments.

Related Questions

Sponsored Content

9 Answered Questions

[SOLVED] Why can't Python parse this JSON data?

  • 2010-05-14 15:54:20
  • michele
  • 2503790 View
  • 1344 Score
  • 9 Answer
  • Tags:   python json parsing

22 Answered Questions

[SOLVED] Converting string into datetime

  • 2009-01-21 18:00:29
  • Oli
  • 2350182 View
  • 1811 Score
  • 22 Answer
  • Tags:   python datetime

30 Answered Questions

[SOLVED] Convert JS object to JSON string

6 Answered Questions

[SOLVED] How do I lowercase a string in Python?

25 Answered Questions

[SOLVED] Safely turning a JSON string into an object

  • 2008-09-05 00:12:01
  • Matt Sheppard
  • 1181588 View
  • 1235 Score
  • 25 Answer
  • Tags:   javascript json

17 Answered Questions

[SOLVED] Does Python have a string 'contains' substring method?

32 Answered Questions

[SOLVED] How do I check if a string is a number (float)?

25 Answered Questions

[SOLVED] How do I parse a string to a float or int in Python?

16 Answered Questions

[SOLVED] Convert bytes to a string?

9 Answered Questions

[SOLVED] Converting integer to string in Python?

  • 2009-06-07 10:22:38
  • Hick
  • 2961753 View
  • 1170 Score
  • 9 Answer
  • Tags:   python

Sponsored Content