By michele


2010-05-14 15:54:20 8 Comments

I have this JSON in a file:

{
    "maps": [
        {
            "id": "blabla",
            "iscategorical": "0"
        },
        {
            "id": "blabla",
            "iscategorical": "0"
        }
    ],
    "masks": [
        "id": "valore"
    ],
    "om_points": "value",
    "parameters": [
        "id": "valore"
    ]
}

I wrote this script which prints all of the json text:

json_data=open(file_directory).read()

data = json.loads(json_data)
pprint(data)

How can I parse the file and extract single values?

9 comments

@muratgozel 2018-11-30 09:21:20

As a python3 user,

The difference between load and loads methods is important especially when you read json data from file.

As stated in the docs:

json.load:

Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.

json.loads:

json.loads: Deserialize s (a str, bytes or bytearray instance containing a JSON document) to a Python object using this conversion table.

json.load method can directly read opened json document since it is able to read binary file.

with open('./recipes.json') as data:
  all_recipes = json.load(data)

As a result, your json data available as in a format specified according to this conversion table:

https://docs.python.org/3.7/library/json.html#json-to-py-table

@Bibin Wilson 2018-10-05 13:47:08

There are two types in this parsing.

  1. Parsing data from a file from a system path
  2. Parsing JSON from remote URL.

From a file, you can use the following

import json
json = json.loads(open('/path/to/file.json').read())
value = json['key']
print json['value']

This arcticle explains the full parsing and getting values using two scenarios.Parsing JSON using Python

@Geng Jiawen 2015-05-22 03:44:04

Justin Peel's answer is really helpful, but if you are using Python 3 reading JSON should be done like this:

with open('data.json', encoding='utf-8') as data_file:
    data = json.loads(data_file.read())

Note: use json.loads instead of json.load. In Python 3, json.loads takes a string parameter. json.load takes a file-like object parameter. data_file.read() returns a string object.

@Zearin 2015-07-16 14:55:35

Why should json.load be avoided in favor of .loads in Python 3?

@Geng Jiawen 2015-07-17 02:13:24

@Zearin pls check the official doc.

@Dan Hulme 2016-03-19 17:58:47

The page you linked doesn't say anything about avoiding load.

@Geng Jiawen 2016-07-22 08:38:12

@Dan Hulme, Because data_file.read() return a string.The json.loads take string parameters. The json.load take file-like object.

@Łukasz Rogalski 2016-08-02 09:41:47

This answer read whole file to memory when is does not have to and suggests that in Python 3 JSON files cannot be read lazily, which is untrue. I'm sorry, but it's clear downvote.

@dusktreader 2016-09-30 21:21:39

This answer isn't accurate. There's no reason not to use json.load with an open file handler in python3. Sorry for the downvote, but it doesn't seem like you read the above comments very carefully.

@Corey Goldberg 2017-06-15 15:28:43

this answer works, but there is no need to convert to a string by reading file contents explicitly just so you can use json.loads. using json.load and a file-like object does this for you and is a better solution.

@newpeople 2017-07-28 14:42:55

+1 This answer is great! Thank you for that and pulled me from going far for looking for a function that can use strings cause I only work with strings and network request that are not file!

@Geng Jiawen 2018-03-23 03:42:05

@ŁukaszRogalski I know, but for most file, the memory is not an issue.

@Escher 2018-04-12 11:11:58

Since python 3.6, json.loads now supports bytes, not just string.

@Justin Peel 2010-05-14 16:10:15

I think what Ignacio is saying is that your JSON file is incorrect. You have [] when you should have {}. [] are for lists, {} are for dictionaries.

Here's how your JSON file should look, your JSON file wouldn't even load for me:

{
    "maps": [
        {
            "id": "blabla",
            "iscategorical": "0"
        },
        {
            "id": "blabla",
            "iscategorical": "0"
        }
    ],
    "masks": {
        "id": "valore"
    },
    "om_points": "value",
    "parameters": {
        "id": "valore"
    }
}

Then you can use your code:

import json
from pprint import pprint

with open('data.json') as f:
    data = json.load(f)

pprint(data)

With data, you can now also find values like so:

data["maps"][0]["id"]
data["masks"]["id"]
data["om_points"]

Try those out and see if it starts to make sense.

@michele 2010-05-14 16:26:20

Ok so I have to control my code because this json file is generated from a java object. Thanks.

@radtek 2014-12-23 18:43:40

serialized data is wrapped with [] , and when you read it in you need f.read(), that is if you use the standard.

@diaryfolio 2015-01-30 15:36:35

Thanks for the solution. i'm getting a unicode symbol while printing it. (eg u'valore' ). How to prevent it?

@CodyBugstein 2015-07-05 07:14:26

Nice but python adds a u' before each key. Any idea why?

@Michael P 2015-08-29 11:56:15

That is why your text is type unicode not string. Most time it is better to have text in unicode for german umlauts and for sharing text results with other modules/programs etc. . So you're good!

@Karthi 2017-04-26 02:34:18

How to know size of the maps array to control index in this example?data["maps"][0]["id"] - Here 0 hard coded.

@Max Heiber 2018-01-16 19:37:08

isn't there a resource leak because the handle to data.json is never closed?

@Nikhil VJ 2018-02-19 15:19:09

In python 3, my json file is an array [] of jsons, it's called valid json by online checkers, and with these commands it loaded perfectly. Perhaps the definitions have changed circa 2018?

@Justin Peel 2018-02-22 18:08:09

@nikhilvj json doesn't need to have {} at the root level. It can start with an array at the root level ([])

@Kyle Bridenstine 2018-07-13 19:27:54

stackoverflow.com/a/27415238/3299397 need this piece for it to work.

@Kyle Bridenstine 2018-08-21 19:00:51

What exception will be thrown if the with call fails? Should this be wrapped in a try catch?

@AbhinavVaidya8 2018-09-28 09:29:58

Nice explanation @JustinPeel.

@Larold 2018-10-12 19:12:27

I'd like to make an observation that is hopefully helpful, and definitely ironic. I find the pprint module to be inferior to the json module for pretty-printing json. If you try them both, I think you'll agree. To display and debug my json data structures, I've been doing: output = json.dumps(data_structure, indent=2, sort_keys=True) print(output) I think you'll find the indent-control, sorting, and intelligent line-wrapping in the dumps() method to be quite to your liking. If my thinking is wrong, someone please let me know.

@sushmit 2017-04-25 17:42:50

if you are in python 3 here is how you can do it

{
  "connection1": {
    "DSN": "con1",
    "UID": "abc",
    "PWD": "1234",
    "connection_string_python":"test1"
  }
  ,
  "connection2": {
    "DSN": "con2",
    "UID": "def",
    "PWD": "1234"
  }
}

The code should look like assuming connection.json file looks like above

connection_file = open('connection.json', 'r')
conn_string = json.load(connection_file)
conn_string['connection1']['connection_string_python'])
connection_file.close()
>>>test1

@siddardha 2017-05-16 18:58:13

this also works in 2.7.5

@Corey Goldberg 2017-06-15 15:31:38

this leaves the file handle open. using a with statement would be better

@moeabdol 2015-03-09 21:38:50

"Ultra JSON" or simply "ujson" can handle having [] in your JSON file input. If you're reading a JSON input file into your program as a list of JSON elements; such as, [{[{}]}, {}, [], etc...] ujson can handle any arbitrary order of lists of dictionaries, dictionaries of lists.

You can find ujson in the Python package index and the API is almost identical to Python's built-in json library.

ujson is also much faster if you're loading larger JSON files. You can see the performance details in comparison to other Python JSON libraries in the same link provided.

@JoboFive 2017-11-07 01:53:18

   # Here you go with modified json file:
   # data.json file : 
    {
        "maps": [
            {
                "id": "blabla",
                "iscategorical": "0"
            },
            {
                "id": "blabla",
                "iscategorical": "0"
            }
        ],
        "masks": [{
            "id": "valore"
        }],
        "om_points": "value",
        "parameters": [{
            "id": "valore"
        }]
    }


   # You can call or print data on console by using below lines

    import json
    from pprint import pprint
    with open('data.json') as data_file:
        data_item = json.load(data_file)
    pprint(data_item)

    print(data_item['parameters'][0]['id'])

    #Output : 
    #pprint(data_item) output as :

    {'maps': [{'id': 'blabla', 'iscategorical': '0'},
              {'id': 'blabla', 'iscategorical': '0'}],
     'masks': [{'id': 'valore'}],
     'om_points': 'value',
     'parameters': [{'id': 'valore'}]}
    #print(data_item['parameters'][0]['id']) output as :
    valore

@Chenxi 2018-06-07 17:24:21

If we would like add a column to count how many observations does "maps" have, how could we write this function?

@Bengt 2012-11-29 20:10:03

Your data.json should look like this:

{
 "maps":[
         {"id":"blabla","iscategorical":"0"},
         {"id":"blabla","iscategorical":"0"}
        ],
"masks":
         {"id":"valore"},
"om_points":"value",
"parameters":
         {"id":"valore"}
}

Your code should be:

import json
from pprint import pprint

with open('data.json') as data_file:    
    data = json.load(data_file)
pprint(data)

Note that this only works in Python 2.6 and up, as it depends upon the with-statement. In Python 2.5 use from __future__ import with_statement, in Python <= 2.4, see Justin Peel's answer, which this answer is based upon.

You can now also access single values like this:

data["maps"][0]["id"]  # will return 'blabla'
data["masks"]["id"]    # will return 'valore'
data["om_points"]      # will return 'value'

@Bengt 2013-02-26 19:57:38

I got a downvote on this. Maybe it was not clear, why I thought another answer was necessary. Added note on compatibility of the with-statement.

@Bengt 2013-05-25 12:10:18

Sorry for the roll back, but the suggested code would keep data_file opened longer than necessary.

@Steve S. 2015-06-16 01:54:06

Referring to 2.6 documentation (docs.python.org/2.6/library/io.html), opening a file in the "with" context will automatically close the file.

@Bengt 2015-06-16 17:45:55

@SteveS. Yes, but not before the context is left. pprinting in the with-context keeps the data_file open longer.

@Gayan Pathirage 2017-03-15 10:16:57

Is there a way to access like data.om_points or data.masks.id?

@Patrick Schaefer 2017-04-03 19:25:31

This works except when I try to use a numbered index like data["maps"][0]["id"] I see error: KeyError: 0

@Nuhman 2018-05-25 04:55:09

@GayanPathirage you access it like data["om_points"] , data["masks"]["id"]. The idea is you can reach any level in a dictionary by specifying the 'key paths'. If you get a KeyError exception it means the key doesn't exist in the path. Look out for typos or check the structure of your dictionary.

@user1743724 2013-05-12 20:47:46

data = []
with codecs.open('d:\output.txt','rU','utf-8') as f:
    for line in f:
       data.append(json.loads(line))

@yasin_alm 2016-03-21 21:43:35

this is the correct solution if you have multiple json objects in a file. json.loads does not decode multiple json objects. Otherwise, you get 'Extra Data' error.

@Earthx9 2016-06-11 12:05:28

This is the best answer. Otherwise, it gives 'Extra Data' error.

@dusktreader 2016-09-30 21:23:07

Having mutliple json objects in a file means that the file itself is not actually valid json. If you have multiple objects to include in a json file, they should be contained in an array at the top level of the file.

Related Questions

Sponsored Content

41 Answered Questions

[SOLVED] How do I sort a dictionary by value?

25 Answered Questions

[SOLVED] How do I list all files of a directory?

  • 2010-07-08 19:31:22
  • duhhunjonn
  • 2955114 View
  • 3251 Score
  • 25 Answer
  • Tags:   python directory

40 Answered Questions

[SOLVED] How do I check whether a file exists without exceptions?

33 Answered Questions

[SOLVED] What is the correct JSON content type?

25 Answered Questions

[SOLVED] How do I parse a string to a float or int in Python?

46 Answered Questions

[SOLVED] Can comments be used in JSON?

  • 2008-10-28 20:39:03
  • Michael Gundlach
  • 1653659 View
  • 6277 Score
  • 46 Answer
  • Tags:   json comments

53 Answered Questions

[SOLVED] How can I pretty-print JSON in a shell script?

17 Answered Questions

6 Answered Questions

[SOLVED] Why does Google prepend while(1); to their JSON responses?

16 Answered Questions

[SOLVED] Parse JSON in JavaScript?

Sponsored Content