By UberJumper


2009-06-12 18:25:02 8 Comments

How can I convert the str representation of a dict, such as the following string, into a dict?

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

I prefer not to use eval. What else can I use?

The main reason for this, is one of my coworkers classes he wrote, converts all input into strings. I'm not in the mood to go and modify his classes, to deal with this issue.

11 comments

@Siva Kameswara Rao Munipalle 2016-07-30 08:19:39

string = "{'server1':'value','server2':'value'}"

#Now removing { and }
s = string.replace("{" ,"")
finalstring = s.replace("}" , "")

#Splitting the string based on , we get key value pairs
list = finalstring.split(",")

dictionary ={}
for i in list:
    #Get Key Value pairs separately to store in dictionary
    keyvalue = i.split(":")

    #Replacing the single quotes in the leading.
    m= keyvalue[0].strip('\'')
    m = m.replace("\"", "")
    dictionary[m] = keyvalue[1].strip('"\'')

print dictionary

@ZF007 2017-12-13 15:08:02

Your code goes wrong when hitting nested dict.

@Om Prakash Sao 2019-03-29 10:28:13

Many mistakes in this approach. What if value of a key contains { or }. What if it is nested dict. What if value contains , ??

@Anatoly Alekseev 2018-07-19 23:20:05

To summarize:

import ast, yaml, json, timeit

descs=['short string','long string']
strings=['{"809001":2,"848545":2,"565828":1}','{"2979":1,"30581":1,"7296":1,"127256":1,"18803":2,"41619":1,"41312":1,"16837":1,"7253":1,"70075":1,"3453":1,"4126":1,"23599":1,"11465":3,"19172":1,"4019":1,"4775":1,"64225":1,"3235":2,"15593":1,"7528":1,"176840":1,"40022":1,"152854":1,"9878":1,"16156":1,"6512":1,"4138":1,"11090":1,"12259":1,"4934":1,"65581":1,"9747":2,"18290":1,"107981":1,"459762":1,"23177":1,"23246":1,"3591":1,"3671":1,"5767":1,"3930":1,"89507":2,"19293":1,"92797":1,"32444":2,"70089":1,"46549":1,"30988":1,"4613":1,"14042":1,"26298":1,"222972":1,"2982":1,"3932":1,"11134":1,"3084":1,"6516":1,"486617":1,"14475":2,"2127":1,"51359":1,"2662":1,"4121":1,"53848":2,"552967":1,"204081":1,"5675":2,"32433":1,"92448":1}']
funcs=[json.loads,eval,ast.literal_eval,yaml.load]

for  desc,string in zip(descs,strings):
    print('***',desc,'***')
    print('')
    for  func in funcs:
        print(func.__module__+' '+func.__name__+':')
        %timeit func(string)        
    print('')

Results:

*** short string ***

json loads:
4.47 µs ± 33.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
builtins eval:
24.1 µs ± 163 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
ast literal_eval:
30.4 µs ± 299 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
yaml load:
504 µs ± 1.29 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

*** long string ***

json loads:
29.6 µs ± 230 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
builtins eval:
219 µs ± 3.92 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
ast literal_eval:
331 µs ± 1.89 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
yaml load:
9.02 ms ± 92.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Conclusion: prefer json.loads

@Michael Campbell 2018-11-26 16:49:07

Except this won't work with his single-quoted string, which was part of his initial problem. Performance was never mentioned.

@Anatoly Alekseev 2018-11-26 21:04:46

You are right. I'll go kill myself then.

@tamerlaha 2016-12-26 09:48:07

no any libs are used:

dict_format_string = "{'1':'one', '2' : 'two'}"
d = {}
elems  = filter(str.isalnum,dict_format_string.split("'"))
values = elems[1::2]
keys   = elems[0::2]
d.update(zip(keys,values))

NOTE: As it has hardcoded split("'") will work only for strings where data is "single quoted".

@Rogerio Silveira 2014-08-27 12:50:13

Use json. the ast library consumes a lot of memory and and slower. I have a process that needs to read a text file of 156Mb. Ast with 5 minutes delay for the conversion dictionary json and 1 minutes using 60% less memory!

@ntg 2016-12-08 15:04:41

but has its limits: try converting the string "{'foo':'bar',}"

@tokhi 2014-08-15 12:07:45

using json.loads:

>>> import json
>>> h = '{"foo":"bar", "foo2":"bar2"}'
>>> type(h)
<type 'str'>
>>> d = json.loads(h)
>>> d
{u'foo': u'bar', u'foo2': u'bar2'}
>>> type(d)
<type 'dict'>

@technazi 2016-05-13 14:28:34

I dont think it answers the OP's answer. How do we use json.laads to convert a string s = "{'muffin' : 'lolz', 'foo' : 'kitty'}" to dict?

@9codie05 2016-07-13 18:24:44

why is this printing 'u' in the output?? eg - str = '{"1":"P", "2":"N", "3":"M"}' d = json.loads(str) print d output is : {u'1': u'P', u'3': u'M', u'2': u'N'}

@tokhi 2016-07-14 09:43:26

@9codie05: u denotes the Unicode representation.

@ntg 2016-12-08 15:08:44

@technazi: json.loads(h.replace("'",'"'))

@ntg 2016-12-08 15:29:24

However, there are limits, e.g.: h= '{"muffin" : "lolz", "foo" : "kitty",}', also h= '{"muffin's" : "lolz", "foo" : "kitty"}', (just noticed part of the same comments in a similar answer... still leaving here for completeness...)

@nostradamus 2017-01-11 08:40:03

In my opinion, that's the shortest and easiest way... Definitely the one I personally prefer.

@MrR 2019-02-23 00:04:37

@nostradamus Too many exceptions, floating point values, tuples, etc. etc.

@Jacob Gabrielson 2009-06-12 18:30:45

Starting in Python 2.6 you can use the built-in ast.literal_eval:

>>> import ast
>>> ast.literal_eval("{'muffin' : 'lolz', 'foo' : 'kitty'}")
{'muffin': 'lolz', 'foo': 'kitty'}

This is safer than using eval. As its own docs say:

>>> help(ast.literal_eval)
Help on function literal_eval in module ast:

literal_eval(node_or_string)
    Safely evaluate an expression node or a string containing a Python
    expression.  The string or node provided may only consist of the following
    Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
    and None.

For example:

>>> eval("shutil.rmtree('mongo')")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "/opt/Python-2.6.1/lib/python2.6/shutil.py", line 208, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/opt/Python-2.6.1/lib/python2.6/shutil.py", line 206, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: 'mongo'
>>> ast.literal_eval("shutil.rmtree('mongo')")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/Python-2.6.1/lib/python2.6/ast.py", line 68, in literal_eval
    return _convert(node_or_string)
  File "/opt/Python-2.6.1/lib/python2.6/ast.py", line 67, in _convert
    raise ValueError('malformed string')
ValueError: malformed string

@David Berger 2009-06-12 18:46:49

makes me cry that we standardized on python 2.4

@Paulo Matos 2012-10-04 10:10:25

I should add that you need to sanitize the string for use with ast.literal_eval. (ensure quotes/double quotes in string are escaped)

@user1176501 2012-12-10 07:38:06

i get this error I am on python 2.6 (x86) on windows 7 x64 File "D:\Python26\lib\ast.py", line 48, in literal_eval node_or_string = parse(node_or_string, mode='eval') File "D:\Python26\lib\ast.py", line 36, in parse return compile(expr, filename, mode, PyCF_ONLY_AST) File "<unknown>", line 1 ^ SyntaxError: invalid syntax

@n611x007 2014-07-04 11:44:51

what about "dict(a=1)" style strings?

@shivshnkr 2017-01-04 14:45:18

This doesn't seem to work for enum value inside a dictionary. Eg: d = "{'col': <Colors.RED: 2>, 'val': 2}"

@Auros132 2018-01-07 16:23:31

why don't use json.dumps and json.loads insead, I found this solution more elevant thant using eval

@Trenton 2018-11-05 21:09:26

@n611x007 that's a statement, so it won't be allowed

@KDM 2018-11-17 16:37:56

I still don't see why it's better than eval(). Is it because it only accepts the structures given?

@KDM 2018-11-17 16:39:25

I have a naive question. Worried that I may have misunderstood "import". Responder says ast is built-in. Why do I need to import it?

@Walk 2017-02-09 11:05:16

You can try this.

    >>> import ast
    >>> data = "{'user': 'bob', 'age': 10, 'grades': ['A', 'F', 'C']}"
    >>> ast.literal_eval(data)

    O/P: {'age': 10, 'grades': ['A', 'F', 'C'], 'user': 'bob'}

    >>> user = ast.literal_eval(data)

    >>> user['age']
    O/P: 10

    >>> user['grades']
    O/P: ['A', 'F', 'C']

    >>> user['user']
    O/P: 'bob'

@CadentOrange 2017-02-09 11:24:16

This is similar to the accepted answer.

@lqhcpsgbl 2016-06-28 03:20:03

To OP's example:

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

We can use Yaml to deal with this kind of non-standard json in string:

>>> import yaml
>>> s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
>>> s
"{'muffin' : 'lolz', 'foo' : 'kitty'}"
>>> yaml.load(s)
{'muffin': 'lolz', 'foo': 'kitty'}

@Eric Marcos 2017-08-18 17:57:13

This will cause 'yes' and 'no' strings to be converted to True / False

@Mohammad Shahid Siddiqui 2017-11-19 18:28:59

It wont work if ':' is present, eg one value is URL

@ir0x539 2013-10-15 21:54:27

http://docs.python.org/2/library/json.html

JSON can solve this problem though its decoder wants double quotes around keys and values. If you don't mind a replace hack...

import json
s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
json_acceptable_string = s.replace("'", "\"")
d = json.loads(json_acceptable_string)
# d = {u'muffin': u'lolz', u'foo': u'kitty'}

NOTE that if you have single quotes as a part of your keys or values this will fail due to improper character replacement. This solution is only recommended if you have a strong aversion to the eval solution.

More about json single quote: jQuery single quote in JSON response

@Mark E. Haase 2015-05-27 04:00:51

{"foo": "b'ar"}

@ir0x539 2015-09-24 23:42:55

correct, thats why i put the NOTE

@Mark E. Haase 2015-09-25 16:52:04

{'foo': (1, 2, 3)}

@h8pathak 2017-02-01 05:13:10

I was looking for this solution. +1 for informing that the decoder wants double quotes around keys and values.

@Finn Årup Nielsen 2017-08-23 12:14:43

Another problem is for "{0: 'Hello'}".

@guival 2018-03-21 16:35:53

This also fails if you have trailing commas (not JSON compliant), eg: "{'muffin' : 'lolz', 'foo' : 'kitty',}"

@AuthorOfTheSurf 2018-05-22 19:31:50

Single-quoted strings, tuple literals, and trailing commas are not valid JSON. json.loads will only work on a valid JSON string. See the spec here: json.org Using json.loads is the safest solution, so use if possible. I would recommend transforming your input into valid JSON if necessary.

@Matias Gonzalez 2018-09-20 13:06:30

Also this solution does not work if you have unicode strings

@Ned Batchelder 2009-06-12 19:09:31

If you can't use Python 2.6, you can use a simple safeeval implmenentation like http://code.activestate.com/recipes/364469/

It piggybacks on the Python compiler so you don't have to do all the gross work yourself.

@Blixt 2009-06-12 18:30:12

If the string can always be trusted, you could use eval (or use literal_eval as suggested; it's safe no matter what the string is.) Otherwise you need a parser. A JSON parser (such as simplejson) would work if he only ever stores content that fits with the JSON scheme.

@Eli Courtwright 2009-06-12 18:37:09

Starting in 2.6, simplejson is included in the Python standard library as the json module.

@Blixt 2009-06-12 18:47:08

Ah, 2.6 seems to be awesome! I've been stuck with 2.5 and have been bad at checking out new features lately... I'll have to read up on all the other new stuff as well I guess =)

@Ben Hoyt 2009-06-14 21:30:16

Yeah, that's a good answer, but note that officially JSON doesn't support single-quoted strings, as given in the original poster's example.

@SIslam 2015-11-03 12:23:19

Super answer I believe- i will experiment!

Related Questions

Sponsored Content

16 Answered Questions

[SOLVED] Check if a given key already exists in a dictionary

  • 2009-10-21 19:05:09
  • Mohan Gulati
  • 2847950 View
  • 2683 Score
  • 16 Answer
  • Tags:   python dictionary

10 Answered Questions

[SOLVED] Iterating over dictionaries using 'for' loops

3 Answered Questions

34 Answered Questions

[SOLVED] How do I sort a dictionary by value?

10 Answered Questions

[SOLVED] Does Python have a string 'contains' substring method?

60 Answered Questions

[SOLVED] How do I read / convert an InputStream into a String in Java?

61 Answered Questions

[SOLVED] What is the difference between String and string in C#?

45 Answered Questions

[SOLVED] How do I convert a String to an int in Java?

15 Answered Questions

[SOLVED] Add new keys to a dictionary?

39 Answered Questions

[SOLVED] How to merge two dictionaries in a single expression?

Sponsored Content