Python request.post Not Accepting Dictionary

Question

here is the data that I want to post,

this is the printed version of the dictionary answers:

{'Emails': set([u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected] ', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected] ', u' [email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u"elberta.o'[email protected]", u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u' [email protected]', u'[email protected]', u'angelica.klocko@ankunding', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', None, u'[email protected] ', u'[email protected]']), '# of Emails per Domain': {u'@hotmail.com': 22, u'@yahoo.com': 18, u'@gmail.com': 18}, 'April Users': 70}

when I run:

 r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data = answers)

I get this error:

{"message": "Could not parse request body into json: Unrecognized token \'Emails\': was expecting (\'true\', \'false\' or \'null\')\n at [Source: [B@311a7237; line: 1, column: 8]"}

Not sure what this means at all, I've tried to parse the data into json, why doesn't it like the 'Email' string?

If I add the json.dump code I get:

TypeError

TypeError: set([u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected] ', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected] ', u' [email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u"elberta.o'[email protected]", u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u' [email protected]', u'[email protected]', u'angelica.klocko@ankunding', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', u'[email protected]', None, u'[email protected] ', u'[email protected]']) is not JSON serializable

JSON doesn't know `set`. Also, you should definitely not post these email adresses here. — L3viathan, Feb 04 '16 at 21:56
Ok interesting, what do you mean by that? Sorry I'm very new to Python — Adam Weitzman, Feb 04 '16 at 22:15
Sorry, I was wrong, but I resolved it now: It works if you encode the JSON yourself and hand `requests.post` a JSON-encoded string as the `data` parameter. — L3viathan, Feb 04 '16 at 22:17
You appear to have posted a number of emails in this question, which is often accidental. If this is the case, please edit your question so it doesn't contain any sensitive information and then flag it so we can get the revision history cleaned out. — josliber, Feb 07 '16 at 23:37

score 10 · Answer 1 · edited May 23 '17 at 10:29

This works:

import requests
import json
answers = {} #...
url = "..."
answers['Emails'] = list(answers['Emails'])
r = requests.post(url, data=json.dumps(answers))

The reason the other way doesn't work is that by default requests sends data with the Content-Type application/x-www-form-urlencoded instead of JSON.

To change that, you can also use json instead of the data parameter:

r = requests.post(url, json=answers)

To debug code using the requests library, you can look at a prepared request before sending it.

score 4 · Accepted Answer · edited May 23 '17 at 10:29

4

TypeError: set([ ... ]) is not JSON serializable

Sets are not JSON serializable. You can either convert all of the sets to lists in the dictionary:

answers = {key: list(value) if isinstance(value, set) else value
           for key, value in answers.items()}

^{(Note that this would not work for nested dictionaries as is).}

Or, use a custom JSON encoder that would handle the set serialization.

Dumping and loading it back with json usually helps in debugging:

import json

answers = json.loads(json.dumps(answers))
r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data=answers)

edited May 23 '17 at 10:29

Community

1
1

answered Feb 04 '16 at 21:57

alecxe

462,703
120
1,088
1,195

good suggestion, I tried that method but I'm getting a TypeError – Adam Weitzman Feb 04 '16 at 22:03
@AdamWeitzman please post the traceback, this might give us a clue what is going on. Thanks! – alecxe Feb 04 '16 at 22:05
@AdamWeitzman yup, exactly, updated the answer. Hope that helps. – alecxe Feb 04 '16 at 22:11
hmmm tried to do it manually with no luck, i'll try the JSON encoding – Adam Weitzman Feb 04 '16 at 22:14
@AdamWeitzman yeah, cause you have other sets in the dictionary, updated the first option. Hope that helps. – alecxe Feb 04 '16 at 22:23

Python request.post Not Accepting Dictionary

2 Answers2