By TedBurrows


2012-05-19 18:45:20 8 Comments

I'm working on a simple script that involves CAS, jspring security check, redirection, etc. I would like to use Kenneth Reitz's python requests because it's a great piece of work! However, CAS requires getting validated via SSL so I have to get past that step first. I don't know what Python requests is wanting? Where is this SSL certificate supposed to reside?

Traceback (most recent call last):
  File "./test.py", line 24, in <module>
  response = requests.get(url1, headers=headers)
  File "build/bdist.linux-x86_64/egg/requests/api.py", line 52, in get
  File "build/bdist.linux-x86_64/egg/requests/api.py", line 40, in request
  File "build/bdist.linux-x86_64/egg/requests/sessions.py", line 209, in request 
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 624, in send
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 300, in _build_response
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 611, in send
requests.exceptions.SSLError: [Errno 1] _ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

19 comments

@xmedeko 2017-08-29 06:06:20

If the request calls are buried somewhere deep in the code and you do not want to install the server certificate, then, just for debug purposes only, it's possible to monkeypatch requests:

import requests.api
import warnings


def requestspatch(method, url, **kwargs):
    kwargs['verify'] = False
    return _origcall(method, url, **kwargs)

_origcall = requests.api.request
requests.api.request = requestspatch
warnings.warn('Patched requests: SSL verification disabled!')

Never use in production!

@Rafael Almeida 2012-10-12 18:19:07

The problem you are having is caused by an untrusted SSL certificate.

Like @dirk mentioned in a previous comment, the quickest fix is setting verify=False:

requests.get('https://example.com', verify=False)

Please note that this will cause the certificate not to be verified. This will expose your application to security risks, such as man-in-the-middle attacks.

Of course, apply judgment. As mentioned in the comments, this may be acceptable for quick/throwaway applications/scripts, but really should not go to production software.

If just skipping the certificate check is not acceptable in your particular context, consider the following options, your best option is to set the verify parameter to a string that is the path of the .pem file of the certificate (which you should obtain by some sort of secure means).

So, as of version 2.0, the verify parameter accepts the following values, with their respective semantics:

  • True: causes the certificate to validated against the library's own trusted certificate authorities (Note: you can see which Root Certificates Requests uses via the Certifi library, a trust database of RCs extracted from Requests: Certifi - Trust Database for Humans).
  • False: bypasses certificate validation completely.
  • Path to a CA_BUNDLE file for Requests to use to validate the certificates.

Source: Requests - SSL Cert Verification

Also take a look at the cert parameter on the same link.

@diyism 2013-02-15 10:33:16

Yes, when i used dotCloud in ubuntu, the same "certificate verify failed" came out. After modified "requests.session(headers=headers, hooks=hooks, verify=False)" in "/usr/local/lib/python2.6/dist-packages/dotcloud/client/clie‌​nt.py", it worked.

@khalid13 2013-02-25 01:13:59

This isn't marked as correct, but I can verify that it works (as opposed to the answers below).

@jfs 2013-09-12 08:38:43

@khalid13: An axe "works" as a headache medicine (no head - no headache). It doesn't mean that it is a good idea to use it that way. verify=False disables host's SSL certificate checking.

@khalid13 2013-09-16 19:27:35

@J.F.Sebastian Honestly, it depends on what you're doing. For my quick/throwaway application, it was more than sufficient.

@Rafael Almeida 2014-05-02 16:51:26

@J.F.Sebastian, edited the answer to include this warning!

@binki 2015-09-29 16:51:07

@diyism making such a change sounds very unsafe…

@Rafael Almeida 2015-12-18 13:35:23

@diyism I agree completely with binki. What was the parameter before? True or a path?

@dave_g_ 2017-03-21 13:29:24

Oddly enough, I needed this on Python 2.7.6 but not 2.7.12.

@Bagelstein 2018-04-12 19:10:14

Where do you set the verify flag = false? All these comments saying to set it to false, none saying how....

@Rafael Almeida 2018-04-12 23:22:26

It’s a keyword argument to the get() function: requests.get('https://example.com', verify=False)

@Odle098 2018-10-12 08:01:08

saved my day, thank you!

@Paul 2018-03-07 23:22:18

I had to upgrade from Python 3.4.0 to 3.4.6

pyenv virtualenv 3.4.6 myvenv
pyenv activate myvenv
pip install -r requirements.txt

@alanjds 2016-09-19 18:57:31

$ pip install -U requests[security]

  • Tested on Python 2.7.6 @ Ubuntu 14.04.4 LTS
  • Tested on Python 2.7.5 @ MacOSX 10.9.5 (Mavericks)

When this question was opened (2012-05) the Requests version was 0.13.1. On version 2.4.1 (2014-09) the "security" extras were introduced, using certifi package if available.

Right now (2016-09) the main version is 2.11.1, that works good without verify=False. No need to use requests.get(url, verify=False), if installed with requests[security] extras.

@Aamir Abro 2016-12-04 22:13:07

fixed by pip install -U requests[security] --no-cache two times and pip install certifi==2015.04.28

@Howiecamp 2017-03-12 22:15:28

@alanjds What if I want to either configure python to trust some ssl cert or to disable certificate verification but globally in the environment, without editing the source code? For example if I download existing Python utilities (eg the AWS CLI) and I want to trust certs or ignore certificate validation for those tools?

@alanjds 2017-03-13 17:36:24

@Howiecamp then you can go via j-f-sebastian answer, I guess: stackoverflow.com/a/12865159/798575

@Howiecamp 2017-03-13 20:54:01

@alanjds But doesn't his answer assume I'm writing the code and/or have access to the code? I'm looking to implement this at the environment level.

@Vincent Claes 2018-01-05 09:56:15

do pip install --upgrade pip before installing the requests security package to avoid other errors

@AniketGole 2015-11-03 07:42:57

If you want to remove the warnings, use the code below.

import urllib3

urllib3.disable_warnings()

and verify=False with request.get or post method

@michael 2017-07-20 20:10:35

After hours of debugging I could only get this to work using the following packages:

requests[security]==2.7.0  # not 2.18.1
cryptography==1.9  # not 2.0

using OpenSSL 1.0.2g 1 Mar 2016

Without these packages verify=False was not working.

I hope this helps someone.

@Tim Ludwinski 2017-07-11 13:26:59

I was having a similar or the same certification validation problem. I read that OpenSSL versions less than 1.0.2, which requests depends upon sometimes have trouble validating strong certificates (see here). CentOS 7 seems to use 1.0.1e which seems to have the problem.

I wasn't sure how to get around this problem on CentOS, so I decided to allow weaker 1024bit CA certificates.

import certifi # This should be already installed as a dependency of 'requests'
requests.get("https://example.com", verify=certifi.old_where())

@Michael 2017-06-14 11:07:43

As mentioned by @Rafael Almeida, the problem you are having is caused by an untrusted SSL certificate. In my case, the SSL certificate was untrusted by my server. To get around this without compromising security, I downloaded the certificate, and installed it on the server (by simply double clicking on the .crt file and then Install Certificate...).

@Peter 2017-01-15 20:10:47

There is currently an issue in the requests module causing this error, present in v2.6.2 to v2.12.4 (ATOW): https://github.com/kennethreitz/requests/issues/2573

Workaround for this issue is adding the following line: requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS'

@Tamás Szelei 2017-03-06 15:48:21

FWIW, it's still present with requests==2.13.0. The above workaround fixes it still.

@chk 2013-04-18 14:29:46

I have found an specific approach for solving a similar issue. The idea is pointing the cacert file stored at the system and used by another ssl based applications.

In Debian (I'm not sure if same in other distributions) the certificate files (.pem) are stored at /etc/ssl/certs/ So, this is the code that work for me:

import requests
verify='/etc/ssl/certs/cacert.org.pem'
response = requests.get('https://lists.cacert.org', verify=verify)

For guessing what pem file choose, I have browse to the url and check which Certificate Authority (CA) has generated the certificate.

EDIT: if you cannot edit the code (because you are running a third app) you can try to add the pem certificate directly into /usr/local/lib/python2.7/dist-packages/requests/cacert.pem (e.g. copying it to the end of the file).

@chk 2013-04-18 14:35:22

Related post for debugging CA_BUNDLE used by python.

@CMCDragonkai 2015-12-11 06:37:13

What about replacing /usr/local/lib/python2.7/dist-packages/requests/cacert.pem with a symlink to the OS store?

@Yong 2015-11-15 07:47:16

I encountered the same issue and ssl certificate verify failed issue when using aws boto3, by review boto3 code, I found the REQUESTS_CA_BUNDLE is not set, so I fixed the both issue by setting it manually:

from boto3.session import Session
import os

# debian
os.environ['REQUESTS_CA_BUNDLE'] = os.path.join(
    '/etc/ssl/certs/',
    'ca-certificates.crt')
# centos
#   'ca-bundle.crt')

For aws-cli, I guess setting REQUESTS_CA_BUNDLE in ~/.bashrc will fix this issue (not tested because my aws-cli works without it).

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt # ca-bundle.crt
export REQUESTS_CA_BUNDLE

@mallyvai 2016-10-19 07:01:51

This fixed my problem! I was using Charles Proxy on Mac to debug a library that made JSON calls to HTTPS APIs. I installed the Charless certificate as specified, added it to the keychain, but Python kept failing with: SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) To fix this, I ended up following your advice about adding REQUESTS_CA_BUNDLE and exporting the Charles certificate from my keychain as a .pem file. Now, it works!

@user565447 2016-11-02 09:15:11

Thanks, the same issue was with opened Fiddler

@Howiecamp 2017-03-12 22:17:20

@user565447 I'm trying to get this working with Fiddler right now. Should setting REQUESTS_CA_BUNDLE to Fiddler's cert work?

@user565447 2017-03-14 09:28:09

Yes , it should work. At least for me it worked.

@ajon 2016-05-06 21:13:57

I fought this problem for HOURS.

I tried to update requests. Then I updated certifi. I pointed verify to certifi.where() (The code does this by default anyways). Nothing worked.

Finally I updated my version of python to python 2.7.11. I was on Python 2.7.5 which had some incompatibilities with the way that the certificates are verified. Once I updated Python (and a handful of other dependencies) it started working.

@Tim Ludwinski 2017-07-11 13:31:20

If you updated OpenSSL to a version > 1.0.1, than that was probably the issue. See my answer bellow. stackoverflow.com/a/44543047/1413201

@crazystick 2017-08-09 16:20:59

Going from Python 2.7.9 to 2.7.10 fixed this for me.

@tintin 2016-03-04 08:38:52

In case you have a library that relies on requests and you cannot modify the verify path (like with pyvmomi) then you'll have to find the cacert.pem bundled with requests and append your CA there. Here's a generic approach to find the cacert.pem location:

windows

C:\>python -c "import requests; print requests.certs.where()"
c:\Python27\lib\site-packages\requests-2.8.1-py2.7.egg\requests\cacert.pem

linux

#  (py2.7.5,requests 2.7.0, verify not enforced)
[email protected]:~/# python -c "import requests; print requests.certs.where()"
/usr/lib/python2.7/dist-packages/certifi/cacert.pem

#  (py2.7.10, verify enforced)
[email protected]:~/# python -c "import requests; print requests.certs.where()"
/usr/local/lib/python2.7/dist-packages/requests/cacert.pem

btw. @requests-devs, bundling your own cacerts with request is really, really annoying... especially the fact that you do not seem to use the system ca store first and this is not documented anywhere.

update

in situations, where you're using a library and have no control over the ca-bundle location you could also explicitly set the ca-bundle location to be your host-wide ca-bundle:

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt python -c "import requests; requests.get('https://somesite.com';)"

@user941581 2016-02-16 04:53:27

I face the same problem using gspread and these commands works for me:

sudo pip uninstall -y certifi
sudo pip install certifi==2015.04.28

@alex-mcleod 2016-02-19 18:54:13

This did it for me. Thanks :)

@dragon788 2016-07-11 19:59:57

This has the downside of reinstalling potentially revoked/untrusted certificates from the older version of certifi, NOT recommended.

@seans 2016-08-29 19:34:18

if for some reason you're forced to stick with an early version of python 2.7, downgrading certifi is the only approach that worked for me

@Boud 2012-05-19 19:20:07

From requests documentation on SSL verification:

Requests can verify SSL certificates for HTTPS requests, just like a web browser. To check a host’s SSL certificate, you can use the verify argument:

>>> requests.get('https://kennethreitz.com', verify=True)

If you don't want to verify your SSL certificate, make verify=False

@TedBurrows 2012-05-19 22:27:39

Well, I added the verify=True, but still received the exact same error. No change. Something else must be required, but don't know what it could be.

@TedBurrows 2012-05-19 23:45:33

I suppose I have now descended into SSL madness. I added this to my initial get...get(url1, headers=headers, cert='/etc/pki/tls/cert.pem', verify=True, config=my_config) So, now I am getting this error. requests.exceptions.SSLError: [Errno 336265225] _ssl.c:351: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib I have not a clue what this means.

@TedBurrows 2012-05-20 03:38:11

Reading thru tons of documents I have learned I don't need the verify=True. That is for validating the certificate from the server. I don't need to do that because the server and the CAS server do that together. So basically, when you do a 'get' to the primary server, you get redirected to the CAS using https (which is ssl). So basically, the verify=True doesn't apply in this case. Still doesn't help me but maybe will help someone else who reads this.

@dirk 2012-08-03 17:38:18

Just set verify=False if you don't want to validate the certificate, iow if you have a self signed certificate

@Matthias Urlichs 2013-10-09 06:30:20

If you have a self-signed certificate, then download it and set verify to its filename. There is no excuse whatsoever for setting verify=False. verify='/path/to/cert.pem'

@Louis Cremen 2015-06-29 03:39:15

Sorry Boud, I needed to down vote this answer as requests does not handle HTTPS requests "like a web browser". If the full SSL chain of trust (including intermediate certificates) is not declared on a server and requires an extra certificate download, you will receive the above SSL verification error. Web browsers will do the extra download and not flag any certificate errors. This is one way that a web browser and Requests differ. There are others. Requests does some verification, but it is not as good as a browser.

@yogesh prasad 2015-05-21 12:01:25

If you don't bother about certificate just use verify=False.

import requests

url = "Write your url here"

returnResponse = requests.get(url, verify=False)

@rhoerbe 2015-08-03 17:29:21

It is not feasible to add options if requests is being called from another package. In that case adding certificates to the cacert bundle is the straight path, e.g. I had to add "StartCom Class 1 Primary Intermediate Server CA", for which I downloaded the root cert into StartComClass1.pem. given my virtualenv is named caldav, I added the certificate with:

cat StartComClass1.pem >> .virtualenvs/caldav/lib/python2.7/site-packages/pip/_vendor/requests/cacert.pem
cat temp/StartComClass1.pem >> .virtualenvs/caldav/lib/python2.7/site-packages/requests/cacert.pem

one of those might be enough, I did not check

@Marius Craciunoiu 2014-06-13 19:19:33

I ran into the same issue. Turns out I hadn't installed the intermediate certificate on my server (just append it to the bottom of your certificate as seen below).

https://www.digicert.com/ssl-support/pem-ssl-creation.htm

Make sure you have the ca-certificates package installed:

sudo apt-get install ca-certificates

Updating the time may also resolve this:

sudo apt-get install ntpdate
sudo ntpdate -u ntp.ubuntu.com

If you're using a self-signed certificate, you'll probably have to add it to your system manually.

@Kenneth Reitz 2016-02-03 00:37:26

Note, this only applies to Requests installations via apt-get, which is modified by Debian/Ubuntu to use system certs. Requests proper ships with its own, carefully curated, CA bundle: certifi.io

@timmy 2017-03-13 22:34:16

Shouldn't the root CA be enough? Why do you need the intermediates?

@jfs 2012-10-12 18:38:46

The name of CA file to use you could pass via verify:

cafile = 'cacert.pem' # http://curl.haxx.se/ca/cacert.pem
r = requests.get(url, verify=cafile)

If you use verify=True then requests uses its own CA set that might not have CA that signed your server certificate.

@jfs 2013-09-12 08:23:51

@9emE0iL18gxCqLT: why do you think that all systems use the path you provided? requests can be packaged for your distribution. Run python -mrequests.certs to find out where it points to.

@CMCDragonkai 2015-12-11 06:36:05

If the Python request's cacert bundle is out of date, how do I update it?

@Kenneth Reitz 2016-02-03 00:35:46

You shouldn't use that cacert.pem from curl. It contains many revoked certs. Check out Certifi (which Requests uses): certifi.io

@jfs 2016-04-20 07:54:06

@KennethReitz: 1- what Requests uses fails for OP (otherwise there weren't be the question) 2- cacert.pem is CA certificates extracted from Mozilla (by cURL) -- it is just an example (if the CA list used by a popular web-browser can't be used as an example then I don't know what can be) -- the point of the answer that you can pass your own CA file if the default list fails.

Related Questions

Sponsored Content

53 Answered Questions

[SOLVED] Calling an external command in Python

28 Answered Questions

[SOLVED] Finding the index of an item given a list containing it in Python

  • 2008-10-07 01:39:38
  • Eugene M
  • 2854027 View
  • 2394 Score
  • 28 Answer
  • Tags:   python list

25 Answered Questions

[SOLVED] How can I safely create a nested directory in Python?

23 Answered Questions

[SOLVED] What is the difference between @staticmethod and @classmethod?

23 Answered Questions

24 Answered Questions

[SOLVED] Difference between append vs. extend list methods in Python

15 Answered Questions

[SOLVED] What are metaclasses in Python?

31 Answered Questions

[SOLVED] How to get the current time in Python

  • 2009-01-06 04:54:23
  • user46646
  • 2338849 View
  • 2085 Score
  • 31 Answer
  • Tags:   python datetime time

14 Answered Questions

[SOLVED] Does Python have a string 'contains' substring method?

5 Answered Questions

[SOLVED] Manually raising (throwing) an exception in Python

Sponsored Content