Please use certifi library

[dupuy]

When I use the Vale Python package I am getting the unfortunate error:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

Fixing this requires finding the OS or Python install certificate bundle and updating it, which is hard for less technical users. If the vale self installer used the certifi package instead, the fix is as simple and platform independent as

pip install -U certifi

The code chane is small too:

>>> import certifi
>>> certifi_context = ssl.create_default_context(cafile=certifi.where())
>>> urlopen("...", context=certifi_context)

[daniperez]

Hi @dupuy ! Thanks for the advise 👍

Can you provide more context as to why that error happens? Unfortunately (or not!) it doesn't happen in my testing environment. I would like to understand why that happens and why an alternative certificate store is needed. I'd like to avoid fixing this for some people but then potentially breaking to other people.

Please bear with me, from my questions you can see how much I ignore about the subject.

[daniperez]

Answering myself:

If you have installed Python 3.6 on OSX and are getting the "SSL: CERTIFICATE_VERIFY_FAILED" error when trying to connect to an https:// site, it's probably because Python 3.6 on OSX has no certificates at all, ... (Stackoverflow)

Is it something like that?

[dupuy]

I'm having this problem using Python 3.10 on macOS Ventura; the more unusual thing is that I was using pipx to install vale – but i have the issue even when I use pip to install it:

python3 -m pip install --upgrade --user pip
Requirement already satisfied: pip in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (24.0)
python3 -m pip install --upgrade --user certifi pipx
Requirement already satisfied: certifi in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (2024.2.2)
Collecting pipx
  Using cached pipx-1.4.3-py3-none-any.whl.metadata (17 kB)
Requirement already satisfied: argcomplete>=1.9.4 in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (from pipx) (3.2.2)
Requirement already satisfied: packaging>=20 in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (from pipx) (23.2)
Requirement already satisfied: platformdirs>=2.1 in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (from pipx) (4.2.0)
Requirement already satisfied: tomli in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (from pipx) (2.0.1)
Requirement already satisfied: userpath!=1.9.0,>=1.6 in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (from pipx) (1.9.1)
Requirement already satisfied: click in /Users/alexdupuy/Library/Python/3.10/lib/python/site-packages (from userpath!=1.9.0,>=1.6->pipx) (8.1.7)
Using cached pipx-1.4.3-py3-none-any.whl (65 kB)
Installing collected packages: pipx
Successfully installed pipx-1.4.3
python3 -m pipx install --force pipx
Installing to existing venv 'pipx'
⚠️  Note: pipx was already on your PATH at
    /Users/alexdupuy/Library/Python/3.10/bin/pipx
  installed package pipx 1.4.3, installed using Python 3.10.11
  These apps are now globally available
    - pipx
done! ✨ 🌟 ✨
#yes | python3 -m pip uninstall pipx
/Users/alexdupuy/.local/bin/pipx ensurepath
/Users/alexdupuy/.local/bin is already in PATH.

⚠️  All pipx binary directories have been added to PATH. If you are sure you
want to proceed, try again with the '--force' flag.

Otherwise pipx is ready to go! ✨ 🌟 ✨
pipx install --force vale
Installing to existing venv 'vale'
  installed package vale 3.0.7.0, installed using Python 3.10.11
  These apps are now globally available
    - vale
done! ✨ 🌟 ✨
* vale not found. Downloading it...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1283, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1329, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 976, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1455, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1071, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1342, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alexdupuy/.local/bin/vale", line 10, in <module>
    sys.exit(main())
  File "/Users/alexdupuy/Library/Application Support/pipx/venvs/vale/lib/python3.10/site-packages/vale/main.py", line 158, in main
    vale_bin_path = download_vale_if_missing()
  File "/Users/alexdupuy/Library/Application Support/pipx/venvs/vale/lib/python3.10/site-packages/vale/main.py", line 119, in download_vale_if_missing
    url = urlopen(url_str)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)>
uninstalled vale! ✨ 🌟 ✨

And similarly with pip.

% pip3 install vale
Collecting vale
  Using cached vale-3.0.7.0-py3-none-any.whl.metadata (3.0 kB)
Using cached vale-3.0.7.0-py3-none-any.whl (5.7 kB)
Installing collected packages: vale
Successfully installed vale-3.0.7.0
alexdupuy@Cuchillos reliabot % vale sync
* vale not found. Downloading it...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1283, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1329, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 976, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1455, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1071, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1342, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/vale", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/vale/main.py", line 158, in main
    vale_bin_path = download_vale_if_missing()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/vale/main.py", line 119, in download_vale_if_missing
    url = urlopen(url_str)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)>:

[dupuy]

Sorry I didn't mean to close the issue, that's what happens when I am commenting right before I go to sleep...

I'm not really sure what the problem is here, I don't have issues with certificates for other programs. I get the problem with all the package-install versions of Python installed on my system. I also have brew-installed Python, but pip won't install into them:

% pip3.12 install vale
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-brew-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.
    
    If you wish to install a non-brew packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I tried brew-installed backrev Python 3.11, for some reason that one does work! But I don't know why.

[daniperez]

Hi @dupuy ! Finally I can come back to you. Let me know what do you think. Please, bear with me, this is not a subject I'm familiar with.

If I implement what you mention in your first comment, I will be circumventing your OS ability to update the certificates and turn vale into some kind of attack vector (or so I read googling the issue). In this case, my guess is that the attack surface is quite limited, but I'm not a security expert and I guess a lot of security issues originate in otherwise small attack vectors.

I could update the documentation to let users know about this problem and how to fix it.

Out of curiosity: did you try this with your brew's python? (from you last comment)

If you wish to install a non-brew-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip.

🍏 I would like to know what other Mac users are experiencing, because I don't own a Mac and it seems some users can still use this (but that I don't know!).