HTTPS downloads fail when system trusted CA store contains an additional trusted root CA for a network proxy

[cxiao]

Version and Platform (required):

• Binary Ninja Version: 5.2.8614-stable
• Edition: Commercial
• OS: Fedora Linux
• OS Version: Fedora 43
• CPU Architecture: x86_64
• Python version: 3.12

Bug Description:
On a machine where I have Binary Ninja installed, all HTTPS traffic goes through a network proxy. I have an additional trusted root CA certificate installed in the system certificate store for this proxy. I noticed that starting with Binary Ninja 5.2, fetches of network resources over HTTPS, such as the update check, fail.

The specific error I see in the log is the following:

[Default] Failed to get update info: hyper_util::client::legacy::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: InvalidCertificate(UnknownIssuer) } })

It seems that the default CoreDownloadProvider may be ignoring the system trusted CA store.

I am on Fedora, but colleagues using Ubuntu and Windows who also have proxied traffic / an additional trusted root CA certificate installed, have also observed this issue, starting with Binary Ninja 5.2.

Steps To Reproduce:

1. Have a system where all HTTPS traffic goes through a network proxy, with an additional trusted root CA for this proxy installed on the system.
2. Launch Binary Ninja, with the Network > Automatically Check Updates (network.enableUpdates) setting set to true.
3. Observe the log message

[Default] Failed to get update info: hyper_util::client::legacy::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: InvalidCertificate(UnknownIssuer) } })

Alternatively:

1. Have a system where all HTTPS traffic goes through a network proxy, with an additional trusted root CA for this proxy installed on the system.
2. Invoke the "Update Channel..." command.
3. Observe that the dialog box never populates with the list of available update channels.
4. Observe the log message

[Default] Error fetching channel list: hyper_util::client::legacy::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: InvalidCertificate(UnknownIssuer) } }) (Details...)

Expected Behavior:

I expect that Binary Ninja, when installed on a system where HTTPS traffic goes through a network proxy, and where the CA certificate for that proxy is installed in the system certificate store, should still be able to download network resources via HTTPS.

Workaround:

This workaround works:

1. Change the value of the Network > Download Provider setting (network.downloadProviderName) from CoreDownloadProvider to PythonDownloadProvider.
2. Set the environment variable REQUESTS_CA_BUNDLE to /etc/ssl/certs/ca-certificates.crt, to have the instance of the requests library used by the Binary Ninja Python environment use the system trusted CA store (relevant requests documentation)

Additional Information:

Some things which may be related:

• I noticed there is already a issue for documenting how Binja interacts with network proxies: Network proxies #5802

• Commit ea914ef, for the WARP component, may be relevant to this problem: the commit message mentions

Unfortunately we cannot use reqwest because enterprise servers provide invalid certificates, so we must use the enterprise download provider to bypass certain certificate validations

• It is entirely possible that the CoreDownloadProvider is using my system's trusted CA store, but that reqwest is very strict about cerftificate validation, and that this particular CA certificate fails the validation.

• I am not sure if the CoreDownloadProvider uses rustls under the hood, and if it does, if it also uses https://github.com/rustls/rustls-native-certs to get the system certificate store. However, I did try the instructions in the rustls-native-certs repository to try setting the SSL_CERT_FILE environment variable (in my case to /etc/ssl/certs/ca-certificates.crt). This did not solve the issue.

[emesare]

Sent a follow up on slack for user testing of build 8631, once the cert is known to validate the branch with the fix will be merged and included in 5.2 R2.

[cxiao]

Thanks very much for the quick followup @emesare! I've tested build 8631, and I'm now getting successful HTTPS connections to master.binary.ninja & cdn.binary.ninja on my system, while using CoreDownloadProvider, with all environment variables mentioned above unset. I no longer have certificate validation issues.

[cxiao]

For reference for anyone looking at this issue: The fix is now in dev versions of Binja on versions 5.3.8636 and later.