Fix error queue leak in _detect_private_key on OpenSSL 3.x#205
Draft
toddr-bot wants to merge 1 commit into
Draft
Fix error queue leak in _detect_private_key on OpenSSL 3.x#205toddr-bot wants to merge 1 commit into
toddr-bot wants to merge 1 commit into
Conversation
_detect_private_key() calls EVP_PKEY_get_bn_param() to check for the private exponent d. On public keys this fails and pushes errors onto the OpenSSL error queue, but never clears them. These stale errors can leak into subsequent croakSsl() calls from unrelated operations, potentially producing misleading error messages. Add ERR_clear_error() after the check to keep the queue clean. Also fix _new_key_from_parameters() to pass EVP_PKEY_PUBLIC_KEY (not EVP_PKEY_KEYPAIR) when constructing a public-only key from n and e without d. This matches the actual key type and avoids unnecessary error queue pollution from EVP_PKEY_fromdata.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Clear stale OpenSSL errors after public key detection on 3.x, and use the correct
EVP_PKEY_PUBLIC_KEYselection for public-only keys in_new_key_from_parameters.Why
_detect_private_key()callsEVP_PKEY_get_bn_param()to probe for the private exponentd. On public keys this fails and pushes errors onto the OpenSSL error queue — but never clears them. These stale errors can leak into the nextcroakSsl()call from an unrelated operation, producing a misleading error message.Separately,
_new_key_from_parameters()passesEVP_PKEY_KEYPAIRtoEVP_PKEY_fromdata()even when onlynandeare provided (public-only key). The correct selection isEVP_PKEY_PUBLIC_KEY, which matches the actual key type and avoids unnecessary error queue pollution.The
_get_key_parameters()function already has anERR_clear_error()for exactly this pattern (line 1172) — this fix applies the same discipline to_detect_private_key().How
ERR_clear_error()after theEVP_PKEY_get_bn_paramcall in_detect_private_key()on both the found and not-found paths.EVP_PKEY_PUBLIC_KEYwhendis NULL in the else branch of_new_key_from_parameters().Testing
All 670 tests pass on OpenSSL 3.5.5. Added 3 tests to
error_queue.tcovering public key construction from parameters followed by operations that exercise error reporting.Quality Report
Changes: 2 files changed, 28 insertions(+), 2 deletions(-)
Code scan: clean
Tests: passed (OK)
Branch hygiene: clean
Generated by Kōan