Fix two NameErrors in the e2e-rag reference implementation - #2664
Open
David-Wu1119 wants to merge 1 commit into
Open
Fix two NameErrors in the e2e-rag reference implementation#2664David-Wu1119 wants to merge 1 commit into
David-Wu1119 wants to merge 1 commit into
Conversation
evaluation.py branched on an undefined `has_reranker` when printing reranked results, so verbose output raised NameError whenever reranking had run. Bind it once from the same condition the reranking step uses and share it, rather than repeating the attribute chain. reference_SUT_datasetup.py called log.warning() from the ImportError handler for BeautifulSoup, but `log` is only created after that try/except. A missing bs4 therefore raised NameError instead of the intended warning, so the graceful HAVE_HTML fallback never worked. Move the logging setup above the import attempt.
Contributor
|
MLCommons CLA bot: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two undefined names in the
e2e-ragreference implementation, both flagged byruff --select F821.1.
evaluation.py—has_rerankeris never defined. The reranking step tests availability inline, but the verbose-output branch further down uses a name that does not exist:So
retrieve_and_evaluate(..., verbose=True)raisesNameError: name 'has_reranker' is not definedwhenever reranking actually ran. Fixed by binding the availability check once and using it in both places, which also removes the duplicated attribute chain:2.
reference_SUT_datasetup.py—logis used before it exists. TheImportErrorhandler for BeautifulSoup logs a warning, butlogis only created after thetry/except:The point of that handler is to degrade gracefully via
HAVE_HTML, but on a machine withoutbs4the module fails to import at all withNameError: name 'log' is not defined— so the fallback it implements can never be reached. Fixed by moving the logger setup above the import attempt.Verification
ruff --select F821 e2e-rag/is clean afterwards, and both files parse. The second one is the more consequential of the two: it is the difference between "HTML processing disabled" and the reference SUT not importing.