Skip CEF fields that carry an empty numeric value - #26835
Open
fpetersen-gl wants to merge 3 commits into
Open
Conversation
…a stack trace for each message.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces log noise from Common Event Format (CEF) parsing by treating blank values in numerically-typed CEF extensions as “absent” rather than as conversion errors, while keeping malformed (non-empty) numeric values diagnosable without flooding logs.
Changes:
- Update numeric CEF converters to return
nullfor blank/whitespace values instead of throwing during parsing. - Update
MappedMessageto skip mapped fields whose converted value isnull, and to stop attaching stack traces to conversion warnings. - Add
MappedMessageTestcoverage for empty/malformed numeric values and for unchanged behavior of empty string + unmapped fields.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
graylog2-server/src/test/java/org/graylog/plugins/cef/parser/MappedMessageTest.java |
Adds tests asserting empty numeric values are skipped quietly, malformed values warn without throwable, and string/unmapped empty values remain retained. |
graylog2-server/src/main/java/org/graylog/plugins/cef/parser/MappedMessage.java |
Skips mapped fields when conversion returns null and adjusts logging to reduce noisy stack traces. |
graylog2-server/src/main/java/org/graylog/plugins/cef/parser/CEFMapping.java |
Makes numeric converters return null for blank inputs and annotates conversion methods as nullable. |
changelog/unreleased/issue-26833.toml |
Adds an unreleased changelog entry for the fix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
59
to
62
| } catch (Exception e) { | ||
| LOG.warn("Could not transform CEF field [{}] according to standard. Skipping.", keyName, e); | ||
| LOG.warn("Could not transform CEF field [{}] with value [{}] according to standard. Skipping.", | ||
| keyName, value); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Numeric CEF converters now return
nullfor a blank value instead of throwing.MappedMessageskips a field whose converted value is
nulland logs it at trace level.A value that is malformed rather than empty still logs a warning. That warning now names the
offending value and omits the stack trace, which was identical every time.
Empty string and unmapped fields are untouched. They still store an empty value, so existing
has_field()and_exists_searches behave as before.Motivation and Context
A SecureAuth appliance sends an empty
cfp1on nearly every message. Each one produced aNumberFormatExceptionwith a full stack trace atWARN:The field was already dropped, so only the logging was wrong. The CEF input and
parse_cef()both construct
MappedMessage, so moving CEF parsing into a pipeline was no workaround. Theonly remedy was silencing
org.graylog.plugins.cef, which also hid genuine CEF errors.This affects every numerically typed CEF field, not just
cfp1.MappedMessagehas had nofunctional change since the plugin was merged into core, so all 7.0.x and 7.1.x releases behave
the same way. Reported on 7.0.5.
Fixes #26833.
How Has This Been Tested?
New
MappedMessageTestcovers all five numeric converters, the malformed-value warning, and theunchanged handling of empty string, unmapped, and labelled fields.
Reverting the two production files makes three of the new tests fail, which confirms they pin the
change rather than the behaviour that was already correct:
emptyNumericValueIsNotLoggedAsAWarningemptyValueIsSkippedForEveryNumericTypemalformedNumericValueIsLoggedWithoutAStackTraceTypes of changes
Checklist: