[qualys_vmdr] request 4Gi of memory for agentless deployments - #20579
Open
igcybers wants to merge 1 commit into
Open
[qualys_vmdr] request 4Gi of memory for agentless deployments#20579igcybers wants to merge 1 commit into
igcybers wants to merge 1 commit into
Conversation
igcybers
force-pushed
the
qualys_vmdr-agentless-memory
branch
from
August 6, 2026 16:22
cf2a6f4 to
cc07a64
Compare
The asset_host_detection input previously inherited the default agentless memory allocation, which is not sufficient for the default batch size of 1000 hosts. At publish time the CEL program holds the full host batch, the knowledge base map accumulated across every paginated request for that batch, and the flattened event array in memory simultaneously. Building the events copies the entire host record once per detection, so peak usage scales with hosts x detections-per-host rather than with the batch size alone. Request 4Gi to match the other vulnerability management and security packages owned by this team (rapid7_insightvm, m365_defender, microsoft_defender_endpoint, ti_crowdstrike).
igcybers
force-pushed
the
qualys_vmdr-agentless-memory
branch
from
August 6, 2026 16:28
cc07a64 to
56cef1c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message:
Why 4Gi
asset_host_detectionaccumulates several large structures in CEL state at the same time before it can publish. In[
input.yml.hbs] https://github.com/elastic/integrations/blob/main/packages/qualys_vmdr/data_stream/asset_host_detection/agent/stream/input.yml.hbs#L206-L223):state.worklist.AHD_LISTholds the full host batch -truncation_limitis set frombatch_size, which defaults to 1000. In practice ~700 batch size is the limit with the current memory setup.state.worklist.KB_MAPgrows monotonically across every paginated/api/4.0/fo/knowledge_base/vuln/request for that batch (host_qid_list_query_limit, default 500 QIDs per request).AHD_LIST.map(h, h.DETECTION_LIST.DETECTION.map(v, h.with({...}))).flatten().That last step is the amplifier:
h.with({...})copies the entire host record once per detection and inlines its knowledge base entry, so peak usage scales with hosts × detections-per-host, not withbatch_sizealone and it happens while both source structures are still live.This matches the four packages already owned by
security-service-integrationsthat request 4Gi:rapid7_insightvm(same vulnerability-management and CDR workload shape),m365_defender,microsoft_defender_endpointandti_crowdstrike.Checklist
changelog.ymlfile.How to test this PR locally
This change only adds the agentless resource request to the package manifest; there is no ingest, mapping or dashboard change, so the existing tests are unaffected.
To validate the manifest against the spec: