Skip to content

Fix OpenSearch readiness for alternative field mappings #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
afoote-mitre merged 1 commit into from
Aug 24, 2026
Merged

Fix OpenSearch readiness for alternative field mappings #139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ The Search API will be served at `localhost:3000`.

`GET /api/health-check` is a process liveness check. It returns HTTP 200 when the Node.js process is running and deliberately does not contact OpenSearch. Use it where a dependency outage should not restart otherwise healthy API tasks.

`GET /api/ready-check` first verifies that the configured CVE index exposes the field capabilities required by the `/search` contract, then sends a minimal zero-result search. This checks required mappings, connectivity, index availability, and the same search permission used by the API without retrieving CVE records or counting all matches. It returns HTTP 200 when search requests can be served. Configuration errors, connection failures, timeouts, incompatible `cve-core` SearchReader contracts, missing or incompatible field mappings, and a missing index return a sanitized HTTP 503 `SEARCH_NOT_READY` response with `Retry-After: 5`. Each OpenSearch readiness request is limited to five seconds by default and can be changed with `SearchReadinessTimeoutMs`.
`GET /api/ready-check` first verifies that the configured CVE index exposes compatible field capabilities for each logical `/search` feature, then sends a minimal zero-result search. CNA, ADP, legacy, and CVSS-version fields that implement the same feature are treated as alternatives, while mapped fields must still use compatible searchable types. This checks mappings, connectivity, index availability, and the same search permission used by the API without retrieving CVE records or counting all matches. It returns HTTP 200 when search requests can be served. Configuration errors, connection failures, timeouts, incompatible `cve-core` SearchReader contracts, missing logical capabilities, incompatible mapped fields, and a missing index return a sanitized HTTP 503 `SEARCH_NOT_READY` response with `Retry-After: 5`. Each OpenSearch readiness request is limited to five seconds by default and can be changed with `SearchReadinessTimeoutMs`.

An AWS load balancer can use `/api/ready-check` to stop routing search traffic to an unready task while the container orchestrator continues using `/api/health-check` for process liveness. Coordinate that probe selection with the infrastructure team before changing deployed health-check behavior.

Expand Down
4 changes: 2 additions & 2 deletions api-docs/openapi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@
"/api/ready-check": {
"get": {
"summary": "OpenSearch readiness check",
"description": "Validates the field capabilities required by the search contract, then sends a minimal zero-result search to the configured CVE index. Returns 200 when mappings, index availability, and search permission are valid, or a sanitized 503 response when a readiness check fails.",
"description": "Validates compatible field capabilities for each logical search feature, then sends a minimal zero-result search to the configured CVE index. Alternative CNA, ADP, legacy, and CVSS-version fields may satisfy the same feature. Returns 200 when mappings, index availability, and search permission are valid, or a sanitized 503 response when a readiness check fails.",
"operationId": "readinessCheck",
"responses": {
"200": {
Expand Down Expand Up @@ -861,4 +861,4 @@
}
}
}
}
}
2 changes: 1 addition & 1 deletion routes/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ router.get(
/*
#swagger.operationId = 'readinessCheck'
#swagger.summary = "OpenSearch readiness check"
#swagger.description = "Validates the field capabilities required by the search contract, then sends a minimal zero-result search to the configured CVE index. Returns 200 when mappings, index availability, and search permission are valid, or a sanitized 503 response when a readiness check fails."
#swagger.description = "Validates compatible field capabilities for each logical search feature, then sends a minimal zero-result search to the configured CVE index. Alternative CNA, ADP, legacy, and CVSS-version fields may satisfy the same feature. Returns 200 when mappings, index availability, and search permission are valid, or a sanitized 503 response when a readiness check fails."
#swagger.responses[200] = {
description: 'API is ready to serve search requests',
headers: {
Expand Down
151 changes: 100 additions & 51 deletions utils/searchReadiness.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,56 +36,78 @@ function containerDateRequirements(relativePaths) {
}));
}

const FIELD_CAPABILITY_REQUIREMENTS = Object.freeze([
{
field: 'cveMetadata.cveId.keyword',
types: ['keyword'],
aggregatable: true
},
{ field: 'cveMetadata.datePublished', types: ['date'] },
{ field: 'cveMetadata.dateUpdated', types: ['date'] },
...keywordRequirements([
'noRejected',
'cvssV3Severity',
'cvssV3_0Severity',
'cvssV3_1Severity',
'cvssV4Severity',
'cvssV2Metrics',
'cvssV3Metrics',
'cvssV3_0Metrics',
'cvssV3_1Metrics',
'cvssV4Metrics',
'cpeName',
'virtualMatchString',
'referenceURL',
'disputed'
]),
...unique(searchScope.resolveFields('keywordSearch')).map((field) => ({
field,
types: ['text', 'keyword']
})),
...unique(searchScope.resolveFields('cweId')).map((field) => ({
field,
types: ['text', 'keyword']
})),
...unique(searchScope.resolveFields('cvssV2Severity')).map((field) => ({
field,
types: NUMERIC_FIELD_TYPES
})),
...unique(searchScope.resolveFields('isVulnerable')).map((field) => ({
field,
types: ['boolean']
})),
...containerKeywordRequirements([
'metrics.other.type',
'metrics.other.content.options.Exploitation',
'metrics.other.content.options.Automatable',
'metrics.other.content.options.Technical Impact',
'metrics.other.content.role'
function capabilityGroup(name, requirements) {
return Object.freeze({
name,
requirements: Object.freeze(requirements)
});
}

const FIELD_CAPABILITY_GROUPS = Object.freeze([
capabilityGroup('cveId', [
{
field: 'cveMetadata.cveId.keyword',
types: ['keyword'],
aggregatable: true
}
]),
...containerDateRequirements(['metrics.other.content.timestamp', 'metrics.other.content.dateAdded'])
capabilityGroup('datePublished', [{ field: 'cveMetadata.datePublished', types: ['date'] }]),
capabilityGroup('dateUpdated', [{ field: 'cveMetadata.dateUpdated', types: ['date'] }]),
capabilityGroup('noRejected', keywordRequirements(['noRejected'])),
capabilityGroup('cvssV3_0Severity', keywordRequirements(['cvssV3_0Severity'])),
capabilityGroup('cvssV3_1Severity', keywordRequirements(['cvssV3_1Severity'])),
capabilityGroup('cvssV4Severity', keywordRequirements(['cvssV4Severity'])),
capabilityGroup('cvssV2Metrics', keywordRequirements(['cvssV2Metrics'])),
capabilityGroup('cvssV3_0Metrics', keywordRequirements(['cvssV3_0Metrics'])),
capabilityGroup('cvssV3_1Metrics', keywordRequirements(['cvssV3_1Metrics'])),
capabilityGroup('cvssV4Metrics', keywordRequirements(['cvssV4Metrics'])),
capabilityGroup('cpeApplicability', keywordRequirements(['cpeName'])),
capabilityGroup('referenceURL', keywordRequirements(['referenceURL'])),
capabilityGroup('disputed', keywordRequirements(['disputed'])),
capabilityGroup(
'keywordSearch',
unique(searchScope.resolveFields('keywordSearch')).map((field) => ({
field,
types: ['text', 'keyword']
}))
),
capabilityGroup(
'cweId',
unique(searchScope.resolveFields('cweId')).map((field) => ({
field,
types: ['text', 'keyword']
}))
),
capabilityGroup(
'cvssV2Severity',
unique(searchScope.resolveFields('cvssV2Severity')).map((field) => ({
field,
types: NUMERIC_FIELD_TYPES
}))
),
capabilityGroup(
'isVulnerable',
unique(searchScope.resolveFields('isVulnerable')).map((field) => ({
field,
types: ['boolean']
}))
),
capabilityGroup('ssvcType', containerKeywordRequirements(['metrics.other.type'])),
capabilityGroup('ssvcExploitation', containerKeywordRequirements(['metrics.other.content.options.Exploitation'])),
capabilityGroup('ssvcAutomatable', containerKeywordRequirements(['metrics.other.content.options.Automatable'])),
capabilityGroup(
'ssvcTechnicalImpact',
containerKeywordRequirements(['metrics.other.content.options.Technical Impact'])
),
capabilityGroup('ssvcRole', containerKeywordRequirements(['metrics.other.content.role'])),
capabilityGroup('ssvcTimestamp', containerDateRequirements(['metrics.other.content.timestamp'])),
capabilityGroup('kevDateAdded', containerDateRequirements(['metrics.other.content.dateAdded']))
]);

const FIELD_CAPABILITY_REQUIREMENTS = Object.freeze(
FIELD_CAPABILITY_GROUPS.flatMap(({ requirements }) => requirements)
);

class SearchMappingContractError extends Error {
constructor(fields) {
super('Configured CVE index does not expose the field mappings required by the Search API.');
Expand All @@ -106,17 +128,43 @@ function supportsRequirement(capabilities, requirement) {
});
}

function hasMappedCapability(capabilities) {
return (
capabilities &&
typeof capabilities === 'object' &&
!Array.isArray(capabilities) &&
Object.keys(capabilities).some((type) => type !== 'unmapped')
);
}

function validateFieldCapabilities(body) {
const capabilities = body?.fields;
if (!capabilities || typeof capabilities !== 'object' || Array.isArray(capabilities)) {
throw new SearchMappingContractError(FIELD_CAPABILITY_REQUIREMENTS.map(({ field }) => field));
}

const incompatibleFields = FIELD_CAPABILITY_REQUIREMENTS.filter(
(requirement) => !supportsRequirement(capabilities[requirement.field], requirement)
).map(({ field }) => field);
const incompatibleFields = [];

for (const group of FIELD_CAPABILITY_GROUPS) {
let hasCompatibleAlternative = false;

for (const requirement of group.requirements) {
const fieldCapabilities = capabilities[requirement.field];
if (!hasMappedCapability(fieldCapabilities)) continue;

if (supportsRequirement(fieldCapabilities, requirement)) {
hasCompatibleAlternative = true;
} else {
incompatibleFields.push(requirement.field);
}
}

if (!hasCompatibleAlternative) {
incompatibleFields.push(...group.requirements.map(({ field }) => field));
}
}

if (incompatibleFields.length) throw new SearchMappingContractError(incompatibleFields);
if (incompatibleFields.length) throw new SearchMappingContractError(unique(incompatibleFields));

return true;
}
Expand All @@ -126,6 +174,7 @@ function getRequiredFieldNames() {
}

module.exports = {
FIELD_CAPABILITY_GROUPS,
FIELD_CAPABILITY_REQUIREMENTS,
SearchMappingContractError,
getRequiredFieldNames,
Expand Down
48 changes: 45 additions & 3 deletions utils/searchReadiness.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,52 @@ describe('search readiness mapping contract', () => {
expect(searchReadiness.validateFieldCapabilities(validCapabilities())).toBe(true);
});

test('rejects missing fields', () => {
test('rejects a missing capability group', () => {
const body = validCapabilities();
const missingField = searchReadiness.getRequiredFieldNames()[0];
delete body.fields[missingField];
const group = searchReadiness.FIELD_CAPABILITY_GROUPS[0];
group.requirements.forEach(({ field }) => delete body.fields[field]);

expect(() => searchReadiness.validateFieldCapabilities(body)).toThrow(
searchReadiness.SearchMappingContractError
);
});

test('accepts an unmapped optional field when another field provides the capability', () => {
const body = validCapabilities();
const group = searchReadiness.FIELD_CAPABILITY_GROUPS.find(({ requirements }) => requirements.length > 1);
body.fields[group.requirements[0].field] = {
unmapped: {
searchable: false,
aggregatable: false
}
};

expect(searchReadiness.validateFieldCapabilities(body)).toBe(true);
});

test('accepts the optional legacy and container variants absent from the dev index', () => {
const body = validCapabilities();
[
'containers.cna.x_legacyV4Record.CVE_data_meta.STATE.keyword',
'containers.adp.metrics.cvssV3_0.baseSeverity.keyword',
'containers.adp.metrics.cvssV2_0.vectorString.keyword',
'containers.adp.metrics.cvssV3_0.vectorString.keyword',
'containers.adp.metrics.cvssV2_0.baseScore',
'containers.cna.metrics.other.content.dateAdded'
].forEach((field) => delete body.fields[field]);

expect(searchReadiness.validateFieldCapabilities(body)).toBe(true);
});

test('rejects an incompatible mapped alternative even when the group has a compatible field', () => {
const body = validCapabilities();
const group = searchReadiness.FIELD_CAPABILITY_GROUPS.find(({ requirements }) => requirements.length > 1);
body.fields[group.requirements[0].field] = {
text: {
searchable: false,
aggregatable: false
}
};

expect(() => searchReadiness.validateFieldCapabilities(body)).toThrow(
searchReadiness.SearchMappingContractError
Expand Down
Loading