Harden v0.5.0 search correctness and release readiness

README.md

@@ -99,6 +99,8 @@ An AWS load balancer can use `/api/ready-check` to stop routing search traffic t
 
 Use the optional `scope` field to control which parts of each record participate in matching. Values are case-insensitive: `FULL` (the default) searches base record fields, CNA data, and every ADP provider; `CNA_ONLY` searches base record fields and CNA data; and `ADP_ONLY` searches base record fields and every ADP provider. Scope changes matching only, so successful responses always contain the complete CVE record. CPE applicability filters use CNA data and return HTTP 400 when combined with `ADP_ONLY`.
 
+Date filters use the API's full UTC timestamp format and inclusive boundaries. When both boundaries for publication, modification, SSVC timestamp, or KEV date-added filters are supplied, the start must be before or equal to the end.
+
 Scope applies to filter matching as follows:
 
 | Filters                                                                          | Data used for matching                                                                |
@@ -117,7 +119,7 @@ SSVC metrics can be filtered with `ssvcExploitation` (`NONE`, `POC`, or `ACTIVE`
 
 Use `knownExploited` to include or exclude records with a structured KEV metric in containers selected by `scope`. The optional `kevDateAddedStartDate` and `kevDateAddedEndDate` boundaries are inclusive, require `knownExploited: true`, and use the API's full UTC timestamp format. Stored KEV `dateAdded` values are interpreted as midnight UTC.
 
-Use `referenceURL` for a case-insensitive literal substring search over reference URLs in containers selected by `scope`. The filter accepts one string; user-supplied wildcard syntax is treated literally.
+Use `referenceURL` for a case-insensitive literal substring search over reference URLs in containers selected by `scope`. The filter accepts one string of at least three characters; user-supplied wildcard syntax is treated literally.
 
 Use the strict boolean `disputed` filter to require (`true`) or exclude (`false`) the exact `disputed` tag in containers selected by `scope`. This does not provide a general-purpose record tag filter.
 
@@ -132,7 +134,7 @@ String inputs are bounded before query construction:
 | `virtualMatchString`         |                                                       2,048 characters |
 | `versionStart`, `versionEnd` |                                                    256 characters each |
 | `ssvcRole`                   |                                                         256 characters |
-| `referenceURL`               |                                                       2,048 characters |
+| `referenceURL`               |                                             3 through 2,048 characters |
 | CVSS metric fields           | 256 characters, 32 slash-delimited tokens, and 32 characters per token |
 
 Requests that exceed a limit return HTTP 400 before contacting OpenSearch. These bounds limit input size and query complexity without changing the documented CPE matching semantics.
@@ -291,7 +293,7 @@ This is a process-wide safety limit, not a precise distributed quota. Each ECS t
 
 ### Semantic Post-Filtering
 
-SSVC filters and date-bounded positive `knownExploited` searches use semantic post-filtering because OpenSearch ordinary-object arrays cannot guarantee that multiple conditions matched the same structured metric entry. These searches retrieve up to 10,000 lightweight candidates, apply same-entry checks in Node.js, and fetch the requested page as complete CVE records. The reported total and pagination are complete only within that bounded candidate set; qualifying records beyond the first 10,000 OpenSearch candidates may not be represented. This preserves the existing bounded post-filter behavior until PIT-based pagination is implemented. SSVC and KEV candidates do not use the temporary CPE cache or cold-search queue. A `knownExploited` filter without date boundaries uses a direct OpenSearch query and retains ordinary exact-total behavior.
+CVSS vector filters, SSVC filters, and date-bounded positive `knownExploited` searches use semantic post-filtering because OpenSearch ordinary-object arrays cannot guarantee that multiple conditions matched the same vector or structured metric entry. These searches retrieve up to 10,000 lightweight candidates, apply same-entry checks in Node.js, and fetch the requested page as complete CVE records. Every token supplied to one CVSS vector filter must occur in the same eligible vector. The reported total and pagination are complete only within that bounded candidate set; qualifying records beyond the first 10,000 OpenSearch candidates may not be represented. This preserves the existing bounded post-filter behavior until PIT-based pagination is implemented. CVSS, SSVC, and KEV candidates do not use the temporary CPE cache or cold-search queue. A `knownExploited` filter without date boundaries uses a direct OpenSearch query and retains ordinary exact-total behavior.
 
 ### CPE Post-Filter Cache
 

api-docs/openapi.json

@@ -1,7 +1,7 @@
 {
   "openapi": "3.0.2",
   "info": {
-    "version": "0.4.0-beta",
+    "version": "0.5.0-beta",
     "title": "CVE Search API",
     "description": "CVE Search API to retrieve all matching CVE records after applying request inputs as filters.",
     "contact": {
@@ -310,7 +310,7 @@
     "schemas": {
       "SearchRequest": {
         "type": "object",
-        "description": "Optional filters and pagination controls for a CVE record search. Enum-valued fields are accepted case-insensitively and normalized to the documented canonical values.",
+        "description": "Optional filters and pagination controls for a CVE record search. Enum-valued fields are accepted case-insensitively and normalized to the documented canonical values. When both boundaries of a date range are supplied, the start must be before or equal to the end.",
         "additionalProperties": false,
         "properties": {
           "scope": {
@@ -575,9 +575,10 @@
           },
           "referenceURL": {
             "type": "string",
+            "minLength": 3,
             "maxLength": 2048,
             "example": "vendor.example/security/advisory",
-            "description": "Returns records whose scope-selected references contain this case-insensitive literal URL substring. Wildcard syntax is not supported."
+            "description": "Returns records whose scope-selected references contain this case-insensitive literal URL substring. Wildcard syntax is not supported. The substring must contain at least 3 characters."
           },
           "disputed": {
             "type": "boolean",

api-docs/openapi.test.js

@@ -58,6 +58,7 @@ describe('OpenAPI contract', () => {
         expect(operation.parameters || []).toStrictEqual([]);
         expect(mediaType.schema).toStrictEqual({ $ref: '#/components/schemas/SearchRequest' });
         expect(schema.additionalProperties).toBe(false);
+        expect(schema.description).toContain('the start must be before or equal to the end');
         expect(Object.keys(schema.properties)).toStrictEqual(constants.SEARCH_QUERY_PARAMS);
         Object.values(schema.properties).forEach((property) => {
             expect(property.description).toEqual(expect.any(String));
@@ -84,6 +85,7 @@ describe('OpenAPI contract', () => {
         expect(schema.properties.knownExploited.type).toBe('boolean');
         expect(schema.properties.kevDateAddedStartDate.description).toContain('midnight UTC');
         expect(schema.properties.referenceURL.description).toContain('literal URL substring');
+        expect(schema.properties.referenceURL.minLength).toBe(constants.MIN_REFERENCE_URL_LENGTH);
         expect(schema.properties.disputed.type).toBe('boolean');
         expect(schema.properties.isVulnerable.description).toContain('same CPE Match Criteria entry');
         expect(schema.properties.isVulnerable.description).toContain('False or omitted');

controllers/middleware.js

@@ -55,6 +55,12 @@ const dateParams = [
     'kevDateAddedStartDate',
     'kevDateAddedEndDate'
 ];
+const dateRangePairs = [
+    ['lastModStartDate', 'lastModEndDate'],
+    ['pubStartDate', 'pubEndDate'],
+    ['ssvcTimestampStartDate', 'ssvcTimestampEndDate'],
+    ['kevDateAddedStartDate', 'kevDateAddedEndDate']
+];
 const versionRangeParams = ['versionStart', 'versionStartType', 'versionEnd', 'versionEndType'];
 const versionRangeValueParams = ['versionStart', 'versionEnd'];
 const versionRangeTypeParams = ['versionStartType', 'versionEndType'];
@@ -186,6 +192,15 @@ function validateParams(req, res, next) {
             }
         }
 
+        if (
+            typeof params.referenceURL === 'string' &&
+            params.referenceURL.trim().length < constants.MIN_REFERENCE_URL_LENGTH
+        ) {
+            return res
+                .status(400)
+                .json(error.searchFieldMinimumLengthError('referenceURL', constants.MIN_REFERENCE_URL_LENGTH));
+        }
+
         if (
             typeof params.keywordSearch === 'string' &&
             countWhitespaceDelimitedTokens(params.keywordSearch) > constants.MAX_KEYWORD_SEARCH_TOKENS
@@ -358,6 +373,16 @@ function validateParams(req, res, next) {
             }
         }
 
+        for (const [startParam, endParam] of dateRangePairs) {
+            if (
+                hasParam(params, startParam) &&
+                hasParam(params, endParam) &&
+                Date.parse(params[startParam]) > Date.parse(params[endParam])
+            ) {
+                return res.status(400).json(error.dateRangeOrderError(startParam, endParam));
+            }
+        }
+
         next();
     } catch (err) {
         next(err);

controllers/middleware.test.js

@@ -818,6 +818,43 @@ describe('middleware validation function', () => {
         }
     );
 
+    test.each([
+        ['lastModStartDate', 'lastModEndDate'],
+        ['pubStartDate', 'pubEndDate'],
+        ['ssvcTimestampStartDate', 'ssvcTimestampEndDate'],
+        ['kevDateAddedStartDate', 'kevDateAddedEndDate']
+    ])('ValidateParams rejects an inverted %s and %s range.', (startParam, endParam) => {
+        const params = {
+            [startParam]: '2024-12-31T00:00:00.000Z',
+            [endParam]: '2024-01-01T00:00:00.000Z'
+        };
+        if (startParam.startsWith('kev')) params.knownExploited = true;
+
+        validate(params);
+
+        expectValidationError(
+            'DATE_RANGE_ORDER',
+            `The ${startParam} parameter must be before or equal to ${endParam}.`
+        );
+    });
+
+    test.each([
+        ['lastModStartDate', 'lastModEndDate'],
+        ['pubStartDate', 'pubEndDate'],
+        ['ssvcTimestampStartDate', 'ssvcTimestampEndDate'],
+        ['kevDateAddedStartDate', 'kevDateAddedEndDate']
+    ])('ValidateParams accepts equal inclusive %s and %s boundaries.', (startParam, endParam) => {
+        const params = {
+            [startParam]: '2024-01-01T00:00:00.000Z',
+            [endParam]: '2024-01-01T00:00:00.000Z'
+        };
+        if (startParam.startsWith('kev')) params.knownExploited = true;
+
+        validate(params);
+
+        expectValidationSuccess();
+    });
+
     test.each([
         ['2024'],
         ['2024-01'],
@@ -956,6 +993,21 @@ describe('middleware validation function', () => {
         expectValidationError('DISPUTED_BOOLEAN', 'The disputed parameter must be a boolean.');
     });
 
+    test('ValidateParams rejects referenceURL substrings shorter than the minimum', () => {
+        validate({ referenceURL: 'ab' });
+
+        expectValidationError(
+            'SEARCH_FIELD_MINIMUM_LENGTH',
+            `The referenceURL field must be at least ${constants.MIN_REFERENCE_URL_LENGTH} characters.`
+        );
+    });
+
+    test('ValidateParams accepts referenceURL substrings at the minimum length', () => {
+        validate({ referenceURL: 'abc' });
+
+        expectValidationSuccess();
+    });
+
     test('ValidateParams delegates unexpected errors to next()', () => {
         const err = new Error('query failure');
         Object.defineProperty(req, 'query', {

controllers/searchController.js

@@ -76,22 +76,6 @@ function buildWebSearchSort(sort) {
     return undefined;
 }
 
-function applySearchPostFilters(resp, postFilters, responseOptions, metrics) {
-    if (!searchPostFilters.hasSearchPostFilters(postFilters)) return resp;
-
-    const { hitsContainer, candidateCount, filteredHits } = searchPostFilters.filterSearchHits(resp, postFilters);
-    if (!Array.isArray(hitsContainer?.hits)) return resp;
-
-    const pagedHits = paginateHits(filteredHits, responseOptions);
-    if (metrics) {
-        metrics.candidateCount = candidateCount;
-        metrics.filteredCount = filteredHits.length;
-        metrics.returnedCount = pagedHits.length;
-    }
-
-    return buildPagedSearchResponse(resp, filteredHits.length, pagedHits);
-}
-
 function getHitCveId(hit) {
     return hit?._source?.cveMetadata?.cveId ?? hit?._id;
 }
@@ -127,7 +111,7 @@ async function executeOpenSearch(body, operation, signal, metrics) {
     });
 }
 
-async function executeStandardSearch(searchOptions, queryStringObjects, postFilters, responseOptions, metrics, signal) {
+async function executeStandardSearch(searchOptions, queryStringObjects, metrics, signal) {
     metrics.strategy = 'standard';
 
     const searchStart = searchPerformance.markTime();
@@ -140,11 +124,7 @@ async function executeStandardSearch(searchOptions, queryStringObjects, postFilt
     metrics.openSearchElapsedMs = searchPerformance.elapsedMs(searchStart);
     metrics.openSearchTookMs = searchData?.took;
 
-    const postFilterStart = searchPerformance.markTime();
-    const resp = applySearchPostFilters(searchResp, postFilters, responseOptions, metrics);
-    metrics.postFilterElapsedMs = searchPerformance.elapsedMs(postFilterStart);
-
-    return resp;
+    return searchResp;
 }
 
 async function fetchFullHitsByCveIds(cveIds, metrics, signal) {
@@ -284,7 +264,7 @@ async function executeOptimizedPostFilterSearch(
 
 async function executeSearch(searchOptions, queryStringObjects, postFilters, responseOptions, metrics, signal) {
     if (!searchPostFilters.hasSearchPostFilters(postFilters)) {
-        return executeStandardSearch(searchOptions, queryStringObjects, postFilters, responseOptions, metrics, signal);
+        return executeStandardSearch(searchOptions, queryStringObjects, metrics, signal);
     }
 
     return executeOptimizedPostFilterSearch(
@@ -460,7 +440,6 @@ async function webSearchHandler(req, res, next) {
 }
 
 module.exports = {
-    applySearchPostFilters,
     buildWebSearchSort,
     SEARCH: searchHandler,
     WEB_SEARCH: webSearchHandler