fix: csv feed preview shows the header row as an importable attribute

[righel]

BLUF

• Priority: medium
• Follow-up to [medium] fix: csv feed with header=true imports zero attributes #389. That PR fixed the header skip in the ingestion task; preview_csv_feed never had one, so the feed wizard previews the header row as a real attribute.
• Also fixes a KeyError: 'header' that escapes fetch_csv_feed entirely for a csvConfig written before the flag existed.
• Root cause of both: preview and ingest each decided independently whether row 0 is data. They now share one helper.

What was wrong

1. The preview never skipped the header

preview_csv_feed ran process_csv_feed_row over every parsed line. With header on or off the output was identical — the header came back as an ip-dst attribute valued "value":

header=True  preview values=['value', '1.1.1.1', '2.2.2.2', '3.3.3.3', '4.4.4.4']
header=False preview values=['value', '1.1.1.1', '2.2.2.2', '3.3.3.3', '4.4.4.4']

The header also ate one of the limit slots, so a 5-row preview showed 4 real rows.

This matters because the frontend defaults the toggle on (AddFeedCsv.vue:31, header: saved?.header ?? true), so it is the default path, and TestCSVFeedModal.vue:64 renders testResult.preview straight from this endpoint.

2. KeyError: 'header' killed the whole ingestion task

db_feed.settings["csvConfig"]["header"] sits outside the per-row try/except, so a config without the key did not degrade to a failed row — it propagated out of fetch_csv_feed:

E       KeyError: 'header'
app/worker/tasks.py:463: KeyError

What this changes

• New feeds_repository.csv_feed_has_header(), used by both the preview endpoint and the ingestion task, so the two can no longer disagree about whether row 0 is data. It reads the flag with .get(..., False).
• preview_csv_feed skips the header when building preview, and reads one extra line so a header no longer costs a data row.

rows deliberately still carries the header: AddFeedCsv.vue:73-75 derives the column labels from rows[0] and slices it off itself, so stripping it server-side would break the wizard. Only preview — what would actually be imported — skips it. CsvPreview.vue renders rows and preview as two independent tables, so the differing lengths are fine.

After:

header=True  rows=[['value'], ['1.1.1.1'], … ['5.5.5.5']]   ← header retained
             preview values=['1.1.1.1', '2.2.2.2', '3.3.3.3', '4.4.4.4', '5.5.5.5']
header=False unchanged

How it was verified

10 mock-only tests (no database, OpenSearch or network): TestCsvFeedHasHeader and TestPreviewCsvFeed in api/app/tests/repositories/test_feeds.py, plus one case added to TestFetchCsvFeedTask from #389.

Each fix was reverted individually to confirm the tests discriminate rather than merely pass:

• test_header_row_is_not_previewed_as_an_attribute → fails
• test_a_header_does_not_cost_a_data_row → fails
• test_a_missing_header_key_is_treated_as_no_header → fails with KeyError

The other four pass before and after by design — they guard against an over-correcting fix (header still present in rows, no-header path untouched).

flake8 clean. No schema, route or migration change, so docs/features/api/openapi.json is unaffected.

Note: the container ships black 26.3.1 while .pre-commit-config.yaml pins 25.1.0, and the two disagree about the test file merged in #389. Only the lines added here were formatted. The isort failures on feeds.py, tasks.py and test_feeds.py are pre-existing on main; no imports were added.

🤖 Generated with Claude Code

[github-actions]

OpenAPI changes

No changes to the OpenAPI spec.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.59%. Comparing base (9363bdf) to head (02b78f4).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #393      +/-   ##
==========================================
+ Coverage   84.56%   84.59%   +0.03%     
==========================================
  Files         209      209              
  Lines       19425    19465      +40     
==========================================
+ Hits        16427    16467      +40     
  Misses       2998     2998              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.