Feature/sc 46895/linker bulk corrector

[nsantacruz]

This pull request introduces a new "Linker Bulk Corrector" feature for staff users, providing a UI and backend API for searching, navigating, and bulk re-parsing stored linker citation spans. It also adds support for tracking reasons when adding dataset examples and includes several related backend enhancements and optimizations.

New Feature: Linker Bulk Corrector

• Adds a staff-only "Linker Bulk Corrector" page and route to the frontend (ReaderApp.jsx, ReaderPanel.jsx, reader/views.py, sefaria/urls_library.py, static/css/linker-bulk-corrector.css). [1] [2] [3] [4] [5] [6] [7] [8]
• Implements corresponding API endpoints for search, navigation, and (re-)parsing citations and datasets (api/linker_bulk_corrector_views.py, sefaria/urls_shared.py). [1] [2] [3]

Backend Enhancements for Linker Tasks

• Adds batch parsing and bulk dataset re-parsing Celery tasks, and synchronous helpers for batch operations (sefaria/helper/linker/tasks.py, sefaria/helper/linker_resource_panel_admin.py). [1] [2] [3]
• Registers the new "bulk_reparse_dataset" action in the linker editor history (sefaria/model/linker_editor_history.py).

Dataset Example Improvements

• Adds support for an optional reason field when adding or updating linker dataset examples, including backend, model, and schema updates (sefaria/helper/linker_resource_panel_admin.py, sefaria/model/linker_dataset_example.py). [1] [2] [3] [4] [5]

Database and Model Optimizations

• Adds new database indices for marked_up_text_chunks and linker_output collections to optimize queries by ref, versionTitle, and language (sefaria/system/database.py).
• Adds a disqualificationReason field to the LinkerOutput model schema to track why a citation span might be disqualified (sefaria/model/marked_up_text_chunk.py).

These changes collectively provide staff with powerful tools for managing and correcting linker citations at scale, while improving tracking and performance for related data operations.

[gitvelocity-reviewer]

📊 Code Quality Score: 61/100

Base Score 61 × ESF 1.0 (Extra Large: 1360 effective lines, 16 files) = 61

Category	Score	Factors
🔭 Scope	16/20	The change spans 16 files across backend helper (linker_bulk_corrector.py, 576 lines), Celery tasks (tasks.py), API views (linker_bulk_corrector_views.py), model schema additions (linker_dataset_example.py, marked_up_text_chunk.py, linker_editor_history.py), DB index definitions (database.py), URL routing (urls_library.py, urls_shared.py), Django view (reader/views.py), frontend page (LinkerBulkCorrectorPage.jsx, 514 lines), CSS (linker-bulk-corrector.css, 331 lines), and template (base.html). Four new API endpoints and one new page route are introduced.
🏗️ Architecture	13/20	A new linker_bulk_corrector helper module is introduced, mirroring the existing linker_resource_panel_admin pattern. Two new Celery tasks (parse_linker_citations_batch_task, bulk_reparse_dataset_task) are added to the existing tasks.py. A new view module linker_bulk_corrector_views.py follows the linker_admin_views pattern with a shared _handle base class. Compound indices on (ref, versionTitle, language) are added to marked_up_text_chunks and linker_output collections. No new external dependencies are introduced.
⚙️ Implementation	15/20	group_citation_spans groups and sorts citation spans across LinkerOutput documents by a composite key. navigate_dataset implements cursor-based forward/backward navigation with a NAVIGATION_SCAN_LIMIT guard, calling parse_linker_citation_sync per item in a loop. persist_citation_resolution coordinates updates across LinkerOutput, MarkedUpTextChunk, and Link collections, including reuse of existing links via _update_generated_link. render_citation_snippet generates a windowed HTML snippet and applies _mark_snippet_anchors via regex substitution. The React component manages dataset state, localStorage persistence, keyboard shortcuts (ArrowRight/l, ArrowLeft/h, r), Celery task polling at 1500ms intervals, and a 20-iteration navigation loop.
⚠️ Risk	12/20	persist_citation_resolution writes to LinkerOutput and MarkedUpTextChunk documents and calls tracker.delete on Link objects, making it a multi-collection mutation path. bulk_reparse_dataset_task runs as a Celery task and iterates over all filtered citations in a dataset, calling persist_citation_resolution for each. All endpoints are gated by StaffRequiredMixin. No feature flag or rollback plan is documented. DB index additions are additive and safe.
✅ Quality	2/15	No tests accompany the PR. linker_bulk_corrector.py (576 lines), linker_bulk_corrector_views.py, LinkerBulkCorrectorPage.jsx, and the two new Celery tasks in tasks.py have no test coverage. parse_linker_citations_batch_sync and parse_linker_citations_batch in linker_resource_panel_admin.py are also untested.
🔒 Perf / Security	3/5	Compound indices on (ref, versionTitle, language) are added proactively for the query patterns used in _book_query and _load_item. search_citations dispatches a single batch Celery task via parse_linker_citations_batch_sync rather than N individual tasks. navigate_dataset makes one synchronous Celery call per item in a loop (up to NAVIGATION_SCAN_LIMIT = 200), which is inconsistent with the batch approach used in search. All four API endpoints require staff authentication via StaffRequiredMixin.

Was this score accurate? 👍 Yes · 👎 No

How this was scored →

_{Scored by GitVelocity · How are scores calculated?}