Changes prakhar - #17

You have commit messages with errors ⧗ input: feat/fix: finalize async AI processing architecture and patch critical security/data bugs Infrastructure & Architecture: - Completely decoupled file uploading from the main API thread using Redis and Celery. - AI (sentence-transformers) now runs safely in background workers. - Implemented Server-Sent Events (SSE) for live frontend status tracking. - Restored unified hybrid search endpoint. Security & Vulnerability Patches: - Patched 'Zip Slip' directory traversal vulnerability in archive extraction. - Secured upload route against path traversal and malicious overwrites using UUID prefixing. - Prevented temp-directory memory leaks on corrupted ZIP uploads. Data Integrity & Database: - Resolved race conditions in Celery workers using IntegrityError try/rollback blocks. - Forced strict failure propagation so Celery correctly reports 'FAILURE' instead of swallowing exceptions. - Added 'state' column to File model to track indexing progress. - Added 'relation_type' to FileRelationship model. - Fixed one-directional relationship bug by implementing symmetric ORM queries. - Cleaned up app/database namespace collisions. ✖ body's lines must not be longer than 100 characters [body-max-line-length] ✖ subject may not be empty [subject-empty] ✖ type may not be empty [type-empty] ✖ found 3 problems, 0 warnings ⓘ Get help: https://github.com/conventional-changelog/commitlint/#what-is-commitlint ⧗ input: feat: upgrade to background processing with redis and celery This update completely changes how we handle file uploads by moving the heavy AI work into the background so the user doesn't have to wait. Here is everything that changed: - Added Redis and Celery worker to docker-compose to handle background jobs. - Updated Dockerfile to cache Python packages for faster builds. - Fixed the database migrations and added a 'state' column to track file progress (processing, indexed, failed). - Fixed the circular import crashes in the database models. - Updated the upload route to instantly send jobs to the queue and return a task ID. - Added a new '/status' live stream endpoint to show real-time progress to the frontend. - Added an automatic cleanup step in the worker so temporary files are always deleted. - Brought back the GET /files/ endpoint so the dashboard can list all uploaded documents. ✖ body's lines must not be longer than 100 characters [body-max-line-length] ✖ found 1 problems, 0 warnings ⓘ Get help: https://github.com/conventional-changelog/commitlint/#what-is-commitlint ⧗ input: No change ✖ subject may not be empty [subject-empty] ✖ type may not be empty [type-empty] ✖ found 2 problems, 0 warnings ⓘ Get help: https://github.com/conventional-changelog/commitlint/#what-is-commitlint

Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v2. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/