QA Co-pilot

AI-powered QA assistant for *Test Project — search test cases, generate automation code, and debug failures using a 6-stage hybrid RAG pipeline.

What It Does

Capability	How
Search	Find relevant test cases, bug reports, API docs, or requirements by natural language query
Generate	Write Playwright Python / TypeScript or Selenium Python test code from a description
Debug	Paste a failing test stack trace — get root cause + fix suggestion
Explore	Live RAG debugger showing every pipeline stage: rewrite → route → hybrid search → rerank

Tech Stack

Layer	Technology	Version
LLM	Groq `llama-3.3-70b-versatile`	128k context, SSE streaming
Embeddings	`BAAI/bge-m3` — 1024d dense + sparse in one pass	FlagEmbedding 1.2.11
Reranker	`BAAI/bge-reranker-v2-m3` — cross-encoder relevance scoring	FlagEmbedding 1.2.11
Vector DB	Qdrant (local file-store by default, cloud-ready via `QDRANT_URL`)	1.12.1
Backend	FastAPI + Python 3.11+	0.115.5
Auth	JWT · bcrypt (direct) · 3 roles	access 8h · refresh 7d
User DB	SQLite via SQLAlchemy	2.0.36
Frontend	React 18 + Vite + TypeScript	Zustand · lucide-react
PDF parsing	pdfplumber	0.11.4
DOCX parsing	python-docx	1.1.2
Code chunking	tree-sitter AST-aware (falls back to sliding window)	≥0.25.0
Chunking	tiktoken `cl100k_base`, 512 tokens / 50 overlap	≥0.8.0

RAG Pipeline

User Query
    │
    ▼  1. QUERY REWRITE
       Groq rewrites follow-up questions using chat history
       into a standalone, self-contained search query
    │
    ▼  2. LLM ROUTER
       Groq JSON-mode classifies query → selects 1-2 of 5 collections
       Regex heuristics pre-bias (VWO-NNN, BUG-NNN, TC-NNN, api, report, requirement...)
    │
    ▼  3. EMBED (BGE-M3)
       Produces dense (1024d) + sparse vectors in a single model pass
    │
    ▼  4. HYBRID SEARCH (Qdrant)
       Per collection: dense top-12 + sparse top-12
       Fused with Reciprocal Rank Fusion (RRF, k=60)
    │
    ▼  5. CROSS-ENCODER RERANK (BGE-Reranker-v2-m3)
       Reads (question + chunk) pairs jointly → top-5 final chunks
    │
    ▼  6. GENERATE (Groq Llama 3.3 70B)
       System prompt + chat history + <doc> context blocks
       Streams tokens via SSE: meta → sources → token... → done
       Errors (network, API key, rate limit) streamed as error events — server never crashes

Data Sources → Collections

Collection	What goes in	Supported formats	Chunk kind tags
`selenium_code`	Selenium automation source	`.py`, `.java`, `.js`, `.ts`	`function_definition`, `class_definition`, `sliding_window`
`playwright_code`	Playwright specs, fixtures, page objects	`.py`, `.js`, `.ts`, `.jsx`, `.tsx`	`function_definition`, `test`, `sliding_window`
`vwo_testcases`	Manual test cases with JIRA IDs	`.csv`, `.xlsx`, `.xls`	`csv_row`
`vwo_docs`	API docs, requirements, test reports, feature specs	`.pdf`, `.docx`, `.md`, `.txt`	`api_doc`, `requirement`, `test_report`, `pdf_doc`, `markdown_section`
`vwo_bugs`	JIRA bug reports, client communications	`.md`, `.csv`	`jira_bug`, `client_communication`

Data Directory Structure

data/
├── repos/
│   ├── selenium/          ← Selenium source code (.py, .java)
│   └── playwright/        ← Playwright specs (.ts, .js, .py)
├── test_cases/            ← CSV / Excel manual test cases
├── api/                   ← API documentation (PDF, MD, TXT)  [NEW]
├── reports/               ← Test reports (PDF, MD)            [NEW]
├── req/                   ← Requirements: DOCX, PDF, TXT, MD  [NEW]
├── comm/                  ← Client communications (MD, TXT)   [NEW]
└── bugs/                  ← JIRA bug exports (MD, CSV)

Sample data included:

File	Collection	Kind	Chunks
`test_cases/vwo_testcases_sample.csv`	`vwo_testcases`	csv_row	10
`test_cases/testcases_vwo_100.csv`	`vwo_testcases`	csv_row	478
`test_cases/testcases_vwo_100.xlsx`	`vwo_testcases`	csv_row	450
`api/Vwo App Api Documentation Template.pdf`	`vwo_docs`	api_doc	14
`reports/Vwo App Testing Test Report Template.pdf`	`vwo_docs`	test_report	9
`req/Product Requirements Document_ VWO Login Dashboard.docx`	`vwo_docs`	requirement	4
`req/vwo_ab_testing_guide.txt`	`vwo_docs`	requirement	1
`req/vwo_login_prd.txt`	`vwo_docs`	requirement	1
`comm/clientdiscussion_bug2.md`	`vwo_bugs`	client_communication	11
`bugs/Bug_VWO_26.md`	`vwo_bugs`	jira_bug	1
`bugs/Bug_VWO_32.md`	`vwo_bugs`	jira_bug	1
`repos/selenium/*.py`	`selenium_code`	function_definition	4
`repos/playwright/.ts / .js`	`playwright_code`	function_definition	3

Total ingested: 987 chunks across 5 collections

Project Structure

Chapter_09A_Project_QACopilot/
├── backend/
│   ├── main.py                   FastAPI app — CORS, router mounts
│   ├── config.py                 All settings — models, paths, knobs
│   │                             Includes: DOCS_DIRS, COMM_DIR, API_DOCS_DIR, REPORTS_DIR, REQ_DIR
│   ├── .env.example              Template for environment variables
│   │
│   ├── api/
│   │   ├── auth.py               POST /register /login /refresh  GET /me
│   │   ├── chat.py               POST /  (SSE stream)  POST /explore  GET /sessions/list
│   │   └── ingest.py             POST /run  GET /status /counts
│   │
│   ├── rag/
│   │   ├── embedder.py           BGE-M3 — dense + sparse embeddings
│   │   ├── reranker.py           BGE-Reranker-v2-m3 (with RRF fallback on error)
│   │   ├── router.py             LLM classifier + regex hints (BUG-NNN, api, report, comm...)
│   │   ├── retriever.py          Full 6-stage pipeline + trace output
│   │   ├── generator.py          Groq streaming + error-safe SSE yielding
│   │   └── chunker.py            Sliding-window + CSV row chunker
│   │
│   ├── db/
│   │   ├── qdrant_client.py      Qdrant client — bootstrap, hybrid search, RRF, upsert
│   │   └── user_db.py            SQLite — User model, ChatSession, JWT helpers
│   │
│   ├── ingest/
│   │   ├── _runner.py            Shared embed + upsert helper
│   │   ├── ingest_all.py         Run all 5 pipelines (CLI entry point)
│   │   ├── ingest_code.py        Selenium + Playwright repos (tree-sitter / sliding window)
│   │   ├── ingest_testcases.py   CSV / Excel test cases (row-per-chunk)
│   │   ├── ingest_docs.py        Scans api/, reports/, req/, pdfs/ — PDF + DOCX + MD + TXT
│   │   └── ingest_bugs.py        Scans bugs/ (JIRA) + comm/ (client communications)
│   │
│   └── requirements.txt
│
├── frontend/
│   └── src/
│       ├── pages/
│       │   ├── Chat.tsx          SSE streaming chat + "Thinking..." state + error bubbles
│       │   └── Explorer.tsx      Live RAG debugger — all 4 pipeline stages
│       └── components/
│           └── Navbar.tsx        Live ingest status pill (polls every 3s/30s)
│
├── data/
│   ├── repos/selenium/    ← Selenium source files
│   ├── repos/playwright/  ← Playwright spec files
│   ├── test_cases/        ← CSV + Excel test cases
│   ├── api/               ← API documentation
│   ├── reports/           ← Test reports
│   ├── req/               ← Requirements (DOCX, TXT, PDF)
│   ├── comm/              ← Client communications
│   └── bugs/              ← JIRA bug exports
│
├── KT/
│   └── rag_explorer.html         Offline knowledge-transfer page
│
├── CLAUDE.md
├── README.md
├── USER_MANUAL.md
└── FEATURE_ROADMAP.md

Setup & Run

Prerequisites

Python 3.11+
Node.js 18+
A Groq API key (free tier works)
~4 GB free disk space (models downloaded once on first use)

1 — Backend

cd backend
python -m venv .venv
.venv\Scripts\Activate.ps1        # Windows PowerShell

pip install -r requirements.txt   # downloads BGE-M3 ~2GB + Reranker ~1.1GB on first run

cp .env.example .env
# Edit .env — set GROQ_API_KEY and JWT_SECRET_KEY

2 — Place your data

Drop files into the appropriate subdirectory under data/:

Folder	What to put there
`data/repos/selenium/`	Selenium repo (git clone or copy)
`data/repos/playwright/`	Playwright repo (git clone or copy)
`data/test_cases/`	CSV or Excel test case files
`data/api/`	API documentation PDFs, markdown, text files
`data/reports/`	Test report PDFs or markdown files
`data/req/`	Requirements: `.docx`, `.pdf`, `.txt`, `.md`
`data/comm/`	Client communication exports (`.md`, `.txt`)
`data/bugs/`	JIRA bug exports (`.md` one per bug, or `.csv`)

3 — Ingest

# Stop server first (Qdrant file-store is single-process)
# Then run from backend/ directory:

python -c "import sys; sys.path.insert(0,'.'); from ingest.ingest_all import main; main()"

# Full rebuild (drops + recreates all collections):
python -c "import sys; sys.path.insert(0,'.'); from ingest.ingest_all import main; main(recreate=True)"

# Single collection:
python -c "import sys; sys.path.insert(0,'.'); from ingest.ingest_docs import ingest_docs; ingest_docs()"
python -c "import sys; sys.path.insert(0,'.'); from ingest.ingest_bugs import ingest_bugs; ingest_bugs()"

4 — Start backend

# IMPORTANT: always --workers 1
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 1

5 — Start frontend

cd frontend && npm install && npm run dev
# http://localhost:5173

API Reference

Auth

Method	Endpoint	Description
POST	`/api/auth/register`	`{name, email, password, role}`
POST	`/api/auth/login`	form: `username`, `password` → tokens
POST	`/api/auth/refresh`	`{refresh_token}` → new tokens
GET	`/api/auth/me`	Current user info

Chat

Method	Endpoint	Description
POST	`/api/chat/`	SSE stream: `{message, framework?, session_id?, forced_collections?}`
POST	`/api/chat/explore`	Full RAG trace JSON (no streaming)
GET	`/api/chat/sessions/list`	All sessions for current user
GET	`/api/chat/{id}/history`	Session messages
DELETE	`/api/chat/{id}`	Delete session

SSE events: meta · sources · token · error · done

Frameworks: Playwright Python · Playwright TypeScript · Selenium Python

Ingest (admin only)

Method	Endpoint	Description
POST	`/api/ingest/run?recreate=false`	Start background ingest
GET	`/api/ingest/status`	`{running, results, error}`
GET	`/api/ingest/counts`	Chunk count per collection

Environment Variables

Variable	Default	Description
`GROQ_API_KEY`	—	Required. From console.groq.com
`JWT_SECRET_KEY`	placeholder	Required. Long random string
`EMBED_MODEL`	`BAAI/bge-m3`	HuggingFace embedding model
`RERANK_MODEL`	`BAAI/bge-reranker-v2-m3`	HuggingFace reranker
`BGE_USE_FP16`	`1`	Set `0` on strict CPU-only machines
`QDRANT_PATH`	`../qdrant_db`	Local Qdrant file-store path
`QDRANT_URL`	—	Qdrant server/cloud URL
`DATA_DIR`	`../data`	Root data directory
`SQLITE_URL`	`sqlite:///./qa_copilot.db`	User + session DB

Common Issues

"Failed to fetch" → Backend not running. Start it in your own PowerShell terminal.

Qdrant file lock → Get-Process -Name "python*" | Stop-Process -Force, then restart.

Reranker slow (15–30s) → Set BGE_USE_FP16=1 in .env and restart.

DOCX not parsed → Ensure python-docx is installed: pip install python-docx==1.1.2

Unicode error during ingest on Windows → Print statements in ingest use ASCII arrows (->), not Unicode. If you see charmap errors, check for emoji/arrow characters in your custom data files.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

QA Co-pilot

What It Does

Tech Stack

RAG Pipeline

Data Sources → Collections

Data Directory Structure

Project Structure

Setup & Run

Prerequisites

1 — Backend

2 — Place your data

3 — Ingest

4 — Start backend

5 — Start frontend

API Reference

Auth

Chat

Ingest (admin only)

Environment Variables

Common Issues

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
KT		KT
backend		backend
data		data
frontend		frontend
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
FEATURE_ROADMAP.md		FEATURE_ROADMAP.md
README.md		README.md
USER_MANUAL.md		USER_MANUAL.md

Folders and files

Latest commit

History

Repository files navigation

QA Co-pilot

What It Does

Tech Stack

RAG Pipeline

Data Sources → Collections

Data Directory Structure

Project Structure

Setup & Run

Prerequisites

1 — Backend

2 — Place your data

3 — Ingest

4 — Start backend

5 — Start frontend

API Reference

Auth

Chat

Ingest (admin only)

Environment Variables

Common Issues

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages