An agentic mock-interview platform that ingests a resume, infers realistic target roles, runs an adaptive multimodal interview, and produces an explainable coaching report.
Built for the hackathon problem statement: "Intelligent Mock Interview Agent".
- Node.js 18+ (tested on Node 24)
- pnpm 9+
- A Google Gemini API key — free at https://aistudio.google.com/app/apikey
- The Deepgram and JSearch keys are already pre-filled in
.env.local
# 1. Install
pnpm install
# 2. Open .env.local and paste your Gemini key into GEMINI_API_KEY
# (Deepgram + JSearch keys are already populated)
# 3. Start Convex (datastore + serverless functions, live-reload).
# On first run it provisions the deployment and writes the Convex
# URL into .env.local.
npx convex dev
# 4. In a second terminal, run the Next.js dev server
pnpm dev
# 5. Open http://localhost:3000Two processes: npx convex dev (Convex — the datastore + serverless
functions) and pnpm dev (Next.js). Convex is a managed serverless
datastore, so there is still no SQL database or docker to set up.
- Land on the homepage, click Upload resume → Begin.
- Drop a PDF/text resume on the dashboard.
- Server extracts text via
pdf-parseand asks Gemini to infer skills, seniority, domain, and 3–5 realistic target roles. - In parallel, JSearch returns ~15 live postings keyed off the top inferred roles, then Gemini scores each one against the candidate profile.
- Server extracts text via
- Pick the role you're training for, hit Start adaptive interview.
- Grant camera + mic. The interview studio opens.
- Adaptive loop — for each question:
- Click Begin answer → Deepgram streams a live transcript with filler-word counting.
- The candidate's webcam is analysed in-browser at 4 fps for engagement, composure, posture, and stress (browser
FaceDetector+ brightness/motion heuristics — no model download). - Click Submit answer → Gemini evaluates the transcript against the question's expected concepts, the orchestrator adapts difficulty, and the next question is generated.
- After ~6–8 questions the report page renders an explainable verdict with per-question detail.
src/
├── app/
│ ├── page.tsx # landing
│ ├── dashboard/page.tsx # resume + jobs + role selection
│ ├── interview/page.tsx # live interview studio
│ ├── report/page.tsx # final report
│ └── api/
│ ├── resume/parse/route.ts # PDF → ResumeAnalysis (Gemini)
│ ├── jobs/recommend/route.ts # resume + JSearch → ranked JobMatch[] (Gemini)
│ ├── interview/
│ │ ├── question/route.ts # adaptive next-question generator (Gemini)
│ │ ├── evaluate/route.ts # per-answer scoring (Gemini + Deepgram heuristics)
│ │ ├── transcribe/route.ts # fallback Deepgram REST transcribe
│ │ └── report/route.ts # final feedback report (Gemini)
│ └── deepgram/token/route.ts # ephemeral key for browser STT
├── components/
│ ├── ui/Button.tsx, Card.tsx # primitives
│ ├── Logo.tsx, ConvexClientProvider.tsx
│ ├── dashboard/{ResumeUpload, RoleCard, JobCard, HistoryPanel}.tsx
│ └── interview/ScoreMeter.tsx
├── hooks/
│ ├── useDeepgramStream.ts # live STT + filler-word capture
│ └── useVisualMetrics.ts # browser CV: face detection + motion/brightness heuristics
├── lib/
│ ├── gemini.ts # JSON-mode Gemini wrapper
│ ├── deepgram.ts # transcript analysis + comm/confidence scoring
│ ├── jsearch.ts # RapidAPI JSearch client
│ ├── pdf.ts # pdf-parse wrapper
│ ├── anon.ts # anonymous browser-id hook (localStorage)
│ ├── types.ts # all interview/job/report types
│ └── utils.ts
├── convex/ # datastore (anon-id scoped)
│ ├── schema.ts # resumes / jobs / sessions tables
│ └── resumes.ts · jobs.ts · sessions.ts # queries + mutations
└── globals.css # tiny: tailwind directives + base only
All component-level styles live inside the TSX as Tailwind classes — globals.css only carries directives, base, and a film-grain overlay.
| # | Module | Implementation |
|---|---|---|
| 1 | Context Understanding | api/resume/parse — Gemini structured output: skills, domain, seniority, inferred roles, focus areas |
| 2 | Interview Orchestrator | api/interview/question — looks at last answer scores, current difficulty, asked types, then chooses next question + adapts difficulty 1–5 |
| 3 | Audio Intelligence | useDeepgramStream (live) + lib/deepgram.ts::analyzeTranscript → WPM, filler ratio, confidence, hesitation count → communicationScore |
| 4 | Visual Intelligence | useVisualMetrics — heuristic CV: browser FaceDetector for face presence + iris-center deviation, brightness variance, frame-to-frame motion → engagement / composure / posture / stress |
| 5 | Technical Evaluation | api/interview/evaluate — Gemini scores correctness + depth, identifies missing concepts, drafts ideal answer |
| 6 | Feedback & Coaching | api/interview/report — aggregates per-answer scores, weights 0.5·tech + 0.25·comm + 0.25·conf, produces specific (not generic) coaching |
| 7 | Job Recommendations | api/jobs/recommend — JSearch fanout → Gemini ranks each job 0–100 with 3 fit reasons |
Per answer, three signals are produced and combined:
technical = mean(correctness_score, depth_score) // Gemini
communication = clarity·40 + pace·30 + fluency·30 − filler_penalty // from Deepgram metrics
confidence = audio_confidence + visual(engagement, composure − stress) // multimodal
Final overall_score = 0.5·technical + 0.25·communication + 0.25·confidence.
The orchestrator's adaptation rule:
- last
correctness_score < 40→ drop difficulty by 1, switch to a foundational question, inject brief encouragement - last
correctness_score > 80→ raise difficulty by 1, ask system-design / edge-case - type mix enforced: at least one behavioural in 6, no consecutive duplicates
- Gemini 2.5 Flash for every reasoning call — free tier, sub-second JSON. Single provider keeps prompts unified.
- Deepgram Nova-2 streaming over Whisper — Whisper is batch-only; Deepgram gives interim results, word-level confidence, and
filler_words: truefor hesitation scoring. - Heuristic CV in browser (
FaceDetector+ brightness/motion) — explicitly allowed by the problem statement ("Simplified CV models acceptable, heuristics-based scoring"). Zero server cost, no WASM download. Falls back to motion-only ifFaceDetectoris missing (Firefox). - Convex datastore — resumes, ranked jobs, and interview sessions/reports persist in Convex, scoped by an anonymous browser id (localStorage). Its reactive queries keep the active interview in sync without polling, and completed runs surface as dashboard history.
- No auth — an anonymous per-browser id replaces accounts; the single-user demo flow does not need login.
- Heuristic CV is coarser than MediaPipe FaceMesh; swapping in
@mediapipe/tasks-visionwould give iris landmarks for true eye-contact detection. - Deepgram key is shipped to the browser. Production should mint short-lived keys via
keys/short-lived. - Convex history is keyed by an anonymous browser id (no real auth), so it doesn't follow a user across devices — adding accounts would fix that.
After parsing a backend-leaning resume the system typically returns:
{
"seniority": "junior",
"domain": "backend",
"skills": { "strong": ["Node.js","PostgreSQL","REST"], "moderate": ["Docker"], "weak": ["distributed systems","caching"] },
"inferred_roles": [
{ "title": "Junior Backend Engineer", "confidence": 86, "fit_reason": "Strong evidence of Node + Postgres in 2 shipped projects", ... },
{ "title": "Backend Software Engineer (FinTech)", "confidence": 72, ... }
]
}A typical report after a 7-question interview:
{
"overall_score": 71,
"hire_recommendation": "yes",
"top_3_improvements": [
"Quantify project impact — 'reduced query latency from 800ms to 120ms' lands harder than 'improved performance'",
"Cut filler words; you averaged 11/answer — the longest pause was 3.2s",
"Address eye contact — composure dropped to 0.41 on system-design questions"
]
}┌────────── browser (Next.js client) ──────────┐
│ ResumeUpload InterviewStudio Report │
│ │ │ ▲ │
│ │ │ │ live signals │
│ │ ▼ │ │
│ │ useDeepgramStream useVisualMetrics│
│ │ (WebSocket) (Canvas + FaceDetector)
└───────┼──────────────┼────────────────────────┘
│ │
▼ ▼
┌──────────────── Next.js API routes ──────────┐
│ /resume/parse /interview/question │
│ /jobs/recommend /interview/evaluate │
│ /deepgram/token /interview/report │
└────┬──────────────┬────────────┬─────────────┘
│ │ │
▼ ▼ ▼
Gemini Deepgram JSearch
2.5 Flash Nova-2 (RapidAPI)
MIT — built for hackathon evaluation.