feature implemented

[SagiEv]

Role Fit Analysis Implementation Plan

This plan outlines the architecture and steps to implement the deterministic and AI-powered role fit analysis, integrated across applications, details, and analytics.

User Review Required

Important

Please review the updated deterministic logic, which now relies on smart keyword inference (Seniority Mapping) to handle edge cases like missing explicit experience requirements. I have also added the necessary migration, backfill scripts, and the detailed unit test plan based on your provided Job Descriptions.

Proposed Changes

1. Database Schema Updates & Migrations

• applications table updates:
  • Add fit_score_deterministic (Integer, 0-100). (Note: Color rendering is handled dynamically by the frontend based on this score).
  • Add fit_analysis_ai (JSONB) to store the optional AI report results (score, percentages, short explanation).

Supabase Migration Script (backend/migrations/003_add_fit_analysis.sql):

-- Add deterministic score column
ALTER TABLE applications 
ADD COLUMN fit_score_deterministic INTEGER CHECK (fit_score_deterministic >= 0 AND fit_score_deterministic <= 100);

-- Add AI analysis JSONB column
ALTER TABLE applications 
ADD COLUMN fit_analysis_ai JSONB;

-- Create an index to speed up analytics queries filtering by fit score
CREATE INDEX idx_applications_fit_score ON applications(fit_score_deterministic);

2. Backend Service (Node.js): Advanced Deterministic Calculation

Goal: A fast, clever, precise, and highly maintainable reproducible function to calculate a 0-100 fit score without AI.

• Create backend/services/fitAnalysis.service.js containing calculateDeterministicFit(candidate, jdText).
• Create backend/config/fitRules.json (or constant map) to store seniority mappings and synonyms.
• Smart Experience parsing heuristics:
  1. Explicit Search: Parse explicit required experience using robust Regex (e.g., 2-5 years, 2+ years).
  2. Implicit Fallback (Seniority Inference): If explicit years are missing, search for role/seniority keywords in the JD to infer the required range:
     • Staff / Principal / Architect: Target 7+ years.
     • Team Lead / Manager: Target 5+ years.
     • Senior: Target 4-5+ years.
     • Junior / Entry Level / Graduate / No Seniority keyword: Target 0-2 years (meaning a candidate with 5+ years might be flagged as overqualified/above target).
  3. Compare candidate's total experience against this resolved target range.
• Skills & Role Matching:
  • Retrieve candidate skills from skills table, profile text, and projects tech stack.
  • Match known skills and role keywords against the info text using a synonym dictionary.
• Scoring Logic:
  • Base calculation on weighted components (e.g., Experience match, Skill match density).
  • Deduct points for missing critical/implied seniority targets.

3. Backend Service (Python/FastAPI): AI-Powered Analysis

Goal: Deep analysis using Groq/LLM as an optional enhancement.

• Create a new FastAPI endpoint (e.g., POST /analyze-fit).
• Input: Candidate Data (JSON) and Job Description (text).
• Prompt: Instruct the model to evaluate the fit and return structured JSON.
• Output (Structured JSON):
  • AI Score (0-100)
  • Short summary explaining fit (concise, not long text)
  • Percentage matches (e.g., Skills Match: 80%, Experience Match: 60%).

4. Triggering Analysis on Application Creation

• Update backend/controllers/applications.controller.js (createApplication method).
• On application creation:
  1. Automatically run the clever deterministic fit analysis.
  2. Save fit_score_deterministic to the database.
  3. Check user settings (app_settings table) to see if AI fit analysis is enabled and which model to use.
  4. If enabled, fire the AI analysis asynchronously in the background and save the result to fit_analysis_ai once completed.

5. Frontend UI: Settings, Applications Table & Detail Page

• SettingsPage.jsx:
  • Update the "AI Feature Routing" section to include a new configuration block for Job Fit Analysis. This allows users to optionally toggle AI fit analysis and select which model (Groq, OpenAI, Claude, Gemini) should be used.
• ApplicationsPage.jsx (Table View):
  • Add a new column "Fit".
  • Dynamically calculate and display a colored indicator (Green 🟢, Yellow 🟡, Red 🔴) based purely on the fit_score_deterministic integer from the DB.
• ApplicationDetailPage.jsx:
  • Create a RoleFitAnalysis component.
  • Show the deterministic score as a baseline.
  • If fit_analysis_ai exists, render the concise AI report.

6. Analytics Integration

• AnalyticsPage.jsx:
  • Use the saved fit_score_deterministic and fit_analysis_ai data in the analytics queries.
  • Create charts exploring trends (e.g., "Interview Rate vs. Fit Score", "Most common missing skills in Yellow/Red matches").

7. Production Backfill Script

We will create a Node.js script backend/scripts/backfillFitScores.js to retroactively calculate and save the deterministic fit score for all existing applications on the production server.

// backend/scripts/backfillFitScores.js
require('dotenv').config();
const { createClient } = require('@supabase/supabase-js');
const fitAnalysisService = require('../services/fitAnalysis.service');

const supabase = createClient(process.env.SUPABASE_URL, process.env.SUPABASE_SERVICE_ROLE_KEY);

async function backfill() {
  console.log('Starting backfill of deterministic fit scores...');
  
  // 1. Fetch all users
  const { data: users, error: usersErr } = await supabase.from('app_settings').select('user_id');
  if (usersErr) throw usersErr;

  for (const { user_id } of users) {
    console.log(`Processing user ${user_id}`);
    
    // 2. Fetch candidate profile and skills
    const { data: profile } = await supabase.from('profile').select('*').single(); // Adjust based on user context
    const { data: skills } = await supabase.from('skills').select('*');
    // ... Fetch projects and experiences similarly ...
    
    const candidateData = { profile, skills /*, projects, experiences */ };

    // 3. Fetch applications missing deterministic score
    const { data: apps, error: appsErr } = await supabase
      .from('applications')
      .select('id, info')
      .is('fit_score_deterministic', null);
      
    if (appsErr || !apps) continue;
    
    for (const app of apps) {
      if (!app.info) continue; // Skip if no JD text
      
      const score = fitAnalysisService.calculateDeterministicFit(candidateData, app.info);
      
      await supabase
        .from('applications')
        .update({ fit_score_deterministic: score })
        .eq('id', app.id);
        
      console.log(`Updated app ${app.id} with score ${score}`);
    }
  }
  
  console.log('Backfill complete!');
}

backfill().catch(console.error);

Verification Plan & Unit Tests

Backend Unit Tests (Jest)

We will create backend/tests/fitAnalysis.test.js to thoroughly test the deterministic engine using real-world Job Descriptions.

Test Cases to Implement:

1. Skai - AI Engineer (Explicit Range):
   • JD Snippet: "2–5 years of experience in software development."
   • Test Expectation: Regex successfully extracts [2, 5] years. Candidate with 3 years gets max experience points. Candidate with 0 years gets heavily penalized.
2. Glassix - Junior Fullstack Developer (Implicit Seniority):
   • JD Snippet: Mentions "Junior Fullstack Developer" but no explicit years.
   • Test Expectation: Fallback triggers "Junior" logic targeting 0-2 years. A candidate with 6 years gets penalized as overqualified; a candidate with 1 year fits perfectly.
3. Appdome - Software Engineer (Explicit Minimum):
   • JD Snippet: "2+ years of experience developing backend systems in Python or Java"
   • Test Expectation: Regex extracts minimum 2 years (e.g., [2, 99]). Candidate with 1 year is borderline, candidate with 3 years fits perfectly.

How to Run Tests Locally

Before pushing to staging, run the following commands in your terminal to execute the unit tests and ensure the deterministic parsing works perfectly on the provided sample JDs:

cd backend
npm run test tests/fitAnalysis.test.js

(Ensure all assertions pass and the Regex handles variations like 2–5 vs 2-5 vs 2+)

Manual Verification

• Execute backfillFitScores.js on staging and verify applications get scores.
• Add a new application with a job description.
• Verify the Applications Table shows the color indicator computed by the frontend.
• Open the Application Details page and verify the AI report (if enabled).

Job Fit Analysis Feature Walkthrough

The Job Fit Analysis feature has been fully implemented, providing both a deterministic baseline score based on keyword matching and an optional AI-powered analysis to give deep insights into how well your profile matches a job description.

1. AI Feature Routing Configuration

You can now toggle and configure the AI provider for Job Fit Analysis in the Settings page under the "AI Integration" tab.

• It uses the standard fallback to groq if enabled and no provider is specified.
• The toggle allows you to disable AI analysis if you only want the deterministic score to save API credits.

2. Deterministic & AI Scoring

When a new application is added with a job description:

• A Deterministic Fit Score (0-100) is calculated instantly by comparing your profile, skills, and experience with the keywords, required years of experience, and seniority level derived from the job description.
• An AI-Powered Analysis is fired asynchronously (if enabled) in the background to provide a detailed explanation of why the role is a good, partial, or poor fit.

3. Applications Table Integration

The Applications Table now includes a "Fit" column:

• 🟢 Green dot for Good Fit (≥ 80)
• 🟡 Yellow dot for Partial Fit (50 - 79)
• 🔴 Red dot for Poor Fit (< 50)
• Hovering over the dot reveals the exact deterministic score.

4. Application Details Page

When viewing a specific application, a new Role Fit Analysis card has been added.

• It displays the overall score and a badge indicating the fit level.
• It shows the detailed AI analysis explaining the match.
• If AI analysis is missing, it explains that the score is a deterministic baseline.

5. Analytics Integration

The Analytics page now includes a Job Fit Insights section:

• Avg. Fit Score: The overall average fit score across all jobs.
• Interviews Avg. Fit: The average fit score of jobs that led to an interview.
• Rejections Avg. Fit: The average fit score of jobs that led to a rejection.
• A breakdown of how many jobs fell into Good, Partial, and Poor fit categories.

6. Backfill Script

A script backend/scripts/backfillFitScores.js has been created. Running this script via Node on your production server will calculate the deterministic score for all existing applications that have a job description but no score yet.

cd backend
node scripts/backfillFitScores.js

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
job-pilot	Ready	Preview	Sep 13, 2026 5:21pm UTC

[SagiEv]

Make Fit Scoring Stricter (But Reasonable)

The goal of this change is to prevent candidates with 0 years of experience from scoring "High Yellow" (68% - 75%) on roles that strictly require Senior-level experience (4+ years), while still rewarding them reasonably if they possess a perfect tech-stack match.

Open Questions

Before we proceed, please review the proposed logic below and confirm if you agree with the penalties.

Important

Does the "Hard Cap" rule for the Deterministic algorithm (Max 65% if you lack the minimum experience) sound fair to you? Or would you prefer it just scales down mathematically without a hard ceiling?

Proposed Changes

Backend (Node.js) - Deterministic Math

We will update the calculateDeterministicFit logic to implement a progressive penalty system for missing experience, rather than a flat 20% score.

[MODIFY] backend/services/fitAnalysis.service.js

• Progressive Experience Penalty:
  • If candidate is 0-1 years below target: expScore = 50
  • If candidate is 1-2 years below target: expScore = 20
  • If candidate is > 2 years below target: expScore = 0 (Currently this is 20)
• Score Ceiling (Hard Cap):
  • If the candidate's total experience is strictly less than the minimum required by the JD, their maximum possible total score is capped at 65% (Partial Fit). This prevents a 100% skill match from carrying a 0-year candidate to a 70%+ score on a 4+ year requirement.

Backend (Python) - AI Prompt

Large Language Models need explicit boundaries, or they will hallucinate "Experience Match" based on how many skills match.

[MODIFY] backend/ai_service/role_fit/router.py

• Update PROMPT_TEMPLATE with a strict scoring rubric:
  • Isolate Experience from Skills: Explicitly instruct the AI that "Experience Match" must be scored purely on tenure/years, and must not be inflated by a good skill match.
  • Mathematical Penalty: Instruct the AI: "If the JD requires X years, and the candidate has Y years (where Y < X), aggressively penalize the Experience Match. If the gap is > 2 years, Experience Match should be 0-10%."
  • Overall Score Guardrails: Add a rule: "If a candidate is severely underqualified in tenure (e.g., 0 years for a Mid/Senior role), the ai_score should reflect a 'Poor' or 'Partial' fit (Max 65-70), regardless of how well their skills match."

Verification Plan

Automated Tests

• Update backend/tests/fitAnalysis.test.js (if it exists) to verify that a candidate with 0 years applying for a 4+ year role cannot score above 65 deterministically.

Manual Verification

• Re-run the scratch_wix.js script with the 0-year candidate on the Wix JD and verify the score drops to a reasonable level (e.g., ~30-40% deterministic, ~40-50% AI).