Repository navigation
Add deterministic + AI-powered role fit analysis #80
Description
Add deterministic + AI-powered role fit analysis
Goal
Use the candidate's profile/CV and the job requirements to estimate how well the candidate fits a role.
The system should provide:
- A deterministic fit score as the baseline/fallback.
- An optional AI-powered fit analysis as an enhancement.
- Clear explanations for why the candidate was classified as a good, partial, or poor fit.
The deterministic score must work without AI so that every supported job can receive a fit analysis.
Inputs
Use:
- Profile/CV
- Years of experience
- Job description
- Required experience
- Required skills
- Preferred skills
- Role/seniority
- Required qualifications
- Other relevant job requirements
1. Deterministic fit score
Before using AI, calculate a deterministic score based on structured requirements and keyword/skill matching.
Skills matching
Compare the candidate's skills against skills mentioned in the job description.
- Required skill is present in candidate skills → strong positive signal.
- Preferred skill is present → smaller positive signal.
- Required skill is missing → negative signal.
- Skill appearing only in the job description must not be assumed to be possessed by the candidate.
Normalize obvious variations/synonyms where possible, for example:
React.js↔ReactPostgreSQL↔PostgresJavaScript↔JS
The deterministic calculation must be explainable and reproducible.
Experience matching
Compare the candidate's years of experience with the job's required experience range.
Examples:
| Job requirement | Candidate | Assessment |
|---|---|---|
| 0–1 years | 1 year | Fits |
| Up to 2 years | 1 year | Fits |
| 2–4 years | 1 year | Almost fits / below target |
| 2–4 years | 3 years | Fits |
| 2–4 years | 5 years | Above target / potentially overqualified |
| 8+ years | 4 years | Significant gap |
Use experience categories such as:
- Fits — candidate is within the expected range.
- Almost fits — candidate is slightly below the target.
- Does not fit — candidate is substantially below the requirement.
- Above target — candidate exceeds the requested experience range.
Example:
Job requires:
2–4 years
Candidate:1 year
Result:Almost fits — Experience below target range
Deterministic scoring
Calculate a score from 0–100 using multiple signals:
- Required skills match
- Preferred skills match
- Experience fit
- Role similarity
- Seniority match
- Required qualifications
- Major missing requirements
A possible initial weighting:
| Factor | Weight |
|---|---|
| Required skills | 40% |
| Experience | 25% |
| Role/seniority similarity | 15% |
| Required qualifications | 10% |
| Preferred skills | 10% |
The exact weighting should be configurable but stable.
Missing a major mandatory requirement should significantly reduce the score and may optionally cap the maximum score.
The deterministic score must not be based exclusively on years of experience.
2. Deterministic fit indicator
Map the deterministic score to a clear visual indicator:
- 🟢 Green — Good fit
- 🟡 Yellow — Partial / questionable fit
- 🔴 Red — Poor fit
Suggested thresholds:
- 80–100: Green
- 50–79: Yellow
- 0–49: Red
Example:
82/100 — 🟢 Good fit
The UI should show the main reasons behind the score.
Example:
Why: 4/5 required skills matched, experience is within the target range, and the candidate has similar role experience. Missing AWS experience prevents a higher score.
3. Optional AI-powered fit analysis
AI analysis should be an optional enhancement, not a dependency.
If AI is enabled, provide the same candidate/job inputs to the model and evaluate:
- Relevant experience, not just total years
- Skill relevance and depth
- Similarity between previous roles and the target role
- Seniority alignment
- Required qualifications
- Major missing requirements
- Transferable experience
- Potential concerns that keyword matching cannot detect
The AI should return:
- AI fit score:
0–100 - Green/Yellow/Red classification
- Main reasons
- Major gaps
- Optional strengths
AI must not invent experience, skills, qualifications, or achievements that are not supported by the candidate's profile/CV.
4. Show both scores when AI is enabled
When AI analysis is available, display both assessments separately.
Do not silently replace the deterministic score with the AI score.
Example:
Deterministic score: 72/100 — 🟡 Partial fit
AI score: 78/100 — 🟡 Partial fitMain reasons:
- ✅ 4/5 required skills matched
⚠️ 1 year experience vs. required 2–4 years- ✅ Strong role similarity
- ❌ Missing AWS experience
- ✅ Relevant transferable experience identified by AI
The deterministic score remains the fallback when AI is unavailable.
5. AI unavailable fallback
If AI is unavailable, fails, times out, or is disabled, the system must still provide the deterministic analysis.
Example:
72/100 — 🟡 Partial fit
AI analysis unavailable. Showing deterministic fit analysis.
AI failure must never prevent the user from receiving a fit score.
6. UI
Every supported job should display a fit analysis.
Example:
Software Engineer
Deterministic fit
🟡 65/100 — Partial fit
Why:
- ✅ 4/5 required skills matched
⚠️ 1 year experience vs. required 2–4 years- ✅ Similar previous role
- ❌ Missing required AWS experience
AI analysis
🟡 72/100 — Partial fit
Strong technical skill overlap and relevant role experience, but the candidate is below the requested experience range and does not demonstrate AWS experience.
7. Important scoring rules
- Fit must not be determined solely by years of experience.
- Required skills should have more weight than preferred skills.
- Missing major mandatory requirements should materially affect the score.
- A candidate with fewer years of experience can still receive a reasonable score if their skills and role experience are highly relevant.
- A candidate with many years of experience should not automatically receive a high score if they lack important required skills.
- Preferred skills should improve the score but should not outweigh major missing required requirements.
- Years of experience should be evaluated against the required range, not as a standalone score.
- Role similarity and seniority should be considered.
- Required qualifications should be considered separately from skills where applicable.
- AI must only use information available in the supplied profile/CV and job description.
- The deterministic score must be reproducible for the same inputs and scoring configuration.
- AI analysis must be optional.
- AI failure or unavailability must not prevent fit analysis.
Acceptance Criteria
- Every supported job can receive a fit analysis without requiring AI.
- A deterministic 0–100 score is calculated from experience, skills, role/seniority, qualifications, and other relevant requirements.
- Deterministic scoring includes keyword/skill matching between the candidate profile and job description.
- Required and preferred skills are weighted differently.
- Experience ranges such as
0–1,up to 2,2–4,5–8, and8+ yearsare handled appropriately. - Experience mismatches are reflected in the score and explanation.
- Experience alone cannot determine the final fit score.
- Missing major required skills/qualifications materially affect the score.
- The deterministic score is mapped to Green/Yellow/Red.
- Users can see the main reasons behind the deterministic classification.
- AI analysis is optional.
- When AI is enabled, users can see the AI assessment separately from the deterministic score.
- When AI is unavailable, the deterministic score continues to work as the fallback.
- AI does not invent candidate skills, experience, qualifications, or achievements.
- The same deterministic inputs and scoring configuration produce the same score.
- Tests cover a clear fit case → Green.
- Tests cover a borderline fit case → Yellow.
- Tests cover a poor fit case → Red.
- Tests cover a candidate below the required experience range.
- Tests cover a candidate within the required experience range.
- Tests cover a candidate above the required experience range.
- Tests cover strong skills but insufficient experience.
- Tests cover strong experience but missing required skills.
- Tests cover missing major mandatory requirements.
- Tests cover AI being unavailable and deterministic fallback being used.
- Tests verify deterministic results are stable for identical inputs.
Activity