You are the incoming Senior Data Scientist at RetailX AI, a £2.4B UK grocery and general merchandise retailer operating 750 stores and an e-commerce platform with 2.3 million weekly orders.
The board has given the AI division 12 weeks to demonstrate a credible, production-ready enterprise AI transformation — or they will outsource the function entirely.
You have inherited messy data, incomplete pipelines, ungoverned GenAI pilots, and zero models in production. Your job is to fix all of it.
| Problem | Baseline | Target | Value |
|---|---|---|---|
| Customer churn | 23.4% annual | 13.4% | £192M ARR at risk |
| Weekly stock-outs | 18.7% of SKUs | 3.7% | £3.2M holding cost reduction |
| Average basket value | £34.20 | £36.94 | +8% revenue uplift |
| Fraud / refund losses | 2.3% of transactions | 0.3% | £4.1M annual loss reduction |
| Support cost per contact | £8.40 | £6.30 | 25% cost reduction via RAG |
| Phase | Title | Key Deliverable |
|---|---|---|
| 1 | Onboarding | Environment setup, repo familiarisation |
| 2 | Business Discovery & AI Strategy | BRD, AI strategy, stakeholder register, roadmap |
| 3 | Enterprise Data Architecture | PostgreSQL star schema, ERD, ADRs, repo standards |
| 4 | PySpark Data Engineering | ETL pipeline, Great Expectations validation, data quality report |
| 5 | Statistics & Experimentation | A/B test framework, causal inference, statistical report |
| 6 | Customer Intelligence & Churn | Segmentation, CLV model, churn model (AUC 0.84), SHAP |
| 7 | Demand Forecasting & Optimisation | Prophet forecast (MAPE 11.8%), inventory optimisation, Power BI |
| 8 | Recommendations & Knowledge Graph | Hybrid recommender, Neo4j, FastAPI endpoint |
| 9 | Fraud & Real-Time Risk | Anomaly detection, fraud API (precision 0.87), runbook |
| 10 | Bayesian Marketing Mix Model | PyMC MMM, channel ROI (TV £3.40/£1), budget recommendation |
| 11 | GenAI, RAG & LLM Ops | RAG assistant, LLM observability, hallucination testing, LLM card |
| 12 | MLOps, Security & Responsible AI | MLflow registry, Docker, CI/CD, drift monitoring, responsible AI |
| 13 | Board Defence | Executive deck, ROI model, cloud cost estimate, interview defence |
| Category | Tools |
|---|---|
| Language | Python 3.10 |
| Database | PostgreSQL + DBeaver |
| Data Engineering | PySpark, Great Expectations, Pandera |
| ML | Scikit-learn, XGBoost, LightGBM, SHAP, Prophet, PyMC |
| MLOps | MLflow, Docker, GitHub Actions, Evidently AI |
| APIs | FastAPI |
| Apps | Streamlit Community Cloud |
| Dashboards | Power BI Desktop |
| Graph | Neo4j Community Edition |
| GenAI | LangChain, ChromaDB, Hugging Face |
| Testing | pytest (unit / integration / API / model / LLM eval) |
| Diagrams | Draw.io |
| Version Control | Git + GitHub |
RetailX-AI-Transformation/
├── .github/workflows/ # CI/CD pipelines
├── .vscode/ # Editor config
├── config/ # Environment configuration
├── data/ # raw → staging → curated
├── docs/ # All documentation
│ ├── business/ # BRD, AI strategy, user stories
│ ├── architecture/ # ERD, ADRs, data contracts
│ ├── governance/ # Responsible AI, security, LLM governance
│ ├── executive/ # Board deck, ROI model, reports
│ ├── interview-defence/ # STAR answers, technical deep dives
│ ├── model-cards/ # Churn, fraud, recommendation, LLM
│ └── runbooks/ # Fraud, monitoring, incident response
├── notebooks/ # Phase-by-phase Jupyter notebooks
├── src/ # Production Python source
│ ├── config/ # Settings management
│ ├── logging/ # Structured logging
│ ├── utils/ # Shared utilities
│ ├── ingestion/ # PySpark ETL
│ ├── features/ # Feature engineering
│ ├── models/ # ML models
│ ├── forecasting/ # Demand forecasting
│ ├── recommendations/ # Recommendation engine
│ ├── fraud/ # Fraud detection
│ └── genai/ # RAG pipeline
├── api/ # FastAPI endpoints
├── streamlit/ # Streamlit applications
├── docker/ # Dockerfiles
├── sql/ # DDL, DML, analytics queries
├── tests/ # Full test suite
└── assets/ # Screenshots, exported reports
# 1. Clone the repository
git clone https://github.com/alain-Sortnext/RetailX-AI-Transformation.git
cd RetailX-AI-Transformation
# 2. Set up environment
conda env create -f environment.yml
conda activate retailx-ai
# 3. Configure environment variables
cp .env.example .env
# Edit .env with your local settings
# 4. Start services
docker-compose up -d
# 5. Run tests to confirm setup
pytest tests/ -vSee PROJECT_SETUP.md for full setup instructions.
- Branching:
main→develop→feature/phase-N-description - Commits: Conventional commits format (
feat:,fix:,docs:,test:,chore:) - PRs: Minimum 1 reviewer, all CI checks must pass, linked to GitHub Issue
- Issues: One issue per phase task — use the provided templates
- Code review: Minimum 10 code reviews documented across the project
See CONTRIBUTING.md for full standards.
All work must comply with:
- UK GDPR Article 22 — automated decision-making must have human review
- Data Protection Act 2018 — UK-specific implementation
- ICO AI Guidance (July 2023) — transparency, accountability, fairness
- ICO Explaining Decisions Made with AI — model explanations required
See docs/governance/responsible-ai-framework.md (produced in Phase 12).
Before the board presentation (Week 12), all 31 items in SUBMISSION_CHECKLIST.md must be ✅.
Built with Project Lab by SortNext — https://sortnext.com