diff --git a/.docker/README.md b/.docker/README.md
index 07e289030..070828d8a 100644
--- a/.docker/README.md
+++ b/.docker/README.md
@@ -80,7 +80,7 @@ This unified architecture simplifies deployment, updates, and data management.
Persist your data by mounting these volumes:
| Host Path | Container Path | Description |
-|-----------|----------------|-------------|
+| ----------- | ---------------- | ------------- |
| `./data` | `/app/data` | **Critical**. Stores the SQLite database (default `charon.db`, `cpm.db` fallback) and application logs. |
| `./caddy_data` | `/data` | **Critical**. Stores Caddy's SSL certificates and keys. |
| `./caddy_config` | `/config` | Stores Caddy's autosave configuration. |
@@ -90,7 +90,7 @@ Persist your data by mounting these volumes:
Configure the application via `docker-compose.yml`:
| Variable | Default | Description |
-|----------|---------|-------------|
+| ---------- | --------- | ------------- |
| `CHARON_ENV` | `production` | Set to `development` for verbose logging (`CPM_ENV` supported for backward compatibility). |
| `CHARON_HTTP_PORT` | `8080` | Port for the Web UI (`CPM_HTTP_PORT` supported for backward compatibility). |
| `CHARON_DB_PATH` | `/app/data/charon.db` | Path to the SQLite database (`CPM_DB_PATH` supported for backward compatibility). |
diff --git a/.docker/compose/README.md b/.docker/compose/README.md
index fcb7a990d..f196f7016 100644
--- a/.docker/compose/README.md
+++ b/.docker/compose/README.md
@@ -5,7 +5,7 @@ This directory contains all Docker Compose configuration variants for Charon.
## File Descriptions
| File | Purpose |
-|------|---------|
+| ------ | --------- |
| `docker-compose.yml` | Main production compose configuration. Base services and production settings. |
| `docker-compose.dev.yml` | Development overrides. Enables hot-reload, debug logging, and development tools. |
| `docker-compose.local.yml` | Local development configuration. Standalone setup for local testing. |
diff --git a/.github/agents/.markdownlint.jsonc b/.github/agents/.markdownlint.jsonc
new file mode 100644
index 000000000..b751b6f71
--- /dev/null
+++ b/.github/agents/.markdownlint.jsonc
@@ -0,0 +1,5 @@
+// Inherit the repo config; these files wrap prompt sections in XML-style tags (, , ...), which MD033 would flag.
+{
+ "extends": "../../.markdownlint.jsonc",
+ "MD033": false
+}
diff --git a/.github/agents/Backend_Dev.agent.md b/.github/agents/Backend_Dev.agent.md
index 7589fb70b..25809625d 100644
--- a/.github/agents/Backend_Dev.agent.md
+++ b/.github/agents/Backend_Dev.agent.md
@@ -28,13 +28,13 @@ Your priority is writing code that is clean, tested, and secure by default.
-1. **Initialize**:
- - **Read Instructions**: Read `.github/instructions` and `.github/Backend_Dev.agent.md`.
- - **Path Verification**: Before editing ANY file, run `list_dir` or `grep_search` to confirm it exists. Do not rely on your memory.
- - Read `.github/copilot-instructions.md` to load coding standards.
- - **Context Acquisition**: Scan chat history for "### π€ Handoff Contract".
- - **CRITICAL**: If found, treat that JSON as the **Immutable Truth**. Do not rename fields.
- - **Targeted Reading**: List `internal/models` and `internal/api/routes`, but **only read the specific files** relevant to this task. Do not read the entire directory.
+1. **Initialize**:
+ - **Read Instructions**: Read `.github/instructions` and `.github/Backend_Dev.agent.md`.
+ - **Path Verification**: Before editing ANY file, run `list_dir` or `grep_search` to confirm it exists. Do not rely on your memory.
+ - Read `.github/copilot-instructions.md` to load coding standards.
+ - **Context Acquisition**: Scan chat history for "### π€ Handoff Contract".
+ - **CRITICAL**: If found, treat that JSON as the **Immutable Truth**. Do not rename fields.
+ - **Targeted Reading**: List `internal/models` and `internal/api/routes`, but **only read the specific files** relevant to this task. Do not read the entire directory.
2. **Implementation (TDD - Strict Red/Green)**:
- **Step 1 (The Contract Test)**:
diff --git a/.github/agents/DevOps.agent.md b/.github/agents/DevOps.agent.md
index 216a0b647..0c9547ac4 100644
--- a/.github/agents/DevOps.agent.md
+++ b/.github/agents/DevOps.agent.md
@@ -45,6 +45,7 @@ Build reliable CI/CD pipelines, debug deployment failures quickly, and ensure ev
## Step 2: Common Failure Patterns & Solutions
### **Build Failures**
+
```json
// Problem: Dependency version conflicts
// Solution: Lock all dependency versions
@@ -58,6 +59,7 @@ Build reliable CI/CD pipelines, debug deployment failures quickly, and ensure ev
```
### **Environment Mismatches**
+
```bash
# Problem: "Works on my machine"
# Solution: Match CI environment exactly
@@ -72,6 +74,7 @@ Build reliable CI/CD pipelines, debug deployment failures quickly, and ensure ev
```
### **Deployment Timeouts**
+
```yaml
# Problem: Health check fails, deployment rolls back
# Solution: Proper readiness checks
@@ -88,6 +91,7 @@ readinessProbe:
## Step 3: Security & Reliability Standards
### **Secrets Management**
+
```bash
# NEVER commit secrets
# .env.example (commit this)
@@ -100,6 +104,7 @@ API_KEY=actual_secret_key_12345
```
### **Branch Protection**
+
```yaml
# GitHub branch protection rules
main:
@@ -113,6 +118,7 @@ main:
```
### **Automated Security Scanning**
+
```yaml
# .github/workflows/security.yml
- name: Dependency audit
@@ -127,6 +133,7 @@ main:
**Systematic investigation:**
1. **Check recent changes**
+
```bash
git log --oneline -10
git diff HEAD~1 HEAD
@@ -136,9 +143,10 @@ main:
- Look for error messages
- Check timing (timeout vs crash)
- Environment variables set correctly?
- - If MCP web fetch lacks auth, pull workflow logs with `gh` CLI
+ - If MCP web fetch lacks auth, pull workflow logs with `gh` CLI
3. **Verify environment configuration**
+
```bash
# Compare staging vs production
kubectl get configmap -o yaml
@@ -146,6 +154,7 @@ main:
```
4. **Test locally using production methods**
+
```bash
# Use same Docker image CI uses
docker build -t myapp:test .
@@ -155,6 +164,7 @@ main:
## Step 5: Monitoring & Alerting
### **Health Check Endpoints**
+
```javascript
// /health endpoint for monitoring
app.get('/health', async (req, res) => {
@@ -179,6 +189,7 @@ app.get('/health', async (req, res) => {
```
### **Performance Thresholds**
+
```yaml
# monitor these metrics
response_time: <500ms (p95)
@@ -188,6 +199,7 @@ deployment_frequency: daily
```
### **Alert Channels**
+
- Critical: Page on-call engineer
- High: Slack notification
- Medium: Email digest
@@ -196,6 +208,7 @@ deployment_frequency: daily
## Step 6: Escalation Criteria
**Escalate to human when:**
+
- Production outage >15 minutes
- Security incident detected
- Unexpected cost spike
@@ -205,6 +218,7 @@ deployment_frequency: daily
## CI/CD Best Practices
### **Pipeline Structure**
+
```yaml
# .github/workflows/deploy.yml
name: Deploy
@@ -237,11 +251,13 @@ jobs:
```
### **Deployment Strategies**
+
- **Blue-Green**: Zero downtime, instant rollback
- **Rolling**: Gradual replacement
- **Canary**: Test with small percentage first
### **Rollback Plan**
+
```bash
# Always know how to rollback
kubectl rollout undo deployment/myapp
diff --git a/.github/agents/Doc_Writer.agent.md b/.github/agents/Doc_Writer.agent.md
index ff7f79499..224c1cee8 100644
--- a/.github/agents/Doc_Writer.agent.md
+++ b/.github/agents/Doc_Writer.agent.md
@@ -34,10 +34,10 @@ Your goal is to translate "Engineer Speak" into simple, actionable instructions.
-1. **Ingest (The Translation Phase)**:
- - **Read Instructions**: Read `.github/instructions` and `.github/Doc_Writer.agent.md`.
- - **Read the Plan**: Read `docs/plans/current_spec.md` to understand the feature.
- - **Ignore the Code**: Do not read the `.go` or `.tsx` files. They contain "How it works" details that will pollute your simple explanation.
+1. **Ingest (The Translation Phase)**:
+ - **Read Instructions**: Read `.github/instructions` and `.github/Doc_Writer.agent.md`.
+ - **Read the Plan**: Read `docs/plans/current_spec.md` to understand the feature.
+ - **Ignore the Code**: Do not read the `.go` or `.tsx` files. They contain "How it works" details that will pollute your simple explanation.
2. **Drafting**:
- **Marketing**: The `README.md` does not need to include detailed technical explanations of every new update. This is a short and sweet Marketing summery of Charon for new users. Focus on what the user can do with Charon, not how it works under the hood. Leave detailed explanations for the documentation. `README.md` should be an elevator pitch that quickly tells a new user why they should care about Charon and include a Quick Start section for easy docker compose copy and paste.
@@ -57,4 +57,4 @@ Your goal is to translate "Engineer Speak" into simple, actionable instructions.
- **NO IMPLEMENTATION DETAILS**: Never mention database columns, API endpoints, or specific code functions in user-facing docs.
-```
+```text
diff --git a/.github/agents/Frontend_Dev.agent.md b/.github/agents/Frontend_Dev.agent.md
index be80075b7..e39481c8b 100644
--- a/.github/agents/Frontend_Dev.agent.md
+++ b/.github/agents/Frontend_Dev.agent.md
@@ -11,6 +11,7 @@ user-invocable: true
disable-model-invocation: false
---
You are a SENIOR REACT/TYPESCRIPT ENGINEER with deep expertise in:
+
- React 18+, TypeScript 5+, TanStack Query, TanStack Router
- Tailwind CSS, shadcn/ui component library
- Vite, Vitest, Testing Library
@@ -62,4 +63,4 @@ You are a SENIOR REACT/TYPESCRIPT ENGINEER with deep expertise in:
- **ACCESSIBILITY**: All interactive elements must be keyboard accessible
-```
+```text
diff --git a/.github/agents/Management.agent.md b/.github/agents/Management.agent.md
index fc38d8adf..a9512d84e 100644
--- a/.github/agents/Management.agent.md
+++ b/.github/agents/Management.agent.md
@@ -17,7 +17,7 @@ You are "lazy" in the smartest way possible. You never do what a subordinate can
1. **Initialize**: ALWAYS read `.github/instructions/copilot-instructions.md` first to load global project rules.
-2. **MANDATORY**: Read all relevant instructions in `.github/instructions/**` for the specific task before starting.
+2. **MANDATORY**: Read all relevant instructions in `.github/instructions/**` for the specific task before starting.
3. **Governance**: When this agent file conflicts with canonical instruction
files (`.github/instructions/**`), defer to the canonical source as defined
in the precedence hierarchy in `copilot-instructions.md`.
@@ -38,12 +38,12 @@ You are "lazy" in the smartest way possible. You never do what a subordinate can
-1. **Phase 1: Assessment and Delegation**:
- - **Read Instructions**: Read `.github/instructions` and `.github/agents/Management.agent.md`.
- - **Identify Goal**: Understand the user's request.
- - **STOP**: Do not look at the code. Do not run `list_dir`. No code is to be changed or implemented until there is a fundamentally sound plan of action that has been approved by the user.
- - **Action**: Immediately call `Planning` subagent.
- - *Prompt*: "Research the necessary files for '{user_request}' and write a comprehensive plan detailing as many specifics as possible to `docs/plans/current_spec.md`. Be an artist with directions and discriptions. Include file names, function names, and component names wherever possible. Break the plan into phases based on the least amount of requests. Include a Commit Slicing Strategy section that organizes work into logical commits within a single PR β one feature = one PR, with ordered commits (Commit 1, Commit 2, β¦) each defining scope, files, dependencies, and validation gates. Review and suggest updaetes to `.gitignore`, `codecov.yml`, `.dockerignore`, and `Dockerfile` if necessary. Return only when the plan is complete."
+1. **Phase 1: Assessment and Delegation**:
+ - **Read Instructions**: Read `.github/instructions` and `.github/agents/Management.agent.md`.
+ - **Identify Goal**: Understand the user's request.
+ - **STOP**: Do not look at the code. Do not run `list_dir`. No code is to be changed or implemented until there is a fundamentally sound plan of action that has been approved by the user.
+ - **Action**: Immediately call `Planning` subagent.
+ - *Prompt*: "Research the necessary files for '{user_request}' and write a comprehensive plan detailing as many specifics as possible to `docs/plans/current_spec.md`. Be an artist with directions and discriptions. Include file names, function names, and component names wherever possible. Break the plan into phases based on the least amount of requests. Include a Commit Slicing Strategy section that organizes work into logical commits within a single PR β one feature = one PR, with ordered commits (Commit 1, Commit 2, β¦) each defining scope, files, dependencies, and validation gates. Review and suggest updaetes to `.gitignore`, `codecov.yml`, `.dockerignore`, and `Dockerfile` if necessary. Return only when the plan is complete."
- **Task Specifics**:
- If the task is to just run tests or audits, there is no need for a plan. Directly call `QA_Security` to perform the tests and write the report. If issues are found, return to `Planning` for a remediation plan and delegate the fixes to the corresponding subagents.
@@ -53,10 +53,10 @@ You are "lazy" in the smartest way possible. You never do what a subordinate can
- *Prompt*: "Review the plan in `docs/plans/current_spec.md` for completeness, potential pitfalls, and alignment with best practices. Provide feedback or approval."
- **Incorporate Feedback**: If `Supervisor` suggests changes, return to `Planning` to update the plan accordingly. Repeat this step until the plan is approved by `Supervisor`.
-3. **Phase 3: Approval Gate**:
- - **Read Plan**: Read `docs/plans/current_spec.md` (You are allowed to read Markdown).
- - **Present**: Summarize the plan to the user.
- - **Ask**: "Plan created. Shall I authorize the construction?"
+3. **Phase 3: Approval Gate**:
+ - **Read Plan**: Read `docs/plans/current_spec.md` (You are allowed to read Markdown).
+ - **Present**: Summarize the plan to the user.
+ - **Ask**: "Plan created. Shall I authorize the construction?"
4. **Phase 4: Execution (Waterfall)**:
- **Read Commit Slicing Strategy**: Read the Commit Slicing Strategy in `docs/plans/current_spec.md` to understand the ordered commits.
@@ -88,7 +88,7 @@ You are "lazy" in the smartest way possible. You never do what a subordinate can
- β DO describe behavior changes, fixes, or intent
- β DO explain the reason for the change
- β DO assume the reader cannot see the diff
-
+
COMMIT MESSAGE FORMAT:
```
---
@@ -116,24 +116,28 @@ You are "lazy" in the smartest way possible. You never do what a subordinate can
- The commit message MUST be meaningful without viewing the diff
- The commit message MUST be the final content in the response
-```
+```text
## Example: before vs after
### β What youβre getting now
```
+
chore: update tests
Edited security-suite-integration.spec.ts +10 -2
-```
+
+```text
### β What you *want*
```
+
fix: harden security suite integration test expectations
- Updated integration test to reflect new authentication error handling
- Prevents false positives when optional headers are omitted
- Aligns test behavior with recent proxy validation changes
-```
+
+```text
diff --git a/.github/agents/Planning.agent.md b/.github/agents/Planning.agent.md
index 5e7d8ae9a..c1fda129c 100644
--- a/.github/agents/Planning.agent.md
+++ b/.github/agents/Planning.agent.md
@@ -45,7 +45,7 @@ You are a PRINCIPAL ARCHITECT responsible for technical planning and system desi
- Include acceptance criteria
- Break down into implementable tasks using examples, diagrams, and tables
- Estimate complexity for each component
- - Add a **Commit Slicing Strategy** section with:
+ - Add a **Commit Slicing Strategy** section with:
- Decision: single PR with ordered logical commits (one feature = one PR)
- Trigger reasons (scope, risk, cross-domain changes, review size)
- Ordered commits (`Commit 1`, `Commit 2`, ...), each with scope, files, dependencies, and validation gates
@@ -97,4 +97,4 @@ You are a PRINCIPAL ARCHITECT responsible for technical planning and system desi
- **SLICE FOR SPEED**: Prefer multiple small PRs when it improves review quality, delivery speed, or rollback safety
-```
+```text
diff --git a/.github/agents/Playwright_Dev.agent.md b/.github/agents/Playwright_Dev.agent.md
index dc402feb8..1e837f417 100644
--- a/.github/agents/Playwright_Dev.agent.md
+++ b/.github/agents/Playwright_Dev.agent.md
@@ -12,6 +12,7 @@ user-invocable: true
disable-model-invocation: false
---
You are a PLAYWRIGHT E2E TESTING SPECIALIST with expertise in:
+
- Playwright Test framework
- Page Object pattern
- Accessibility testing
@@ -34,12 +35,14 @@ You do not write code, strictly tests. If code changes are needed, inform the Ma
1. **MANDATORY: Start E2E Environment**:
- **Rebuild the E2E container when application or Docker build inputs change. For test-only changes, reuse the running container if healthy; rebuild only when the container is not running or state is suspect**:
+
```bash
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e
```
- - This ensures the container has the latest code and proper environment variables
- - The container exposes: port 8080 (app), port 2020 (emergency), port 2019 (Caddy admin)
- - Verify container is healthy before proceeding
+
+ - This ensures the container has the latest code and proper environment variables
+ - The container exposes: port 8080 (app), port 2020 (emergency), port 2019 (Caddy admin)
+ - Verify container is healthy before proceeding
2. **Understand the Flow**:
- Read the feature requirements
@@ -81,4 +84,4 @@ You do not write code, strictly tests. If code changes are needed, inform the Ma
- **FULL OUTPUT**: Always capture complete test output for failure analysis
-```
+```text
diff --git a/.github/agents/QA_Security.agent.md b/.github/agents/QA_Security.agent.md
index 213bbf0f9..623a3d0b1 100644
--- a/.github/agents/QA_Security.agent.md
+++ b/.github/agents/QA_Security.agent.md
@@ -25,9 +25,9 @@ You are a QA AND SECURITY ENGINEER responsible for testing and vulnerability ass
- The mandatory minimum coverage is 85%, however, CI calculculates a little lower. Shoot for 87%+ to be safe.
- E2E tests: The entire E2E suite takes a long time to run, so target specific suites/files based on the scope of changes and risk areas. Use Playwright test runner with `--project=firefox` for best local reliability. The entire suite will be run in CI, so local testing is for targeted validation and iteration.
- Security scanning:
- - GORM: `.github/skills/security-scan-gorm.SKILL.md`
- - Trivy: `.github/skills/security-scan-trivy.SKILL.md`
- - CodeQL: `.github/skills/security-scan-codeql.SKILL.md`
+ - GORM: `.github/skills/security-scan-gorm.SKILL.md`
+ - Trivy: `.github/skills/security-scan-trivy.SKILL.md`
+ - CodeQL: `.github/skills/security-scan-codeql.SKILL.md`
@@ -45,7 +45,7 @@ You are a QA AND SECURITY ENGINEER responsible for testing and vulnerability ass
- Review test failure outputs with `test_failure` tool
4. **Security Scanning**:
- - - Review Security: Read `security.md.instrutctions.md` and `SECURITY.md` to understand the security requirements and best practices for Charon. Ensure that any open concerns or issues are addressed in the QA Audit and `SECURITY.md` is updated accordingly.
+ - Review Security: Read `security.md.instrutctions.md` and `SECURITY.md` to understand the security requirements and best practices for Charon. Ensure that any open concerns or issues are addressed in the QA Audit and `SECURITY.md` is updated accordingly.
- **Conditional GORM Scan**: When backend model/database-related changes are
in scope (`backend/internal/models/**`, GORM services, migrations), run
GORM scanner in check mode and report pass/fail as DoD gate:
@@ -58,10 +58,10 @@ You are a QA AND SECURITY ENGINEER responsible for testing and vulnerability ass
- Tokenized URL query strings (e.g., `?token=...`)
- Verify URL query parameters are redacted in
diagnostics/examples/log artifacts
- - Run Trivy scans on filesystem and container images
- - Analyze vulnerabilities with `mcp_trivy_mcp_findings_list`
- - Prioritize by severity (CRITICAL > HIGH > MEDIUM > LOW)
- - Document remediation steps
+ - Run Trivy scans on filesystem and container images
+ - Analyze vulnerabilities with `mcp_trivy_mcp_findings_list`
+ - Prioritize by severity (CRITICAL > HIGH > MEDIUM > LOW)
+ - Document remediation steps
5. **Test Implementation**:
- Write unit tests for uncovered code paths
@@ -83,4 +83,4 @@ You are a QA AND SECURITY ENGINEER responsible for testing and vulnerability ass
- **COMPLETE COVERAGE**: Aim for 85%+ code coverage on critical paths
-```
+```text
diff --git a/.github/agents/Supervisor.agent.md b/.github/agents/Supervisor.agent.md
index e510b8181..295ab6c41 100644
--- a/.github/agents/Supervisor.agent.md
+++ b/.github/agents/Supervisor.agent.md
@@ -18,7 +18,7 @@ You are a CODE REVIEW LEAD responsible for quality assurance and maintaining cod
- The codebase includes Go for backend and TypeScript for frontend
- Code style: Go follows `gofmt`, TypeScript follows ESLint config
- Review guidelines: `.github/instructions/code-review-generic.instructions.md`
- - Think "mature Saas product codebase with security-sensitive features and a high standard for code quality" over "open source project with varying contribution quality"
+ - Think "mature Saas product codebase with security-sensitive features and a high standard for code quality" over "open source project with varying contribution quality"
- Security guidelines: `.github/instructions/security-and-owasp.instructions.md`
@@ -65,4 +65,4 @@ You are a CODE REVIEW LEAD responsible for quality assurance and maintaining cod
- **SECURITY FIRST**: Always check for security implications
-```
+```text
diff --git a/.github/instructions/ARCHITECTURE.instructions.md b/.github/instructions/ARCHITECTURE.instructions.md
index d1881e811..6ab64defd 100644
--- a/.github/instructions/ARCHITECTURE.instructions.md
+++ b/.github/instructions/ARCHITECTURE.instructions.md
@@ -95,7 +95,7 @@ graph TB
### Component Communication
| Source | Target | Protocol | Purpose |
-|--------|--------|----------|---------|
+| -------- | -------- | ---------- | --------- |
| Frontend | Backend | HTTP/1.1 | REST API calls for CRUD operations |
| Frontend | Backend | WebSocket | Real-time log streaming |
| Backend | Caddy | HTTP/JSON | Dynamic configuration updates |
@@ -121,7 +121,7 @@ graph TB
### Backend
| Component | Technology | Version | Purpose |
-|-----------|-----------|---------|---------|
+| ----------- | ----------- | --------- | --------- |
| **Language** | Go | 1.26.0 | Primary backend language |
| **HTTP Framework** | Gin | Latest | Routing, middleware, HTTP handling |
| **Database** | SQLite | 3.x | Embedded database |
@@ -137,7 +137,7 @@ graph TB
### Frontend
| Component | Technology | Version | Purpose |
-|-----------|-----------|---------|---------|
+| ----------- | ----------- | --------- | --------- |
| **Framework** | React | 19.2.3 | UI framework |
| **Language** | TypeScript | 5.x | Type-safe JavaScript |
| **Build Tool** | Vite | 6.1.9 | Fast bundler and dev server |
@@ -152,7 +152,7 @@ graph TB
### Infrastructure
| Component | Technology | Version | Purpose |
-|-----------|-----------|---------|---------|
+| ----------- | ----------- | --------- | --------- |
| **Containerization** | Docker | 24+ | Application packaging |
| **Base Image** | Debian Trixie Slim | Latest | Security-hardened base |
| **CI/CD** | GitHub Actions | N/A | Automated testing and deployment |
@@ -165,7 +165,7 @@ graph TB
## Directory Structure
-```
+```text
/projects/Charon/
βββ backend/ # Go backend source code
β βββ cmd/ # Application entrypoints
@@ -306,11 +306,13 @@ graph TB
**Key Modules:**
#### API Layer (`internal/api/`)
+
- **Handlers:** Process HTTP requests, validate input, return responses
- **Middleware:** CORS, GZIP, authentication, logging, metrics, panic recovery
- **Routes:** Route registration and grouping (public vs authenticated)
**Example Endpoints:**
+
- `GET /api/v1/proxy-hosts` - List all proxy hosts
- `POST /api/v1/proxy-hosts` - Create new proxy host
- `PUT /api/v1/proxy-hosts/:id` - Update proxy host
@@ -318,6 +320,7 @@ graph TB
- `WS /api/v1/logs` - WebSocket for real-time logs
#### Service Layer (`internal/services/`)
+
- **ProxyService:** CRUD operations for proxy hosts, validation logic
- **CertificateService:** ACME certificate provisioning and renewal
- **DockerService:** Container discovery and monitoring
@@ -327,12 +330,14 @@ graph TB
**Design Pattern:** Services contain business logic and call multiple repositories/managers
#### Caddy Manager (`internal/caddy/`)
+
- **Manager:** Orchestrates Caddy configuration updates
- **Config Builder:** Generates Caddy JSON from database models
- **Reload Logic:** Atomic config application with rollback on failure
- **Security Integration:** Injects Cerberus middleware into Caddy pipelines
**Responsibilities:**
+
1. Generate Caddy JSON configuration from database state
2. Validate configuration before applying
3. Trigger Caddy reload via JSON API
@@ -340,22 +345,26 @@ graph TB
5. Integrate security layers (WAF, ACL, Rate Limiting)
#### Security Suite (`internal/cerberus/`)
+
- **ACL (Access Control Lists):** IP-based allow/deny rules, GeoIP blocking
- **WAF (Web Application Firewall):** Coraza engine with OWASP CRS
- **CrowdSec:** Behavior-based threat detection with global intelligence
- **Rate Limiter:** Per-IP request throttling
**Integration Points:**
+
- Middleware injection into Caddy request pipeline
- Database-driven rule configuration
- Metrics collection for security events
#### Database Layer (`internal/database/`)
+
- **Migrations:** Automatic schema versioning with GORM AutoMigrate
- **Seeding:** Default settings and admin user creation
- **Connection Management:** SQLite with WAL mode and connection pooling
**Schema Overview:**
+
- **ProxyHost:** Domain, upstream target, SSL config
- **RemoteServer:** Upstream server definitions
- **CaddyConfig:** Generated Caddy configuration (audit trail)
@@ -372,6 +381,7 @@ graph TB
**Component Architecture:**
#### Pages (`src/pages/`)
+
- **Dashboard:** System overview, recent activity, quick actions
- **ProxyHosts:** List, create, edit, delete proxy configurations
- **Certificates:** Manage SSL/TLS certificates, view expiry
@@ -380,17 +390,20 @@ graph TB
- **Users:** User management (admin only)
#### Components (`src/components/`)
+
- **Forms:** Reusable form inputs with validation
- **Modals:** Dialog components for CRUD operations
- **Tables:** Data tables with sorting, filtering, pagination
- **Layout:** Header, sidebar, navigation
#### API Client (`src/api/`)
+
- Centralized API calls with error handling
- Request/response type definitions
- Authentication token management
**Example:**
+
```typescript
export const getProxyHosts = async (): Promise => {
const response = await fetch('/api/v1/proxy-hosts', {
@@ -402,11 +415,13 @@ export const getProxyHosts = async (): Promise => {
```
#### State Management
+
- **React Context:** Global state for auth, theme, language
- **Local State:** Component-specific state with `useState`
- **Custom Hooks:** Encapsulate API calls and side effects
**Example Hook:**
+
```typescript
export const useProxyHosts = () => {
const [hosts, setHosts] = useState([]);
@@ -425,11 +440,13 @@ export const useProxyHosts = () => {
**Purpose:** High-performance reverse proxy with automatic HTTPS
**Integration:**
+
- Embedded as a library in the Go backend
- Configured via JSON API (not Caddyfile)
- Listens on ports 80 (HTTP) and 443 (HTTPS)
**Features Used:**
+
- Dynamic configuration updates without restarts
- Automatic HTTPS with Let's Encrypt and ZeroSSL
- DNS challenge support for wildcard certificates
@@ -437,6 +454,7 @@ export const useProxyHosts = () => {
- Request logging and metrics
**Configuration Flow:**
+
1. User creates proxy host via frontend
2. Backend validates and saves to database
3. Caddy Manager generates JSON configuration
@@ -461,12 +479,14 @@ For each proxy host, Charon generates **two routes** with the same domain:
- Handlers: Full Cerberus security suite
This pattern is **intentional and valid**:
+
- Emergency route provides break-glass access to security controls
- Main route protects application with enterprise security features
- Caddy processes routes in order (emergency matches first)
- Validator allows duplicate hosts when one has paths and one doesn't
**Example:**
+
```json
// Emergency Route (evaluated first)
{
@@ -488,6 +508,7 @@ This pattern is **intentional and valid**:
**Purpose:** Persistent data storage
**Why SQLite:**
+
- Embedded (no external database server)
- Serverless (perfect for single-user/small team)
- ACID compliant with WAL mode
@@ -495,16 +516,19 @@ This pattern is **intentional and valid**:
- Backup-friendly (single file)
**Configuration:**
+
- **WAL Mode:** Allows concurrent reads during writes
- **Foreign Keys:** Enforced referential integrity
- **Pragma Settings:** Performance optimizations
**Backup Strategy:**
+
- Automated daily backups to `data/backups/`
- Retention: 7 daily, 4 weekly, 12 monthly backups
- Backup during low-traffic periods
**Migrations:**
+
- GORM AutoMigrate for schema changes
- Manual migrations for complex data transformations
- Rollback support via backup restoration
@@ -537,6 +561,7 @@ graph LR
**Purpose:** Prevent brute-force attacks and API abuse
**Implementation:**
+
- Per-IP request counters with sliding window
- Configurable thresholds (e.g., 100 req/min, 1000 req/hour)
- HTTP 429 response when limit exceeded
@@ -547,12 +572,14 @@ graph LR
**Purpose:** Behavior-based threat detection
**Features:**
+
- Local log analysis (brute-force, port scans, exploits)
- Global threat intelligence (crowd-sourced IP reputation)
- Automatic IP banning with configurable duration
- Decision management API (view, create, delete bans)
**Modes:**
+
- **Local Only:** No external API calls
- **API Mode:** Sync with CrowdSec cloud for global intelligence
@@ -561,12 +588,14 @@ graph LR
**Purpose:** IP-based access control
**Features:**
+
- Per-proxy-host allow/deny rules
- CIDR range support (e.g., `192.168.1.0/24`)
- Geographic blocking via GeoIP2 (MaxMind)
- Admin whitelist (emergency access)
**Evaluation Order:**
+
1. Check admin whitelist (always allow)
2. Check deny list (explicit block)
3. Check allow list (explicit allow)
@@ -579,6 +608,7 @@ graph LR
**Engine:** Coraza with OWASP Core Rule Set (CRS)
**Detection Categories:**
+
- SQL Injection (SQLi)
- Cross-Site Scripting (XSS)
- Remote Code Execution (RCE)
@@ -587,12 +617,14 @@ graph LR
- Command Injection
**Modes:**
+
- **Monitor:** Log but don't block (testing)
- **Block:** Return HTTP 403 for violations
### Layer 5: Application Security
**Additional Protections:**
+
- **SSRF Prevention:** Block requests to private IP ranges in webhooks/URL validation
- **HTTP Security Headers:** CSP, HSTS, X-Frame-Options, X-Content-Type-Options
- **Input Validation:** Server-side validation for all user inputs
@@ -610,6 +642,7 @@ graph LR
3. **Direct Database Access:** Manual SQLite update as last resort
**Emergency Token:**
+
- 64-character hex token set via `CHARON_EMERGENCY_TOKEN`
- Grants temporary admin access
- Rotated after each use
@@ -732,6 +765,7 @@ sequenceDiagram
**Rationale:** Simplicity over scalability - target audience is home users and small teams
**Container Contents:**
+
- Frontend static files (Vite build output)
- Go backend binary
- Embedded Caddy server
@@ -784,7 +818,7 @@ CMD ["/app/charon"]
### Port Mapping
| Port | Protocol | Purpose | Bind |
-|------|----------|---------|------|
+| ------ | ---------- | --------- | ------ |
| 8080 | HTTP | Web UI + REST API | 0.0.0.0 |
| 80 | HTTP | Caddy reverse proxy | 0.0.0.0 |
| 443 | HTTPS | Caddy reverse proxy (TLS) | 0.0.0.0 |
@@ -794,14 +828,14 @@ CMD ["/app/charon"]
### Volume Mounts
| Container Path | Purpose | Required |
-|----------------|---------|----------|
+| --- | --- | --- |
| `/app/data` | Database, certificates, backups | **Yes** |
| `/var/run/docker.sock` | Docker container discovery | Optional |
### Environment Variables
| Variable | Purpose | Default | Required |
-|----------|---------|---------|----------|
+| ---------- | --------- | --------- | ---------- |
| `CHARON_ENV` | Environment (production/development) | `production` | No |
| `CHARON_ENCRYPTION_KEY` | 32-byte base64 key for credential encryption | Auto-generated | No |
| `CHARON_EMERGENCY_TOKEN` | 64-char hex for break-glass access | None | Optional |
@@ -841,11 +875,13 @@ services:
### High Availability Considerations
**Current Limitations:**
+
- SQLite does not support clustering
- Single point of failure (one container)
- Not designed for horizontal scaling
**Future Options:**
+
- PostgreSQL backend for HA deployments
- Read replicas for load balancing
- Container orchestration (Kubernetes, Docker Swarm)
@@ -857,6 +893,7 @@ services:
### Local Development Setup
1. **Prerequisites:**
+
```bash
- Go 1.26+ (backend development)
- Node.js 23+ and npm (frontend development)
@@ -865,12 +902,14 @@ services:
```
2. **Clone Repository:**
+
```bash
git clone https://github.com/Wikid82/Charon.git
cd Charon
```
3. **Backend Development:**
+
```bash
cd backend
go mod download
@@ -879,6 +918,7 @@ services:
```
4. **Frontend Development:**
+
```bash
cd frontend
npm install
@@ -887,6 +927,7 @@ services:
```
5. **Full-Stack Development (Docker):**
+
```bash
docker-compose -f .docker/compose/docker-compose.dev.yml up
# Frontend + Backend + Caddy in one container
@@ -895,12 +936,14 @@ services:
### Git Workflow
**Branch Strategy:**
+
- `main`: Stable production branch
- `feature/*`: New feature development
- `fix/*`: Bug fixes
- `chore/*`: Maintenance tasks
**Commit Convention:**
+
- `feat:` New user-facing feature
- `fix:` Bug fix in application code
- `chore:` Infrastructure, CI/CD, dependencies
@@ -909,7 +952,8 @@ services:
- `test:` Adding or updating tests
**Example:**
-```
+
+```text
feat: add DNS-01 challenge support for Cloudflare
Implement Cloudflare DNS provider for automatic wildcard certificate
@@ -945,7 +989,7 @@ Closes #123
### Test Pyramid
-```
+```text
/\ E2E (Playwright) - 10%
/ \ Critical user flows
/____\
@@ -961,6 +1005,7 @@ Closes #123
**Purpose:** Validate critical user flows in a real browser
**Scope:**
+
- User authentication
- Proxy host CRUD operations
- Certificate provisioning
@@ -968,6 +1013,7 @@ Closes #123
- Real-time log streaming
**Execution:**
+
```bash
# Run against Docker container
cd /projects/Charon npx playwright test --project=firefox
@@ -980,10 +1026,12 @@ npx playwright test --debug
```
**Coverage Modes:**
+
- **Docker Mode:** Integration testing, no coverage (0% reported)
- **Vite Dev Mode:** Coverage collection with V8 inspector
**Why Two Modes?**
+
- Playwright coverage requires source maps and raw source files
- Docker serves pre-built production files (no source maps)
- Vite dev server exposes source files for coverage instrumentation
@@ -997,6 +1045,7 @@ npx playwright test --debug
**Coverage Target:** 85% minimum
**Execution:**
+
```bash
# Run all tests
go test ./...
@@ -1009,11 +1058,13 @@ go test -cover ./...
```
**Test Organization:**
+
- `*_test.go` files alongside source code
- Table-driven tests for comprehensive coverage
- Mocks for external dependencies (database, HTTP clients)
**Example:**
+
```go
func TestCreateProxyHost(t *testing.T) {
tests := []struct {
@@ -1053,6 +1104,7 @@ func TestCreateProxyHost(t *testing.T) {
**Coverage Target:** 85% minimum
**Execution:**
+
```bash
# Run all tests
npm test
@@ -1065,6 +1117,7 @@ npm run test:coverage
```
**Test Organization:**
+
- `*.test.tsx` files alongside components
- Mock API calls with MSW (Mock Service Worker)
- Snapshot tests for UI consistency
@@ -1076,12 +1129,14 @@ npm run test:coverage
**Location:** `backend/integration/`
**Scope:**
+
- API endpoint end-to-end flows
- Database migrations
- Caddy manager integration
- CrowdSec API calls
**Execution:**
+
```bash
go test ./integration/...
```
@@ -1091,6 +1146,7 @@ go test ./integration/...
**Automated Hooks (via `.pre-commit-config.yaml`):**
**Fast Stage (< 5 seconds):**
+
- Trailing whitespace removal
- EOF fixer
- YAML syntax check
@@ -1098,11 +1154,13 @@ go test ./integration/...
- Markdown link validation
**Manual Stage (run explicitly):**
+
- Backend coverage tests (60-90s)
- Frontend coverage tests (30-60s)
- TypeScript type checking (10-20s)
**Why Manual?**
+
- Coverage tests are slow and would block commits
- Developers run them on-demand before pushing
- CI enforces coverage on pull requests
@@ -1110,10 +1168,12 @@ go test ./integration/...
### Continuous Integration (GitHub Actions)
**Workflow Triggers:**
+
- `push` to `main`, `feature/*`, `fix/*`
- `pull_request` to `main`
**CI Jobs:**
+
1. **Lint:** golangci-lint, ESLint, markdownlint, hadolint
2. **Test:** Go tests, Vitest, Playwright
3. **Security:** Trivy, CodeQL, Grype, Govulncheck
@@ -1135,6 +1195,7 @@ go test ./integration/...
- **PRERELEASE:** `-beta.1`, `-rc.1`, etc.
**Examples:**
+
- `1.0.0` - Stable release
- `1.1.0` - New feature (DNS provider support)
- `1.1.1` - Bug fix (GORM query fix)
@@ -1145,12 +1206,14 @@ go test ./integration/...
### Build Pipeline (Multi-Platform)
**Platforms Supported:**
+
- `linux/amd64`
- `linux/arm64`
**Build Process:**
1. **Frontend Build:**
+
```bash
cd frontend
npm ci --only=production
@@ -1159,6 +1222,7 @@ go test ./integration/...
```
2. **Backend Build:**
+
```bash
cd backend
go build -o charon cmd/api/main.go
@@ -1166,6 +1230,7 @@ go test ./integration/...
```
3. **Docker Image Build:**
+
```bash
docker buildx build \
--platform linux/amd64,linux/arm64 \
@@ -1214,6 +1279,7 @@ go test ./integration/...
- Level: SLSA Build L3 (hermetic builds)
**Verification Example:**
+
```bash
# Verify image signature
cosign verify \
@@ -1231,6 +1297,7 @@ grype wikid82/charon:latest
### Rollback Strategy
**Container Rollback:**
+
```bash
# List available versions
docker images wikid82/charon
@@ -1241,6 +1308,7 @@ docker-compose up -d --pull always wikid82/charon:1.1.1
```
**Database Rollback:**
+
```bash
# Restore from backup
docker exec charon /app/scripts/restore-backup.sh \
@@ -1277,11 +1345,13 @@ docker exec charon /app/scripts/restore-backup.sh \
### API Extensibility
**REST API Design:**
+
- Version prefix: `/api/v1/`
- Future versions: `/api/v2/` (backward-compatible)
- Deprecation policy: 2 major versions supported
**WebHooks (Future):**
+
- Event notifications for external systems
- Triggers: Proxy host created, certificate renewed, security event
- Payload: JSON with event type and data
@@ -1291,6 +1361,7 @@ docker exec charon /app/scripts/restore-backup.sh \
**Current:** Cerberus security middleware injected into Caddy pipeline
**Future:**
+
- User-defined middleware (rate limiting rules, custom headers)
- JavaScript/Lua scripting for request transformation
- Plugin marketplace for community contributions
@@ -1374,6 +1445,7 @@ docker exec charon /app/scripts/restore-backup.sh \
**GitHub Copilot Instructions:**
All agents (`Planning`, `Backend_Dev`, `Frontend_Dev`, `DevOps`) must reference `ARCHITECTURE.md` when:
+
- Creating new components
- Modifying core systems
- Changing integration points
diff --git a/.github/instructions/a11y.instructions.md b/.github/instructions/a11y.instructions.md
index f6a317500..1882a91ab 100644
--- a/.github/instructions/a11y.instructions.md
+++ b/.github/instructions/a11y.instructions.md
@@ -86,7 +86,7 @@ A skip link MUST be provided to skip blocks of content that appear across severa
}
```
-#### Common keyboard commands:
+#### Common keyboard commands
- `Tab` = Move to the next interactive element.
- `Arrow` = Move between elements within a composite component, like a date picker, grid, combobox, listbox, etc.
@@ -249,7 +249,7 @@ The page title:
Column and row headers MUST be programmatically associated for each cell. In HTML, this is done by using `
` elements. Column headers MUST be defined in the first table row `
`. Row headers must defined in the row they are for. Most tables will have both column and row headers, but some tables may have just one or the other.
-#### Good example - table with both column and row headers:
+#### Good example - table with both column and row headers
```html
@@ -271,7 +271,7 @@ Column and row headers MUST be programmatically associated for each cell. In HTM
```
-#### Good example - table with just column headers:
+#### Good example - table with just column headers
```html
@@ -293,7 +293,7 @@ Column and row headers MUST be programmatically associated for each cell. In HTM
```
-#### Bad example - calendar grid with partial semantics:
+#### Bad example - calendar grid with partial semantics
The following example is a date picker or calendar grid.
@@ -344,13 +344,13 @@ The following example is a date picker or calendar grid.
```
-##### The good:
+##### The good
- It uses `role="grid"` to indicate that it is a grid.
- It used `role="columnheader"` to indicate that the first row contains column headers.
- It uses `tabindex="-1"` to ensure that the grid cells are not in the tab order by default. Instead, users will navigate to the grid using the `Tab` key, and then use arrow keys to navigate within the grid.
-##### The bad:
+##### The bad
- `role=gridcell` elements are not nested within `role=row` elements. Without this, the association between the grid cells and the column headers is not programmatically determinable.
diff --git a/.github/instructions/agent-skills.instructions.md b/.github/instructions/agent-skills.instructions.md
index d0de7307c..918517365 100644
--- a/.github/instructions/agent-skills.instructions.md
+++ b/.github/instructions/agent-skills.instructions.md
@@ -12,6 +12,7 @@ Instructions for creating effective and portable Agent Skills that enhance GitHu
Agent Skills are self-contained folders with instructions and bundled resources that teach AI agents specialized capabilities. Unlike custom instructions (which define coding standards), skills enable task-specific workflows that can include scripts, examples, templates, and reference data.
Key characteristics:
+
- **Portable**: Works across VS Code, Copilot CLI, and Copilot coding agent
- **Progressive loading**: Only loaded when relevant to the user's request
- **Resource-bundled**: Can include scripts, templates, examples alongside instructions
@@ -22,7 +23,7 @@ Key characteristics:
Skills are stored in specific locations:
| Location | Scope | Recommendation |
-|----------|-------|----------------|
+| ---------- | ------- | ---------------- |
| `.github/skills//` | Project/repository | Recommended for project skills |
| `.claude/skills//` | Project/repository | Legacy, for backward compatibility |
| `~/.github/skills//` | Personal (user-wide) | Recommended for personal skills |
@@ -43,7 +44,7 @@ license: Complete terms in LICENSE.txt
```
| Field | Required | Constraints |
-|-------|----------|-------------|
+| ------- | ---------- | ------------- |
| `name` | Yes | Lowercase, hyphens for spaces, max 64 characters (e.g., `webapp-testing`) |
| `description` | Yes | Clear description of capabilities AND use cases, max 1024 characters |
| `license` | No | Reference to LICENSE.txt (e.g., `Complete terms in LICENSE.txt`) or SPDX identifier |
@@ -53,21 +54,25 @@ license: Complete terms in LICENSE.txt
**CRITICAL**: The `description` field is the PRIMARY mechanism for automatic skill discovery. Copilot reads ONLY the `name` and `description` to decide whether to load a skill. If your description is vague, the skill will never be activated.
**What to include in description:**
+
1. **WHAT** the skill does (capabilities)
2. **WHEN** to use it (specific triggers, scenarios, file types, or user requests)
3. **Keywords** that users might mention in their prompts
**Good description:**
+
```yaml
description: Toolkit for testing local web applications using Playwright. Use when asked to verify frontend functionality, debug UI behavior, capture browser screenshots, check for visual regressions, or view browser console logs. Supports Chrome, Firefox, and WebKit browsers.
```
**Poor description:**
+
```yaml
description: Web testing helpers
```
The poor description fails because:
+
- No specific triggers (when should Copilot load this?)
- No keywords (what user prompts would match?)
- No capabilities (what can it actually do?)
@@ -77,7 +82,7 @@ The poor description fails because:
The body contains detailed instructions that Copilot loads AFTER the skill is activated. Recommended sections:
| Section | Purpose |
-|---------|---------|
+| --------- | --------- |
| `# Title` | Brief overview of what this skill enables |
| `## When to Use This Skill` | List of scenarios (reinforces description triggers) |
| `## Prerequisites` | Required tools, dependencies, environment setup |
@@ -92,7 +97,7 @@ Skills can include additional files that Copilot accesses on-demand:
### Supported Resource Types
| Folder | Purpose | Loaded into Context? | Example Files |
-|--------|---------|---------------------|---------------|
+| -------- | --------- | --------------------- | --------------- |
| `scripts/` | Executable automation that performs specific operations | When executed | `helper.py`, `validate.sh`, `build.ts` |
| `references/` | Documentation the AI agent reads to inform decisions | Yes, when referenced | `api_reference.md`, `schema.md`, `workflow_guide.md` |
| `assets/` | **Static files used AS-IS** in output (not modified by the AI agent) | No | `logo.png`, `brand-template.pptx`, `custom-font.ttf` |
@@ -100,7 +105,7 @@ Skills can include additional files that Copilot accesses on-demand:
### Directory Structure Example
-```
+```text
.github/skills/my-skill/
βββ SKILL.md # Required: Main instructions
βββ LICENSE.txt # Recommended: License terms (Apache 2.0 typical)
@@ -119,16 +124,18 @@ Skills can include additional files that Copilot accesses on-demand:
βββ config.template # Config template the AI agent fills in
```
-> **LICENSE.txt**: When creating a skill, download the Apache 2.0 license text from https://www.apache.org/licenses/LICENSE-2.0.txt and save as `LICENSE.txt`. Update the copyright year and owner in the appendix section.
+> **LICENSE.txt**: When creating a skill, download the Apache 2.0 license text from and save as `LICENSE.txt`. Update the copyright year and owner in the appendix section.
### Assets vs Templates: Key Distinction
**Assets** are static resources **consumed unchanged** in the output:
+
- A `logo.png` that gets embedded into a generated document
- A `report-template.html` copied as output format
- A `custom-font.ttf` applied to text rendering
**Templates** are starter code/scaffolds that **the AI agent actively modifies**:
+
- A `scaffold.py` where the AI agent inserts logic
- A `config.template` where the AI agent fills in values based on user requirements
- A `hello-world/` project directory that the AI agent extends with new features
@@ -154,12 +161,13 @@ Use the [scaffold](./templates/scaffold.py) as a starting point.
Skills use three-level loading for efficiency:
| Level | What Loads | When |
-|-------|------------|------|
+| ------- | ------------ | ------ |
| 1. Discovery | `name` and `description` only | Always (lightweight metadata) |
| 2. Instructions | Full `SKILL.md` body | When request matches description |
| 3. Resources | Scripts, examples, docs | Only when Copilot references them |
This means:
+
- Install many skills without consuming context
- Only relevant content loads per task
- Resources don't load until explicitly needed
@@ -179,13 +187,14 @@ This means:
When including scripts, prefer cross-platform languages:
| Language | Use Case |
-|----------|----------|
+| ---------- | ---------- |
| Python | Complex automation, data processing |
| pwsh | PowerShell Core scripting |
| Node.js | JavaScript-based tooling |
| Bash/Shell | Simple automation tasks |
Best practices:
+
- Include help/usage documentation (`--help` flag)
- Handle errors gracefully with clear messages
- Avoid storing credentials or secrets
@@ -194,6 +203,7 @@ Best practices:
### When to Bundle Scripts
Include scripts in your skill when:
+
- The same code would be rewritten repeatedly by the agent
- Deterministic reliability is critical (e.g., file manipulation, API calls)
- Complex logic benefits from being pre-tested rather than generated each time
diff --git a/.github/instructions/agents.instructions.md b/.github/instructions/agents.instructions.md
index 049356188..deebd6255 100644
--- a/.github/instructions/agents.instructions.md
+++ b/.github/instructions/agents.instructions.md
@@ -14,7 +14,7 @@ Instructions for creating effective and maintainable custom agent files that pro
- File naming convention: lowercase with hyphens (e.g., `test-specialist.agent.md`)
- Location: `.github/agents/` directory (repository-level) or `agents/` directory (organization/enterprise-level)
- Purpose: Define specialized agents with tailored expertise, tools, and instructions for specific tasks
-- Official documentation: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/create-custom-agents
+- Official documentation:
## Required Frontmatter
@@ -34,44 +34,52 @@ infer: true
### Core Frontmatter Properties
#### **description** (REQUIRED)
+
- Single-quoted string, clearly stating the agent's purpose and domain expertise
- Should be concise (50-150 characters) and actionable
- Example: `'Focuses on test coverage, quality, and testing best practices'`
#### **name** (OPTIONAL)
+
- Display name for the agent in the UI
- If omitted, defaults to filename (without `.md` or `.agent.md`)
- Use title case and be descriptive
- Example: `'Testing Specialist'`
#### **tools** (OPTIONAL)
+
- List of tool names or aliases the agent can use
- Supports comma-separated string or YAML array format
- If omitted, agent has access to all available tools
- See "Tool Configuration" section below for details
#### **model** (STRONGLY RECOMMENDED)
+
- Specifies which AI model the agent should use
- Supported in VS Code, JetBrains IDEs, Eclipse, and Xcode
- Example: `'Claude Sonnet 4.5'`, `'gpt-4'`, `'gpt-4o'`
- Choose based on agent complexity and required capabilities
#### **target** (OPTIONAL)
+
- Specifies target environment: `'vscode'` or `'github-copilot'`
- If omitted, agent is available in both environments
- Use when agent has environment-specific features
#### **infer** (OPTIONAL)
+
- Boolean controlling whether Copilot can automatically use this agent based on context
- Default: `true` if omitted
- Set to `false` to require manual agent selection
#### **metadata** (OPTIONAL, GitHub.com only)
+
- Object with name-value pairs for agent annotation
- Example: `metadata: { category: 'testing', version: '1.0' }`
- Not supported in VS Code
#### **mcp-servers** (OPTIONAL, Organization/Enterprise only)
+
- Configure MCP servers available only to this agent
- Only supported for organization/enterprise level agents
- See "MCP Server Configuration" section below
@@ -81,22 +89,26 @@ infer: true
### Tool Specification Strategies
**Enable all tools** (default):
+
```yaml
# Omit tools property entirely, or use:
tools: ['*']
```
**Enable specific tools**:
+
```yaml
tools: ['read', 'edit', 'search', 'execute']
```
**Enable MCP server tools**:
+
```yaml
tools: ['read', 'edit', 'github/*', 'playwright/navigate']
```
**Disable all tools**:
+
```yaml
tools: []
```
@@ -106,7 +118,7 @@ tools: []
All aliases are case-insensitive:
| Alias | Alternative Names | Category | Description |
-|-------|------------------|----------|-------------|
+| ------- | ------------------ | ---------- | ------------- |
| `execute` | shell, Bash, powershell | Shell execution | Execute commands in appropriate shell |
| `read` | Read, NotebookRead, view | File reading | Read file contents |
| `edit` | Edit, MultiEdit, Write, NotebookEdit | File editing | Edit and modify files |
@@ -118,18 +130,22 @@ All aliases are case-insensitive:
### Built-in MCP Server Tools
**GitHub MCP Server**:
+
```yaml
tools: ['github/*'] # All GitHub tools
tools: ['github/get_file_contents', 'github/search_repositories'] # Specific tools
```
+
- All read-only tools available by default
- Token scoped to source repository
**Playwright MCP Server**:
+
```yaml
tools: ['playwright/*'] # All Playwright tools
tools: ['playwright/navigate', 'playwright/screenshot'] # Specific tools
```
+
- Configured to access localhost only
- Useful for browser automation and testing
@@ -179,6 +195,7 @@ Structure each sub-agent call with:
2. **prompt**: Detailed instructions with substituted variables
The prompt should include:
+
- Who the sub-agent is (specialist role)
- What context it needs (parameters, paths)
- What to do (concrete tasks)
@@ -232,8 +249,6 @@ Return: Key findings and identified patterns`
- **Sequential execution**: Use `await` to maintain order when steps depend on each other
- **Error handling**: Check results before proceeding to dependent steps
-
-
## Agent Prompt Structure
The markdown content below the frontmatter defines the agent's behavior, expertise, and instructions. Well-structured prompts typically include:
@@ -259,6 +274,7 @@ Agents can define dynamic parameters to extract values from user input and use t
### When to Use Variables
**Use variables when**:
+
- Agent behavior depends on user input
- Need to pass dynamic values to sub-agents
- Want to make agents reusable across different contexts
@@ -266,6 +282,7 @@ Agents can define dynamic parameters to extract values from user input and use t
- Need to track or reference user-provided context
**Examples**:
+
- Extract project name from user prompt
- Capture certification name for pipeline processing
- Identify file paths or directories
@@ -292,6 +309,7 @@ Process [PARAMETER_NAME] to accomplish [task].
### Variable Extraction Methods
#### 1. **Explicit User Input**
+
Ask the user to provide the variable if not detected in the prompt:
```markdown
@@ -309,6 +327,7 @@ Use this information to contextualize all subsequent tasks.
```
#### 2. **Implicit Extraction from Prompt**
+
Automatically extract variables from the user's natural language input:
```javascript
@@ -324,6 +343,7 @@ const basePath = `certifications/${certificationName}`;
```
#### 3. **Contextual Variable Resolution**
+
Use file context or workspace information to derive variables:
```markdown
@@ -463,10 +483,10 @@ Return: Final verdict and executive summary`
This pattern applies to any orchestration scenario: extract variables, call sub-agents with clear context, await results.
-
### Variable Best Practices
#### 1. **Clear Documentation**
+
Always document what variables are expected:
```markdown
@@ -484,6 +504,7 @@ Always document what variables are expected:
```
#### 2. **Consistent Naming**
+
Use consistent variable naming conventions:
```javascript
@@ -506,6 +527,7 @@ const bad_variables = {
```
#### 3. **Validation and Constraints**
+
Document valid values and constraints:
```markdown
@@ -559,6 +581,7 @@ mcp-servers:
Secrets must be configured in repository settings under "copilot" environment.
**Supported syntax**:
+
```yaml
env:
# Environment variable only
@@ -576,16 +599,19 @@ env:
## File Organization and Naming
### Repository-Level Agents
+
- Location: `.github/agents/`
- Scope: Available only in the specific repository
- Access: Uses repository-configured MCP servers
### Organization/Enterprise-Level Agents
+
- Location: `.github-private/agents/` (then move to `agents/` root)
- Scope: Available across all repositories in org/enterprise
- Access: Can configure dedicated MCP servers
### Naming Conventions
+
- Use lowercase with hyphens: `test-specialist.agent.md`
- Name should reflect agent purpose
- Filename becomes default agent name (if `name` not specified)
@@ -594,13 +620,16 @@ env:
## Agent Processing and Behavior
### Versioning
+
- Based on Git commit SHAs for the agent file
- Create branches/tags for different agent versions
- Instantiated using latest version for repository/branch
- PR interactions use same agent version for consistency
### Name Conflicts
+
Priority (highest to lowest):
+
1. Repository-level agent
2. Organization-level agent
3. Enterprise-level agent
@@ -608,6 +637,7 @@ Priority (highest to lowest):
Lower-level configurations override higher-level ones with the same name.
### Tool Processing
+
- `tools` list filters available tools (built-in and MCP)
- No tools specified = all tools enabled
- Empty list (`[]`) = all tools disabled
@@ -615,6 +645,7 @@ Lower-level configurations override higher-level ones with the same name.
- Unrecognized tool names are ignored (allows environment-specific tools)
### MCP Server Processing Order
+
1. Out-of-the-box MCP servers (e.g., GitHub MCP)
2. Custom agent MCP configuration (org/enterprise only)
3. Repository-level MCP configurations
@@ -624,6 +655,7 @@ Each level can override settings from previous levels.
## Agent Creation Checklist
### Frontmatter
+
- [ ] `description` field present and descriptive (50-150 chars)
- [ ] `description` wrapped in single quotes
- [ ] `name` specified (optional but recommended)
@@ -633,6 +665,7 @@ Each level can override settings from previous levels.
- [ ] `infer` set to `false` if manual selection required
### Prompt Content
+
- [ ] Clear agent identity and role defined
- [ ] Core responsibilities listed explicitly
- [ ] Approach and methodology explained
@@ -644,12 +677,14 @@ Each level can override settings from previous levels.
- [ ] Total content under 30,000 characters
### File Structure
+
- [ ] Filename follows lowercase-with-hyphens convention
- [ ] File placed in correct directory (`.github/agents/` or `agents/`)
- [ ] Filename uses only allowed characters
- [ ] File extension is `.agent.md`
### Quality Assurance
+
- [ ] Agent purpose is unique and not duplicative
- [ ] Tools are minimal and necessary
- [ ] Instructions are clear and unambiguous
@@ -660,26 +695,31 @@ Each level can override settings from previous levels.
## Common Agent Patterns
### Testing Specialist
+
**Purpose**: Focus on test coverage and quality
**Tools**: All tools (for comprehensive test creation)
**Approach**: Analyze, identify gaps, write tests, avoid production code changes
### Implementation Planner
+
**Purpose**: Create detailed technical plans and specifications
**Tools**: Limited to `['read', 'search', 'edit']`
**Approach**: Analyze requirements, create documentation, avoid implementation
### Code Reviewer
+
**Purpose**: Review code quality and provide feedback
**Tools**: `['read', 'search']` only
**Approach**: Analyze, suggest improvements, no direct modifications
### Refactoring Specialist
+
**Purpose**: Improve code structure and maintainability
**Tools**: `['read', 'search', 'edit']`
**Approach**: Analyze patterns, propose refactorings, implement safely
### Security Auditor
+
**Purpose**: Identify security issues and vulnerabilities
**Tools**: `['read', 'search', 'web']`
**Approach**: Scan code, check against OWASP, report findings
@@ -687,18 +727,21 @@ Each level can override settings from previous levels.
## Common Mistakes to Avoid
### Frontmatter Errors
+
- β Missing `description` field
- β Description not wrapped in quotes
- β Invalid tool names without checking documentation
- β Incorrect YAML syntax (indentation, quotes)
### Tool Configuration Issues
+
- β Granting excessive tool access unnecessarily
- β Missing required tools for agent's purpose
- β Not using tool aliases consistently
- β Forgetting MCP server namespace (`server-name/tool`)
### Prompt Content Problems
+
- β Vague, ambiguous instructions
- β Conflicting or contradictory guidelines
- β Lack of clear scope definition
@@ -707,6 +750,7 @@ Each level can override settings from previous levels.
- β No examples or context for complex tasks
### Organizational Issues
+
- β Filename doesn't reflect agent purpose
- β Wrong directory (confusing repo vs org level)
- β Using spaces or special characters in filename
@@ -715,6 +759,7 @@ Each level can override settings from previous levels.
## Testing and Validation
### Manual Testing
+
1. Create the agent file with proper frontmatter
2. Reload VS Code or refresh GitHub.com
3. Select the agent from the dropdown in Copilot Chat
@@ -723,6 +768,7 @@ Each level can override settings from previous levels.
6. Confirm output meets expectations
### Integration Testing
+
- Test agent with different file types in scope
- Verify MCP server connectivity (if configured)
- Check agent behavior with missing context
@@ -730,6 +776,7 @@ Each level can override settings from previous levels.
- Validate agent switching and handoffs
### Quality Checks
+
- Run through agent creation checklist
- Review against common mistakes list
- Compare with example agents in repository
@@ -739,29 +786,34 @@ Each level can override settings from previous levels.
## Additional Resources
### Official Documentation
+
- [Creating Custom Agents](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/create-custom-agents)
- [Custom Agents Configuration](https://docs.github.com/en/copilot/reference/custom-agents-configuration)
- [Custom Agents in VS Code](https://code.visualstudio.com/docs/copilot/customization/custom-agents)
- [MCP Integration](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/extend-coding-agent-with-mcp)
### Community Resources
+
- [Awesome Copilot Agents Collection](https://github.com/github/awesome-copilot/tree/main/agents)
- [Customization Library Examples](https://docs.github.com/en/copilot/tutorials/customization-library/custom-agents)
- [Your First Custom Agent Tutorial](https://docs.github.com/en/copilot/tutorials/customization-library/custom-agents/your-first-custom-agent)
### Related Files
+
- [Prompt Files Guidelines](./prompt.instructions.md) - For creating prompt files
- [Instructions Guidelines](./instructions.instructions.md) - For creating instruction files
## Version Compatibility Notes
### GitHub.com (Coding Agent)
+
- β Fully supports all standard frontmatter properties
- β Repository and org/enterprise level agents
- β MCP server configuration (org/enterprise)
- β Does not support `model`, `argument-hint`, `handoffs` properties
### VS Code / JetBrains / Eclipse / Xcode
+
- β Supports `model` property for AI model selection
- β Supports `argument-hint` and `handoffs` properties
- β User profile and workspace-level agents
diff --git a/.github/instructions/code-review-generic.instructions.md b/.github/instructions/code-review-generic.instructions.md
index c8a1ca296..5c1017211 100644
--- a/.github/instructions/code-review-generic.instructions.md
+++ b/.github/instructions/code-review-generic.instructions.md
@@ -19,18 +19,21 @@ When performing a code review, respond in **English** (or specify your preferred
When performing a code review, prioritize issues in the following order:
### π΄ CRITICAL (Block merge)
+
- **Security**: Vulnerabilities, exposed secrets, authentication/authorization issues
- **Correctness**: Logic errors, data corruption risks, race conditions
- **Breaking Changes**: API contract changes without versioning
- **Data Loss**: Risk of data loss or corruption
### π‘ IMPORTANT (Requires discussion)
+
- **Code Quality**: Severe violations of SOLID principles, excessive duplication
- **Test Coverage**: Missing tests for critical paths or new functionality
- **Performance**: Obvious performance bottlenecks (N+1 queries, memory leaks)
- **Architecture**: Significant deviations from established patterns
### π’ SUGGESTION (Non-blocking improvements)
+
- **Readability**: Poor naming, complex logic that could be simplified
- **Optimization**: Performance improvements without functional impact
- **Best Practices**: Minor deviations from conventions
@@ -53,6 +56,7 @@ When performing a code review, follow these principles:
When performing a code review, check for:
### Clean Code
+
- Descriptive and meaningful names for variables, functions, and classes
- Single Responsibility Principle: each function/class does one thing well
- DRY (Don't Repeat Yourself): no code duplication
@@ -62,6 +66,7 @@ When performing a code review, check for:
- Code should be self-documenting; comments only when necessary
### Examples
+
```javascript
// β BAD: Poor naming and magic numbers
function calc(x, y) {
@@ -82,13 +87,16 @@ function calculateDiscount(orderTotal, itemPrice) {
```
### Error Handling
+
- Proper error handling at appropriate levels
- Meaningful error messages
- No silent failures or ignored exceptions
- Fail fast: validate inputs early
- Use appropriate error types/exceptions
+
### Examples
+
```python
# β BAD: Silent failure and generic error
def process_user(user_id):
@@ -126,6 +134,7 @@ When performing a code review, check for security issues:
- **Dependency Security**: Check for known vulnerabilities in dependencies
### Examples
+
```java
// β BAD: SQL injection vulnerability
String query = "SELECT * FROM users WHERE email = '" + email + "'";
@@ -158,6 +167,7 @@ When performing a code review, verify test quality:
- **Mock Appropriately**: Mock external dependencies, not domain logic
### Examples
+
```typescript
// β BAD: Vague name and assertion
test('test1', () => {
@@ -188,6 +198,7 @@ When performing a code review, check for performance issues:
- **Lazy Loading**: Load data only when needed
### Examples
+
```python
# β BAD: N+1 query problem
users = User.query.all()
@@ -242,6 +253,7 @@ Explanation of the impact or reason for the suggestion.
### Example Comments
#### Critical Issue
+
```markdown
**π΄ CRITICAL - Security: SQL Injection Vulnerability**
@@ -265,7 +277,8 @@ stmt.setString(1, email);
```
**Reference:** OWASP SQL Injection Prevention Cheat Sheet
-```
+
+```text
#### Important Issue
```markdown
@@ -290,7 +303,8 @@ test('should process full refund when order is cancelled', () => {
expect(result.status).toBe('refunded');
});
```
-```
+
+```text
#### Suggestion
```markdown
@@ -318,7 +332,8 @@ if (!user || !user.isActive || !user.hasPermission('write')) {
}
// do something
```
-```
+
+```text
## Review Checklist
diff --git a/.github/instructions/commit-message.instructions.md b/.github/instructions/commit-message.instructions.md
index acd0f39f4..a244d29aa 100644
--- a/.github/instructions/commit-message.instructions.md
+++ b/.github/instructions/commit-message.instructions.md
@@ -18,12 +18,11 @@ When generating commit messages automatically:
**Litmus Test**:
If someone reads only the commit message, they should understand:
+
- What changed
- Why it mattered
- What behavior is different now
-```
-
# Git Commit Message Best Practices
Comprehensive guidelines for crafting high-quality commit messages that improve code review efficiency, project documentation, and team collaboration. Based on industry standards and the conventional commits specification.
@@ -41,7 +40,7 @@ Comprehensive guidelines for crafting high-quality commit messages that improve
A Git commit message consists of two parts:
-```
+```text
():
@@ -83,7 +82,7 @@ A Git commit message consists of two parts:
Use these standardized types for consistency and automated tooling:
| Type | Description | Example | When to Use |
-|------|-------------|---------|-------------|
+| ------ | ------------- | --------- | ------------- |
| `feat` | New user-facing feature | `feat: add password reset email` | New functionality visible to users |
| `fix` | Bug fix in application code | `fix: correct validation logic for email` | Fixing a bug that affects users |
| `chore` | Infrastructure, tooling, dependencies | `chore: upgrade Go to 1.21` | CI/CD, build scripts, dependencies |
@@ -100,7 +99,7 @@ Use these standardized types for consistency and automated tooling:
Add scope in parentheses to specify what part of the codebase changed:
-```
+```text
feat(auth): add OAuth2 provider support
fix(api): handle null response from external service
docs(readme): add Docker installation instructions
@@ -108,6 +107,7 @@ chore(deps): upgrade React to 18.3.0
```
**Common Scopes**:
+
- Component names: `(button)`, `(modal)`, `(navbar)`
- Module names: `(auth)`, `(api)`, `(database)`
- Feature areas: `(settings)`, `(profile)`, `(checkout)`
@@ -116,6 +116,7 @@ chore(deps): upgrade React to 18.3.0
## Quick Guidelines
β **DO**:
+
- Use imperative mood: "Add", "Fix", "Update", "Remove"
- Start with lowercase type: `feat:`, `fix:`, `docs:`
- Be specific: "Fix login redirect" not "Fix bug"
@@ -126,6 +127,7 @@ chore(deps): upgrade React to 18.3.0
- Use conventional commit types
β **DON'T**:
+
- End summary with punctuation (`.`, `!`, `?`)
- Use past tense: "Added", "Fixed", "Updated"
- Use vague messages: "Fix stuff", "Update code", "WIP"
@@ -139,7 +141,8 @@ chore(deps): upgrade React to 18.3.0
### β Excellent Examples
#### Simple Feature
-```
+
+```text
feat(auth): add two-factor authentication
Implement TOTP-based 2FA using the speakeasy library.
@@ -149,7 +152,8 @@ Closes #234
```
#### Bug Fix with Context
-```
+
+```text
fix(api): prevent race condition in user updates
Previously, concurrent updates to user profiles could
@@ -162,7 +166,8 @@ Fixes #567
```
#### Documentation Update
-```
+
+```text
docs: add troubleshooting section to README
Include solutions for common installation issues:
@@ -172,7 +177,8 @@ Include solutions for common installation issues:
```
#### Dependency Update
-```
+
+```text
chore(deps): upgrade express from 4.17 to 4.19
Security patch for CVE-2024-12345. No breaking changes
@@ -180,7 +186,8 @@ or API modifications required.
```
#### Breaking Change
-```
+
+```text
feat(api): redesign user authentication endpoint
BREAKING CHANGE: The /api/login endpoint now returns
@@ -193,7 +200,8 @@ Closes #789
```
#### Refactoring
-```
+
+```text
refactor(services): extract user service interface
Move user-related business logic from handlers to a
@@ -204,7 +212,7 @@ Improves testability and separation of concerns.
### β Bad Examples
-```
+```text
β update files
β Too vague - what was updated and why?
@@ -237,14 +245,16 @@ Improves testability and separation of concerns.
Each commit should represent one logical change:
β **Good**: Three separate commits
-```
+
+```text
feat(auth): add login endpoint
feat(auth): add logout endpoint
test(auth): add integration tests for auth endpoints
```
β **Bad**: One commit with everything
-```
+
+```text
feat: implement authentication system
(Contains login, logout, tests, and unrelated CSS changes)
```
@@ -252,12 +262,14 @@ feat: implement authentication system
### Commit Frequency
**Commit often to**:
+
- Keep messages focused and simple
- Make code review easier
- Simplify debugging with `git bisect`
- Reduce risk of lost work
**Good rhythm**:
+
- After completing a logical unit of work
- Before switching tasks or taking a break
- When tests pass for a feature component
@@ -266,7 +278,7 @@ feat: implement authentication system
Include issue references in the footer:
-```
+```text
feat(api): add rate limiting middleware
Implement rate limiting using express-rate-limit to
@@ -277,6 +289,7 @@ Refs #346, #347
```
**Keywords for automatic closing**:
+
- `Closes #123`, `Fixes #123`, `Resolves #123`
- `Closes: #123` (with colon)
- Multiple: `Fixes #123, #124, #125`
@@ -285,7 +298,7 @@ Refs #346, #347
For pair programming or collaborative work:
-```
+```text
feat(ui): redesign dashboard layout
Co-authored-by: Jane Doe
@@ -294,7 +307,7 @@ Co-authored-by: John Smith
### Reverting Commits
-```
+```text
revert: revert "feat(api): add rate limiting"
This reverts commit abc123def456.
@@ -336,11 +349,13 @@ Document your team's commit message conventions:
Use tools to enforce commit message standards:
**commitlint** (Recommended)
+
```bash
npm install --save-dev @commitlint/{cli,config-conventional}
```
**.commitlintrc.json**
+
```json
{
"extends": ["@commitlint/config-conventional"],
@@ -378,7 +393,8 @@ Before committing, verify:
Create a commit template to remind you of the format:
**~/.gitmessage**
-```
+
+```text
# (): (max 50 chars)
# |<---- Using a Maximum Of 50 Characters ---->|
@@ -408,6 +424,7 @@ Create a commit template to remind you of the format:
```
**Enable it**:
+
```bash
git config --global commit.template ~/.gitmessage
```
@@ -430,6 +447,7 @@ git config --global commit.template ~/.gitmessage
```
**Usage**:
+
```bash
git cf "add user authentication" # Creates: feat: add user authentication
git cx "resolve null pointer in handler" # Creates: fix: resolve null pointer in handler
@@ -461,21 +479,24 @@ git rebase -i HEAD~3 # Edit last 3 commits
## Language-Specific Considerations
### Go Projects
-```
+
+```text
feat(http): add middleware for request logging
refactor(db): migrate from database/sql to sqlx
fix(parser): handle edge case in JSON unmarshaling
```
### JavaScript/TypeScript Projects
-```
+
+```text
feat(components): add error boundary component
fix(hooks): prevent infinite loop in useEffect
chore(deps): upgrade React to 18.3.0
```
### Python Projects
-```
+
+```text
feat(api): add FastAPI endpoint for user registration
fix(models): correct SQLAlchemy relationship mapping
test(utils): add unit tests for date parsing
@@ -484,7 +505,7 @@ test(utils): add unit tests for date parsing
## Common Pitfalls and Solutions
| Pitfall | Solution |
-|---------|----------|
+| --------- | ---------- |
| Forgetting to commit | Set reminders, commit frequently |
| Vague messages | Include specific details about what changed |
| Too many changes in one commit | Break into atomic commits |
@@ -498,11 +519,13 @@ test(utils): add unit tests for date parsing
Well-formatted commits enable automatic changelog generation:
**Example Tools**:
+
- `conventional-changelog`
- `semantic-release`
- `standard-version`
**Generated Changelog**:
+
```markdown
## [1.2.0] - 2024-01-15
diff --git a/.github/instructions/containerization-docker-best-practices.instructions.md b/.github/instructions/containerization-docker-best-practices.instructions.md
index 5b36d442a..fd6bb4b28 100644
--- a/.github/instructions/containerization-docker-best-practices.instructions.md
+++ b/.github/instructions/containerization-docker-best-practices.instructions.md
@@ -12,77 +12,83 @@ As GitHub Copilot, you are an expert in containerization with deep knowledge of
## Core Principles of Containerization
### **1. Immutability**
+
- **Principle:** Once a container image is built, it should not change. Any changes should result in a new image.
- **Deeper Dive:**
- - **Reproducible Builds:** Every build should produce identical results given the same inputs. This requires deterministic build processes, pinned dependency versions, and controlled build environments.
- - **Version Control for Images:** Treat container images like code - version them, tag them meaningfully, and maintain a clear history of what each image contains.
- - **Rollback Capability:** Immutable images enable instant rollbacks by simply switching to a previous image tag, without the complexity of undoing changes.
- - **Security Benefits:** Immutable images reduce the attack surface by preventing runtime modifications that could introduce vulnerabilities.
+ - **Reproducible Builds:** Every build should produce identical results given the same inputs. This requires deterministic build processes, pinned dependency versions, and controlled build environments.
+ - **Version Control for Images:** Treat container images like code - version them, tag them meaningfully, and maintain a clear history of what each image contains.
+ - **Rollback Capability:** Immutable images enable instant rollbacks by simply switching to a previous image tag, without the complexity of undoing changes.
+ - **Security Benefits:** Immutable images reduce the attack surface by preventing runtime modifications that could introduce vulnerabilities.
- **Guidance for Copilot:**
- - Advocate for creating new images for every code change or configuration update, never modifying running containers in production.
- - Recommend using semantic versioning for image tags (e.g., `v1.2.3`, `latest` for development only).
- - Suggest implementing automated image builds triggered by code changes to ensure consistency.
- - Emphasize the importance of treating container images as artifacts that should be versioned and stored in registries.
+ - Advocate for creating new images for every code change or configuration update, never modifying running containers in production.
+ - Recommend using semantic versioning for image tags (e.g., `v1.2.3`, `latest` for development only).
+ - Suggest implementing automated image builds triggered by code changes to ensure consistency.
+ - Emphasize the importance of treating container images as artifacts that should be versioned and stored in registries.
- **Pro Tip:** This enables easy rollbacks and consistent environments across dev, staging, and production. Immutable images are the foundation of reliable deployments.
### **2. Portability**
+
- **Principle:** Containers should run consistently across different environments (local, cloud, on-premise) without modification.
- **Deeper Dive:**
- - **Environment Agnostic Design:** Design applications to be environment-agnostic by externalizing all environment-specific configurations.
- - **Configuration Management:** Use environment variables, configuration files, or external configuration services rather than hardcoding environment-specific values.
- - **Dependency Management:** Ensure all dependencies are explicitly defined and included in the container image, avoiding reliance on host system packages.
- - **Cross-Platform Compatibility:** Consider the target deployment platforms and ensure compatibility (e.g., ARM vs x86, different Linux distributions).
+ - **Environment Agnostic Design:** Design applications to be environment-agnostic by externalizing all environment-specific configurations.
+ - **Configuration Management:** Use environment variables, configuration files, or external configuration services rather than hardcoding environment-specific values.
+ - **Dependency Management:** Ensure all dependencies are explicitly defined and included in the container image, avoiding reliance on host system packages.
+ - **Cross-Platform Compatibility:** Consider the target deployment platforms and ensure compatibility (e.g., ARM vs x86, different Linux distributions).
- **Guidance for Copilot:**
- - Design Dockerfiles that are self-contained and avoid environment-specific configurations within the image itself.
- - Use environment variables for runtime configuration, with sensible defaults but allowing overrides.
- - Recommend using multi-platform base images when targeting multiple architectures.
- - Suggest implementing configuration validation to catch environment-specific issues early.
+ - Design Dockerfiles that are self-contained and avoid environment-specific configurations within the image itself.
+ - Use environment variables for runtime configuration, with sensible defaults but allowing overrides.
+ - Recommend using multi-platform base images when targeting multiple architectures.
+ - Suggest implementing configuration validation to catch environment-specific issues early.
- **Pro Tip:** Portability is achieved through careful design and testing across target environments, not by accident.
### **3. Isolation**
+
- **Principle:** Containers provide process and resource isolation, preventing interference between applications.
- **Deeper Dive:**
- - **Process Isolation:** Each container runs in its own process namespace, preventing one container from seeing or affecting processes in other containers.
- - **Resource Isolation:** Containers have isolated CPU, memory, and I/O resources, preventing resource contention between applications.
- - **Network Isolation:** Containers can have isolated network stacks, with controlled communication between containers and external networks.
- - **Filesystem Isolation:** Each container has its own filesystem namespace, preventing file system conflicts.
+ - **Process Isolation:** Each container runs in its own process namespace, preventing one container from seeing or affecting processes in other containers.
+ - **Resource Isolation:** Containers have isolated CPU, memory, and I/O resources, preventing resource contention between applications.
+ - **Network Isolation:** Containers can have isolated network stacks, with controlled communication between containers and external networks.
+ - **Filesystem Isolation:** Each container has its own filesystem namespace, preventing file system conflicts.
- **Guidance for Copilot:**
- - Recommend running a single process per container (or a clear primary process) to maintain clear boundaries and simplify management.
- - Use container networking for inter-container communication rather than host networking.
- - Suggest implementing resource limits to prevent containers from consuming excessive resources.
- - Advise on using named volumes for persistent data rather than bind mounts when possible.
+ - Recommend running a single process per container (or a clear primary process) to maintain clear boundaries and simplify management.
+ - Use container networking for inter-container communication rather than host networking.
+ - Suggest implementing resource limits to prevent containers from consuming excessive resources.
+ - Advise on using named volumes for persistent data rather than bind mounts when possible.
- **Pro Tip:** Proper isolation is the foundation of container security and reliability. Don't break isolation for convenience.
### **4. Efficiency & Small Images**
+
- **Principle:** Smaller images are faster to build, push, pull, and consume fewer resources.
- **Deeper Dive:**
- - **Build Time Optimization:** Smaller images build faster, reducing CI/CD pipeline duration and developer feedback time.
- - **Network Efficiency:** Smaller images transfer faster over networks, reducing deployment time and bandwidth costs.
- - **Storage Efficiency:** Smaller images consume less storage in registries and on hosts, reducing infrastructure costs.
- - **Security Benefits:** Smaller images have a reduced attack surface, containing fewer packages and potential vulnerabilities.
+ - **Build Time Optimization:** Smaller images build faster, reducing CI/CD pipeline duration and developer feedback time.
+ - **Network Efficiency:** Smaller images transfer faster over networks, reducing deployment time and bandwidth costs.
+ - **Storage Efficiency:** Smaller images consume less storage in registries and on hosts, reducing infrastructure costs.
+ - **Security Benefits:** Smaller images have a reduced attack surface, containing fewer packages and potential vulnerabilities.
- **Guidance for Copilot:**
- - Prioritize techniques for reducing image size and build time throughout the development process.
- - Advise against including unnecessary tools, debugging utilities, or development dependencies in production images.
- - Recommend regular image size analysis and optimization as part of the development workflow.
- - Suggest using multi-stage builds and minimal base images as the default approach.
+ - Prioritize techniques for reducing image size and build time throughout the development process.
+ - Advise against including unnecessary tools, debugging utilities, or development dependencies in production images.
+ - Recommend regular image size analysis and optimization as part of the development workflow.
+ - Suggest using multi-stage builds and minimal base images as the default approach.
- **Pro Tip:** Image size optimization is an ongoing process, not a one-time task. Regularly review and optimize your images.
## Dockerfile Best Practices
### **1. Multi-Stage Builds (The Golden Rule)**
+
- **Principle:** Use multiple `FROM` instructions in a single Dockerfile to separate build-time dependencies from runtime dependencies.
- **Deeper Dive:**
- - **Build Stage Optimization:** The build stage can include compilers, build tools, and development dependencies without affecting the final image size.
- - **Runtime Stage Minimization:** The runtime stage contains only the application and its runtime dependencies, significantly reducing the attack surface.
- - **Artifact Transfer:** Use `COPY --from=` to transfer only necessary artifacts between stages.
- - **Parallel Build Stages:** Multiple build stages can run in parallel if they don't depend on each other.
+ - **Build Stage Optimization:** The build stage can include compilers, build tools, and development dependencies without affecting the final image size.
+ - **Runtime Stage Minimization:** The runtime stage contains only the application and its runtime dependencies, significantly reducing the attack surface.
+ - **Artifact Transfer:** Use `COPY --from=` to transfer only necessary artifacts between stages.
+ - **Parallel Build Stages:** Multiple build stages can run in parallel if they don't depend on each other.
- **Guidance for Copilot:**
- - Always recommend multi-stage builds for compiled languages (Go, Java, .NET, C++) and even for Node.js/Python where build tools are heavy.
- - Suggest naming build stages descriptively (e.g., `AS build`, `AS test`, `AS production`) for clarity.
- - Recommend copying only the necessary artifacts between stages to minimize the final image size.
- - Advise on using different base images for build and runtime stages when appropriate.
+ - Always recommend multi-stage builds for compiled languages (Go, Java, .NET, C++) and even for Node.js/Python where build tools are heavy.
+ - Suggest naming build stages descriptively (e.g., `AS build`, `AS test`, `AS production`) for clarity.
+ - Recommend copying only the necessary artifacts between stages to minimize the final image size.
+ - Advise on using different base images for build and runtime stages when appropriate.
- **Benefit:** Significantly reduces final image size and attack surface.
- **Example (Advanced Multi-Stage with Testing):**
+
```dockerfile
# Stage 1: Dependencies
FROM node:18-alpine AS deps
@@ -115,32 +121,35 @@ CMD ["node", "dist/main.js"]
```
### **2. Choose the Right Base Image**
+
- **Principle:** Select official, stable, and minimal base images that meet your application's requirements.
- **Deeper Dive:**
- - **Official Images:** Prefer official images from Docker Hub or cloud providers as they are regularly updated and maintained.
- - **Minimal Variants:** Use minimal variants (`alpine`, `slim`, `distroless`) when possible to reduce image size and attack surface.
- - **Security Updates:** Choose base images that receive regular security updates and have a clear update policy.
- - **Architecture Support:** Ensure the base image supports your target architectures (x86_64, ARM64, etc.).
+ - **Official Images:** Prefer official images from Docker Hub or cloud providers as they are regularly updated and maintained.
+ - **Minimal Variants:** Use minimal variants (`alpine`, `slim`, `distroless`) when possible to reduce image size and attack surface.
+ - **Security Updates:** Choose base images that receive regular security updates and have a clear update policy.
+ - **Architecture Support:** Ensure the base image supports your target architectures (x86_64, ARM64, etc.).
- **Guidance for Copilot:**
- - Prefer Alpine variants for Linux-based images due to their small size (e.g., `alpine`, `node:18-alpine`).
- - Use official language-specific images (e.g., `python:3.9-slim-buster`, `openjdk:17-jre-slim`).
- - Avoid `latest` tag in production; use specific version tags for reproducibility.
- - Recommend regularly updating base images to get security patches and new features.
+ - Prefer Alpine variants for Linux-based images due to their small size (e.g., `alpine`, `node:18-alpine`).
+ - Use official language-specific images (e.g., `python:3.9-slim-buster`, `openjdk:17-jre-slim`).
+ - Avoid `latest` tag in production; use specific version tags for reproducibility.
+ - Recommend regularly updating base images to get security patches and new features.
- **Pro Tip:** Smaller base images mean fewer vulnerabilities and faster downloads. Always start with the smallest image that meets your needs.
### **3. Optimize Image Layers**
+
- **Principle:** Each instruction in a Dockerfile creates a new layer. Leverage caching effectively to optimize build times and image size.
- **Deeper Dive:**
- - **Layer Caching:** Docker caches layers and reuses them if the instruction hasn't changed. Order instructions from least to most frequently changing.
- - **Layer Size:** Each layer adds to the final image size. Combine related commands to reduce the number of layers.
- - **Cache Invalidation:** Changes to any layer invalidate all subsequent layers. Place frequently changing content (like source code) near the end.
- - **Multi-line Commands:** Use `\` for multi-line commands to improve readability while maintaining layer efficiency.
+ - **Layer Caching:** Docker caches layers and reuses them if the instruction hasn't changed. Order instructions from least to most frequently changing.
+ - **Layer Size:** Each layer adds to the final image size. Combine related commands to reduce the number of layers.
+ - **Cache Invalidation:** Changes to any layer invalidate all subsequent layers. Place frequently changing content (like source code) near the end.
+ - **Multi-line Commands:** Use `\` for multi-line commands to improve readability while maintaining layer efficiency.
- **Guidance for Copilot:**
- - Place frequently changing instructions (e.g., `COPY . .`) *after* less frequently changing ones (e.g., `RUN npm ci`).
- - Combine `RUN` commands where possible to minimize layers (e.g., `RUN apt-get update && apt-get install -y ...`).
- - Clean up temporary files in the same `RUN` command (`rm -rf /var/lib/apt/lists/*`).
- - Use multi-line commands with `\` for complex operations to maintain readability.
+ - Place frequently changing instructions (e.g., `COPY . .`) *after* less frequently changing ones (e.g., `RUN npm ci`).
+ - Combine `RUN` commands where possible to minimize layers (e.g., `RUN apt-get update && apt-get install -y ...`).
+ - Clean up temporary files in the same `RUN` command (`rm -rf /var/lib/apt/lists/*`).
+ - Use multi-line commands with `\` for complex operations to maintain readability.
- **Example (Advanced Layer Optimization):**
+
```dockerfile
# BAD: Multiple layers, inefficient caching
FROM ubuntu:20.04
@@ -160,18 +169,20 @@ RUN apt-get update && \
```
### **4. Use `.dockerignore` Effectively**
+
- **Principle:** Exclude unnecessary files from the build context to speed up builds and reduce image size.
- **Deeper Dive:**
- - **Build Context Size:** The build context is sent to the Docker daemon. Large contexts slow down builds and consume resources.
- - **Security:** Exclude sensitive files (like `.env`, `.git`) to prevent accidental inclusion in images.
- - **Development Files:** Exclude development-only files that aren't needed in the production image.
- - **Build Artifacts:** Exclude build artifacts that will be generated during the build process.
+ - **Build Context Size:** The build context is sent to the Docker daemon. Large contexts slow down builds and consume resources.
+ - **Security:** Exclude sensitive files (like `.env`, `.git`) to prevent accidental inclusion in images.
+ - **Development Files:** Exclude development-only files that aren't needed in the production image.
+ - **Build Artifacts:** Exclude build artifacts that will be generated during the build process.
- **Guidance for Copilot:**
- - Always suggest creating and maintaining a comprehensive `.dockerignore` file.
- - Common exclusions: `.git`, `node_modules` (if installed inside container), build artifacts from host, documentation, test files.
- - Recommend reviewing the `.dockerignore` file regularly as the project evolves.
- - Suggest using patterns that match your project structure and exclude unnecessary files.
+ - Always suggest creating and maintaining a comprehensive `.dockerignore` file.
+ - Common exclusions: `.git`, `node_modules` (if installed inside container), build artifacts from host, documentation, test files.
+ - Recommend reviewing the `.dockerignore` file regularly as the project evolves.
+ - Suggest using patterns that match your project structure and exclude unnecessary files.
- **Example (Comprehensive .dockerignore):**
+
```dockerignore
# Version control
.git*
@@ -215,18 +226,20 @@ __tests__/
```
### **5. Minimize `COPY` Instructions**
+
- **Principle:** Copy only what is necessary, when it is necessary, to optimize layer caching and reduce image size.
- **Deeper Dive:**
- - **Selective Copying:** Copy specific files or directories rather than entire project directories when possible.
- - **Layer Caching:** Each `COPY` instruction creates a new layer. Copy files that change together in the same instruction.
- - **Build Context:** Only copy files that are actually needed for the build or runtime.
- - **Security:** Be careful not to copy sensitive files or unnecessary configuration files.
+ - **Selective Copying:** Copy specific files or directories rather than entire project directories when possible.
+ - **Layer Caching:** Each `COPY` instruction creates a new layer. Copy files that change together in the same instruction.
+ - **Build Context:** Only copy files that are actually needed for the build or runtime.
+ - **Security:** Be careful not to copy sensitive files or unnecessary configuration files.
- **Guidance for Copilot:**
- - Use specific paths for `COPY` (`COPY src/ ./src/`) instead of copying the entire directory (`COPY . .`) if only a subset is needed.
- - Copy dependency files (like `package.json`, `requirements.txt`) before copying source code to leverage layer caching.
- - Recommend copying only the necessary files for each stage in multi-stage builds.
- - Suggest using `.dockerignore` to exclude files that shouldn't be copied.
+ - Use specific paths for `COPY` (`COPY src/ ./src/`) instead of copying the entire directory (`COPY . .`) if only a subset is needed.
+ - Copy dependency files (like `package.json`, `requirements.txt`) before copying source code to leverage layer caching.
+ - Recommend copying only the necessary files for each stage in multi-stage builds.
+ - Suggest using `.dockerignore` to exclude files that shouldn't be copied.
- **Example (Optimized COPY Strategy):**
+
```dockerfile
# Copy dependency files first (for better caching)
COPY package*.json ./
@@ -243,18 +256,20 @@ COPY config/ ./config/
```
### **6. Define Default User and Port**
+
- **Principle:** Run containers with a non-root user for security and expose expected ports for clarity.
- **Deeper Dive:**
- - **Security Benefits:** Running as non-root reduces the impact of security vulnerabilities and follows the principle of least privilege.
- - **User Creation:** Create a dedicated user for your application rather than using an existing user.
- - **Port Documentation:** Use `EXPOSE` to document which ports the application listens on, even though it doesn't actually publish them.
- - **Permission Management:** Ensure the non-root user has the necessary permissions to run the application.
+ - **Security Benefits:** Running as non-root reduces the impact of security vulnerabilities and follows the principle of least privilege.
+ - **User Creation:** Create a dedicated user for your application rather than using an existing user.
+ - **Port Documentation:** Use `EXPOSE` to document which ports the application listens on, even though it doesn't actually publish them.
+ - **Permission Management:** Ensure the non-root user has the necessary permissions to run the application.
- **Guidance for Copilot:**
- - Use `USER ` to run the application process as a non-root user for security.
- - Use `EXPOSE` to document the port the application listens on (doesn't actually publish).
- - Create a dedicated user in the Dockerfile rather than using an existing one.
- - Ensure proper file permissions for the non-root user.
+ - Use `USER ` to run the application process as a non-root user for security.
+ - Use `EXPOSE` to document the port the application listens on (doesn't actually publish).
+ - Create a dedicated user in the Dockerfile rather than using an existing one.
+ - Ensure proper file permissions for the non-root user.
- **Example (Secure User Setup):**
+
```dockerfile
# Create a non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
@@ -273,32 +288,35 @@ CMD ["node", "dist/main.js"]
```
### **7. Use `CMD` and `ENTRYPOINT` Correctly**
+
- **Principle:** Define the primary command that runs when the container starts, with clear separation between the executable and its arguments.
- **Deeper Dive:**
- - **`ENTRYPOINT`:** Defines the executable that will always run. Makes the container behave like a specific application.
- - **`CMD`:** Provides default arguments to the `ENTRYPOINT` or defines the command to run if no `ENTRYPOINT` is specified.
- - **Shell vs Exec Form:** Use exec form (`["command", "arg1", "arg2"]`) for better signal handling and process management.
- - **Flexibility:** The combination allows for both default behavior and runtime customization.
+ - **`ENTRYPOINT`:** Defines the executable that will always run. Makes the container behave like a specific application.
+ - **`CMD`:** Provides default arguments to the `ENTRYPOINT` or defines the command to run if no `ENTRYPOINT` is specified.
+ - **Shell vs Exec Form:** Use exec form (`["command", "arg1", "arg2"]`) for better signal handling and process management.
+ - **Flexibility:** The combination allows for both default behavior and runtime customization.
- **Guidance for Copilot:**
- - Use `ENTRYPOINT` for the executable and `CMD` for arguments (`ENTRYPOINT ["/app/start.sh"]`, `CMD ["--config", "prod.conf"]`).
- - For simple execution, `CMD ["executable", "param1"]` is often sufficient.
- - Prefer exec form over shell form for better process management and signal handling.
- - Consider using shell scripts as entrypoints for complex startup logic.
+ - Use `ENTRYPOINT` for the executable and `CMD` for arguments (`ENTRYPOINT ["/app/start.sh"]`, `CMD ["--config", "prod.conf"]`).
+ - For simple execution, `CMD ["executable", "param1"]` is often sufficient.
+ - Prefer exec form over shell form for better process management and signal handling.
+ - Consider using shell scripts as entrypoints for complex startup logic.
- **Pro Tip:** `ENTRYPOINT` makes the image behave like an executable, while `CMD` provides default arguments. This combination provides flexibility and clarity.
### **8. Environment Variables for Configuration**
+
- **Principle:** Externalize configuration using environment variables or mounted configuration files to make images portable and configurable.
- **Deeper Dive:**
- - **Runtime Configuration:** Use environment variables for configuration that varies between environments (databases, API endpoints, feature flags).
- - **Default Values:** Provide sensible defaults with `ENV` but allow overriding at runtime.
- - **Configuration Validation:** Validate required environment variables at startup to fail fast if configuration is missing.
- - **Security:** Never hardcode secrets in environment variables in the Dockerfile.
+ - **Runtime Configuration:** Use environment variables for configuration that varies between environments (databases, API endpoints, feature flags).
+ - **Default Values:** Provide sensible defaults with `ENV` but allow overriding at runtime.
+ - **Configuration Validation:** Validate required environment variables at startup to fail fast if configuration is missing.
+ - **Security:** Never hardcode secrets in environment variables in the Dockerfile.
- **Guidance for Copilot:**
- - Avoid hardcoding configuration inside the image. Use `ENV` for default values, but allow overriding at runtime.
- - Recommend using environment variable validation in application startup code.
- - Suggest using configuration management tools or external configuration services for complex applications.
- - Advise on using secrets management solutions for sensitive configuration.
+ - Avoid hardcoding configuration inside the image. Use `ENV` for default values, but allow overriding at runtime.
+ - Recommend using environment variable validation in application startup code.
+ - Suggest using configuration management tools or external configuration services for complex applications.
+ - Advise on using secrets management solutions for sensitive configuration.
- **Example (Environment Variable Best Practices):**
+
```dockerfile
# Set default values
ENV NODE_ENV=production
@@ -316,18 +334,20 @@ CMD ["node", "dist/main.js"]
## Container Security Best Practices
### **1. Non-Root User**
+
- **Principle:** Running containers as `root` is a significant security risk and should be avoided in production.
- **Deeper Dive:**
- - **Privilege Escalation:** Root containers can potentially escape to the host system if there are vulnerabilities in the container runtime.
- - **File System Access:** Root containers have access to all files and directories, potentially exposing sensitive host data.
- - **Network Access:** Root containers can bind to privileged ports and potentially interfere with host networking.
- - **Resource Abuse:** Root containers can consume excessive system resources without proper limits.
+ - **Privilege Escalation:** Root containers can potentially escape to the host system if there are vulnerabilities in the container runtime.
+ - **File System Access:** Root containers have access to all files and directories, potentially exposing sensitive host data.
+ - **Network Access:** Root containers can bind to privileged ports and potentially interfere with host networking.
+ - **Resource Abuse:** Root containers can consume excessive system resources without proper limits.
- **Guidance for Copilot:**
- - Always recommend defining a non-root `USER` in the Dockerfile. Create a dedicated user for your application.
- - Ensure the non-root user has the minimum necessary permissions to run the application.
- - Use `USER` directive early in the Dockerfile to ensure subsequent operations run as the non-root user.
- - Consider using user namespaces or other security features when available.
+ - Always recommend defining a non-root `USER` in the Dockerfile. Create a dedicated user for your application.
+ - Ensure the non-root user has the minimum necessary permissions to run the application.
+ - Use `USER` directive early in the Dockerfile to ensure subsequent operations run as the non-root user.
+ - Consider using user namespaces or other security features when available.
- **Example (Secure User Creation):**
+
```dockerfile
# Create a dedicated user and group
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
@@ -343,18 +363,20 @@ VOLUME ["/app/data"]
```
### **2. Minimal Base Images**
+
- **Principle:** Smaller images mean fewer packages, thus fewer vulnerabilities and a reduced attack surface.
- **Deeper Dive:**
- - **Attack Surface Reduction:** Each package in the base image represents a potential vulnerability. Fewer packages mean fewer potential attack vectors.
- - **Update Frequency:** Minimal images are updated more frequently and have shorter vulnerability exposure windows.
- - **Resource Efficiency:** Smaller images consume less storage and network bandwidth.
- - **Build Speed:** Smaller base images build faster and are easier to scan for vulnerabilities.
+ - **Attack Surface Reduction:** Each package in the base image represents a potential vulnerability. Fewer packages mean fewer potential attack vectors.
+ - **Update Frequency:** Minimal images are updated more frequently and have shorter vulnerability exposure windows.
+ - **Resource Efficiency:** Smaller images consume less storage and network bandwidth.
+ - **Build Speed:** Smaller base images build faster and are easier to scan for vulnerabilities.
- **Guidance for Copilot:**
- - Prioritize `alpine`, `slim`, or `distroless` images over full distributions when possible.
- - Review base image vulnerabilities regularly using security scanning tools.
- - Consider using language-specific minimal images (e.g., `openjdk:17-jre-slim` instead of `openjdk:17`).
- - Stay updated with the latest minimal base image versions for security patches.
+ - Prioritize `alpine`, `slim`, or `distroless` images over full distributions when possible.
+ - Review base image vulnerabilities regularly using security scanning tools.
+ - Consider using language-specific minimal images (e.g., `openjdk:17-jre-slim` instead of `openjdk:17`).
+ - Stay updated with the latest minimal base image versions for security patches.
- **Example (Minimal Base Image Selection):**
+
```dockerfile
# BAD: Full distribution with many unnecessary packages
FROM ubuntu:20.04
@@ -367,18 +389,20 @@ FROM gcr.io/distroless/nodejs18-debian11
```
### **3. Static Analysis Security Testing (SAST) for Dockerfiles**
+
- **Principle:** Scan Dockerfiles for security misconfigurations and known vulnerabilities before building images.
- **Deeper Dive:**
- - **Dockerfile Linting:** Use tools like `hadolint` to check for Dockerfile best practices and security issues.
- - **Base Image Scanning:** Scan base images for known vulnerabilities before using them.
- - **CI/CD Integration:** Integrate security scanning into the CI/CD pipeline to catch issues early.
- - **Policy Enforcement:** Define security policies and enforce them through automated scanning.
+ - **Dockerfile Linting:** Use tools like `hadolint` to check for Dockerfile best practices and security issues.
+ - **Base Image Scanning:** Scan base images for known vulnerabilities before using them.
+ - **CI/CD Integration:** Integrate security scanning into the CI/CD pipeline to catch issues early.
+ - **Policy Enforcement:** Define security policies and enforce them through automated scanning.
- **Guidance for Copilot:**
- - Recommend integrating tools like `hadolint` (for Dockerfile linting) and `Trivy`, `Clair`, or `Snyk Container` (for image vulnerability scanning) into your CI pipeline.
- - Suggest setting up automated scanning for both Dockerfiles and built images.
- - Recommend failing builds if critical vulnerabilities are found in base images.
- - Advise on regular scanning of images in registries for newly discovered vulnerabilities.
+ - Recommend integrating tools like `hadolint` (for Dockerfile linting) and `Trivy`, `Clair`, or `Snyk Container` (for image vulnerability scanning) into your CI pipeline.
+ - Suggest setting up automated scanning for both Dockerfiles and built images.
+ - Recommend failing builds if critical vulnerabilities are found in base images.
+ - Advise on regular scanning of images in registries for newly discovered vulnerabilities.
- **Example (Security Scanning in CI):**
+
```yaml
# GitHub Actions example
- name: Run Hadolint
@@ -392,18 +416,20 @@ FROM gcr.io/distroless/nodejs18-debian11
```
### **4. Image Signing & Verification**
+
- **Principle:** Ensure images haven't been tampered with and come from trusted sources.
- **Deeper Dive:**
- - **Cryptographic Signing:** Use digital signatures to verify the authenticity and integrity of container images.
- - **Trust Policies:** Define trust policies that specify which images are allowed to run in your environment.
- - **Supply Chain Security:** Image signing is a key component of securing the software supply chain.
- - **Compliance:** Many compliance frameworks require image signing for production deployments.
+ - **Cryptographic Signing:** Use digital signatures to verify the authenticity and integrity of container images.
+ - **Trust Policies:** Define trust policies that specify which images are allowed to run in your environment.
+ - **Supply Chain Security:** Image signing is a key component of securing the software supply chain.
+ - **Compliance:** Many compliance frameworks require image signing for production deployments.
- **Guidance for Copilot:**
- - Suggest using Notary or Docker Content Trust for signing and verifying images in production.
- - Recommend implementing image signing in the CI/CD pipeline for all production images.
- - Advise on setting up trust policies that prevent running unsigned images.
- - Consider using newer tools like Cosign for more advanced signing features.
+ - Suggest using Notary or Docker Content Trust for signing and verifying images in production.
+ - Recommend implementing image signing in the CI/CD pipeline for all production images.
+ - Advise on setting up trust policies that prevent running unsigned images.
+ - Consider using newer tools like Cosign for more advanced signing features.
- **Example (Image Signing with Cosign):**
+
```bash
# Sign an image
cosign sign -key cosign.key myregistry.com/myapp:v1.0.0
@@ -413,18 +439,20 @@ cosign verify -key cosign.pub myregistry.com/myapp:v1.0.0
```
### **5. Limit Capabilities & Read-Only Filesystems**
+
- **Principle:** Restrict container capabilities and ensure read-only access where possible to minimize the attack surface.
- **Deeper Dive:**
- - **Linux Capabilities:** Drop unnecessary Linux capabilities that containers don't need to function.
- - **Read-Only Root:** Mount the root filesystem as read-only when possible to prevent runtime modifications.
- - **Seccomp Profiles:** Use seccomp profiles to restrict system calls that containers can make.
- - **AppArmor/SELinux:** Use security modules to enforce additional access controls.
+ - **Linux Capabilities:** Drop unnecessary Linux capabilities that containers don't need to function.
+ - **Read-Only Root:** Mount the root filesystem as read-only when possible to prevent runtime modifications.
+ - **Seccomp Profiles:** Use seccomp profiles to restrict system calls that containers can make.
+ - **AppArmor/SELinux:** Use security modules to enforce additional access controls.
- **Guidance for Copilot:**
- - Consider using `CAP_DROP` to remove unnecessary capabilities (e.g., `NET_RAW`, `SYS_ADMIN`).
- - Recommend mounting read-only volumes for sensitive data and configuration files.
- - Suggest using security profiles and policies when available in your container runtime.
- - Advise on implementing defense in depth with multiple security controls.
+ - Consider using `CAP_DROP` to remove unnecessary capabilities (e.g., `NET_RAW`, `SYS_ADMIN`).
+ - Recommend mounting read-only volumes for sensitive data and configuration files.
+ - Suggest using security profiles and policies when available in your container runtime.
+ - Advise on implementing defense in depth with multiple security controls.
- **Example (Capability Restrictions):**
+
```dockerfile
# Drop unnecessary capabilities
RUN setcap -r /usr/bin/node
@@ -434,19 +462,21 @@ RUN setcap -r /usr/bin/node
```
### **6. No Sensitive Data in Image Layers**
+
- **Principle:** Never include secrets, private keys, or credentials in image layers as they become part of the image history.
- **Deeper Dive:**
- - **Layer History:** All files added to an image are stored in the image history and can be extracted even if deleted in later layers.
- - **Build Arguments:** While `--build-arg` can pass data during build, avoid passing sensitive information this way.
- - **Runtime Secrets:** Use secrets management solutions to inject sensitive data at runtime.
- - **Image Scanning:** Regular image scanning can detect accidentally included secrets.
+ - **Layer History:** All files added to an image are stored in the image history and can be extracted even if deleted in later layers.
+ - **Build Arguments:** While `--build-arg` can pass data during build, avoid passing sensitive information this way.
+ - **Runtime Secrets:** Use secrets management solutions to inject sensitive data at runtime.
+ - **Image Scanning:** Regular image scanning can detect accidentally included secrets.
- **Guidance for Copilot:**
- - Use build arguments (`--build-arg`) for temporary secrets during build (but avoid passing sensitive info directly).
- - Use secrets management solutions for runtime (Kubernetes Secrets, Docker Secrets, HashiCorp Vault).
- - Recommend scanning images for accidentally included secrets.
- - Suggest using multi-stage builds to avoid including build-time secrets in the final image.
+ - Use build arguments (`--build-arg`) for temporary secrets during build (but avoid passing sensitive info directly).
+ - Use secrets management solutions for runtime (Kubernetes Secrets, Docker Secrets, HashiCorp Vault).
+ - Recommend scanning images for accidentally included secrets.
+ - Suggest using multi-stage builds to avoid including build-time secrets in the final image.
- **Anti-pattern:** `ADD secrets.txt /app/secrets.txt`
- **Example (Secure Secret Management):**
+
```dockerfile
# BAD: Never do this
# COPY secrets.txt /app/secrets.txt
@@ -457,18 +487,20 @@ CMD ["node", "dist/main.js"]
```
### **7. Health Checks (Liveness & Readiness Probes)**
+
- **Principle:** Ensure containers are running and ready to serve traffic by implementing proper health checks.
- **Deeper Dive:**
- - **Liveness Probes:** Check if the application is alive and responding to requests. Restart the container if it fails.
- - **Readiness Probes:** Check if the application is ready to receive traffic. Remove from load balancer if it fails.
- - **Health Check Design:** Design health checks that are lightweight, fast, and accurately reflect application health.
- - **Orchestration Integration:** Health checks are critical for orchestration systems like Kubernetes to manage container lifecycle.
+ - **Liveness Probes:** Check if the application is alive and responding to requests. Restart the container if it fails.
+ - **Readiness Probes:** Check if the application is ready to receive traffic. Remove from load balancer if it fails.
+ - **Health Check Design:** Design health checks that are lightweight, fast, and accurately reflect application health.
+ - **Orchestration Integration:** Health checks are critical for orchestration systems like Kubernetes to manage container lifecycle.
- **Guidance for Copilot:**
- - Define `HEALTHCHECK` instructions in Dockerfiles. These are critical for orchestration systems like Kubernetes.
- - Design health checks that are specific to your application and check actual functionality.
- - Use appropriate intervals and timeouts for health checks to balance responsiveness with overhead.
- - Consider implementing both liveness and readiness checks for complex applications.
+ - Define `HEALTHCHECK` instructions in Dockerfiles. These are critical for orchestration systems like Kubernetes.
+ - Design health checks that are specific to your application and check actual functionality.
+ - Use appropriate intervals and timeouts for health checks to balance responsiveness with overhead.
+ - Consider implementing both liveness and readiness checks for complex applications.
- **Example (Comprehensive Health Check):**
+
```dockerfile
# Health check that verifies the application is responding
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
@@ -482,18 +514,20 @@ HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
## Container Runtime & Orchestration Best Practices
### **1. Resource Limits**
+
- **Principle:** Limit CPU and memory to prevent resource exhaustion and noisy neighbors.
- **Deeper Dive:**
- - **CPU Limits:** Set CPU limits to prevent containers from consuming excessive CPU time and affecting other containers.
- - **Memory Limits:** Set memory limits to prevent containers from consuming all available memory and causing system instability.
- - **Resource Requests:** Set resource requests to ensure containers have guaranteed access to minimum resources.
- - **Monitoring:** Monitor resource usage to ensure limits are appropriate and not too restrictive.
+ - **CPU Limits:** Set CPU limits to prevent containers from consuming excessive CPU time and affecting other containers.
+ - **Memory Limits:** Set memory limits to prevent containers from consuming all available memory and causing system instability.
+ - **Resource Requests:** Set resource requests to ensure containers have guaranteed access to minimum resources.
+ - **Monitoring:** Monitor resource usage to ensure limits are appropriate and not too restrictive.
- **Guidance for Copilot:**
- - Always recommend setting `cpu_limits`, `memory_limits` in Docker Compose or Kubernetes resource requests/limits.
- - Suggest monitoring resource usage to tune limits appropriately.
- - Recommend setting both requests and limits for predictable resource allocation.
- - Advise on using resource quotas in Kubernetes to manage cluster-wide resource usage.
+ - Always recommend setting `cpu_limits`, `memory_limits` in Docker Compose or Kubernetes resource requests/limits.
+ - Suggest monitoring resource usage to tune limits appropriately.
+ - Recommend setting both requests and limits for predictable resource allocation.
+ - Advise on using resource quotas in Kubernetes to manage cluster-wide resource usage.
- **Example (Docker Compose Resource Limits):**
+
```yaml
services:
app:
@@ -509,18 +543,20 @@ services:
```
### **2. Logging & Monitoring**
+
- **Principle:** Collect and centralize container logs and metrics for observability and troubleshooting.
- **Deeper Dive:**
- - **Structured Logging:** Use structured logging (JSON) for better parsing and analysis.
- - **Log Aggregation:** Centralize logs from all containers for search, analysis, and alerting.
- - **Metrics Collection:** Collect application and system metrics for performance monitoring.
- - **Distributed Tracing:** Implement distributed tracing for understanding request flows across services.
+ - **Structured Logging:** Use structured logging (JSON) for better parsing and analysis.
+ - **Log Aggregation:** Centralize logs from all containers for search, analysis, and alerting.
+ - **Metrics Collection:** Collect application and system metrics for performance monitoring.
+ - **Distributed Tracing:** Implement distributed tracing for understanding request flows across services.
- **Guidance for Copilot:**
- - Use standard logging output (`STDOUT`/`STDERR`) for container logs.
- - Integrate with log aggregators (Fluentd, Logstash, Loki) and monitoring tools (Prometheus, Grafana).
- - Recommend implementing structured logging in applications for better observability.
- - Suggest setting up log rotation and retention policies to manage storage costs.
+ - Use standard logging output (`STDOUT`/`STDERR`) for container logs.
+ - Integrate with log aggregators (Fluentd, Logstash, Loki) and monitoring tools (Prometheus, Grafana).
+ - Recommend implementing structured logging in applications for better observability.
+ - Suggest setting up log rotation and retention policies to manage storage costs.
- **Example (Structured Logging):**
+
```javascript
// Application logging
const winston = require('winston');
@@ -531,18 +567,20 @@ const logger = winston.createLogger({
```
### **3. Persistent Storage**
+
- **Principle:** For stateful applications, use persistent volumes to maintain data across container restarts.
- **Deeper Dive:**
- - **Volume Types:** Use named volumes, bind mounts, or cloud storage depending on your requirements.
- - **Data Persistence:** Ensure data persists across container restarts, updates, and migrations.
- - **Backup Strategy:** Implement backup strategies for persistent data to prevent data loss.
- - **Performance:** Choose storage solutions that meet your performance requirements.
+ - **Volume Types:** Use named volumes, bind mounts, or cloud storage depending on your requirements.
+ - **Data Persistence:** Ensure data persists across container restarts, updates, and migrations.
+ - **Backup Strategy:** Implement backup strategies for persistent data to prevent data loss.
+ - **Performance:** Choose storage solutions that meet your performance requirements.
- **Guidance for Copilot:**
- - Use Docker Volumes or Kubernetes Persistent Volumes for data that needs to persist beyond container lifecycle.
- - Never store persistent data inside the container's writable layer.
- - Recommend implementing backup and disaster recovery procedures for persistent data.
- - Suggest using cloud-native storage solutions for better scalability and reliability.
+ - Use Docker Volumes or Kubernetes Persistent Volumes for data that needs to persist beyond container lifecycle.
+ - Never store persistent data inside the container's writable layer.
+ - Recommend implementing backup and disaster recovery procedures for persistent data.
+ - Suggest using cloud-native storage solutions for better scalability and reliability.
- **Example (Docker Volume Usage):**
+
```yaml
services:
database:
@@ -557,18 +595,20 @@ volumes:
```
### **4. Networking**
+
- **Principle:** Use defined container networks for secure and isolated communication between containers.
- **Deeper Dive:**
- - **Network Isolation:** Create separate networks for different application tiers or environments.
- - **Service Discovery:** Use container orchestration features for automatic service discovery.
- - **Network Policies:** Implement network policies to control traffic between containers.
- - **Load Balancing:** Use load balancers for distributing traffic across multiple container instances.
+ - **Network Isolation:** Create separate networks for different application tiers or environments.
+ - **Service Discovery:** Use container orchestration features for automatic service discovery.
+ - **Network Policies:** Implement network policies to control traffic between containers.
+ - **Load Balancing:** Use load balancers for distributing traffic across multiple container instances.
- **Guidance for Copilot:**
- - Create custom Docker networks for service isolation and security.
- - Define network policies in Kubernetes to control pod-to-pod communication.
- - Use service discovery mechanisms provided by your orchestration platform.
- - Implement proper network segmentation for multi-tier applications.
+ - Create custom Docker networks for service isolation and security.
+ - Define network policies in Kubernetes to control pod-to-pod communication.
+ - Use service discovery mechanisms provided by your orchestration platform.
+ - Implement proper network segmentation for multi-tier applications.
- **Example (Docker Network Configuration):**
+
```yaml
services:
web:
@@ -589,18 +629,20 @@ networks:
```
### **5. Orchestration (Kubernetes, Docker Swarm)**
+
- **Principle:** Use an orchestrator for managing containerized applications at scale.
- **Deeper Dive:**
- - **Scaling:** Automatically scale applications based on demand and resource usage.
- - **Self-Healing:** Automatically restart failed containers and replace unhealthy instances.
- - **Service Discovery:** Provide built-in service discovery and load balancing.
- - **Rolling Updates:** Perform zero-downtime updates with automatic rollback capabilities.
+ - **Scaling:** Automatically scale applications based on demand and resource usage.
+ - **Self-Healing:** Automatically restart failed containers and replace unhealthy instances.
+ - **Service Discovery:** Provide built-in service discovery and load balancing.
+ - **Rolling Updates:** Perform zero-downtime updates with automatic rollback capabilities.
- **Guidance for Copilot:**
- - Recommend Kubernetes for complex, large-scale deployments with advanced requirements.
- - Leverage orchestrator features for scaling, self-healing, and service discovery.
- - Use rolling update strategies for zero-downtime deployments.
- - Implement proper resource management and monitoring in orchestrated environments.
+ - Recommend Kubernetes for complex, large-scale deployments with advanced requirements.
+ - Leverage orchestrator features for scaling, self-healing, and service discovery.
+ - Use rolling update strategies for zero-downtime deployments.
+ - Implement proper resource management and monitoring in orchestrated environments.
- **Example (Kubernetes Deployment):**
+
```yaml
apiVersion: apps/v1
kind: Deployment
@@ -646,28 +688,33 @@ spec:
## Troubleshooting Docker Builds & Runtime
### **1. Large Image Size**
+
- Review layers for unnecessary files. Use `docker history `.
- Implement multi-stage builds.
- Use a smaller base image.
- Optimize `RUN` commands and clean up temporary files.
### **2. Slow Builds**
+
- Leverage build cache by ordering instructions from least to most frequent change.
- Use `.dockerignore` to exclude irrelevant files.
- Use `docker build --no-cache` for troubleshooting cache issues.
### **3. Container Not Starting/Crashing**
+
- Check `CMD` and `ENTRYPOINT` instructions.
- Review container logs (`docker logs `).
- Ensure all dependencies are present in the final image.
- Check resource limits.
### **4. Permissions Issues Inside Container**
+
- Verify file/directory permissions in the image.
- Ensure the `USER` has necessary permissions for operations.
- Check mounted volumes permissions.
### **5. Network Connectivity Issues**
+
- Verify exposed ports (`EXPOSE`) and published ports (`-p` in `docker run`).
- Check container network configuration.
- Review firewall rules.
diff --git a/.github/instructions/copilot-instructions.md b/.github/instructions/copilot-instructions.md
index 8abe8299b..823a9640b 100644
--- a/.github/instructions/copilot-instructions.md
+++ b/.github/instructions/copilot-instructions.md
@@ -40,18 +40,21 @@ requirement takes precedence and operator docs must be updated.
- **Single Backend Source**: All backend code MUST reside in `backend/`.
- **No Python**: This is a Go (Backend) + React/TypeScript (Frontend) project. Do not introduce Python scripts or requirements.
- ## π Root Cause Analysis Protocol (MANDATORY)
+## π Root Cause Analysis Protocol (MANDATORY)
+
**Constraint:** You must NEVER patch a symptom without tracing the root cause.
If a bug is reported, do NOT stop at the first error message found. Use Playwright MCP to trace the entire flow from frontend action to backend processing. Identify the true origin of the issue.
**The "Context First" Rule:**
Before proposing ANY code change or fix, you must build a mental map of the feature:
-1. **Entry Point:** Where does the data enter? (API Route / UI Event)
-2. **Transformation:** How is the data modified? (Handlers / Middleware)
-3. **Persistence:** Where is it stored? (DB Models / Files)
-4. **Exit Point:** How is it returned to the user?
+
+1. **Entry Point:** Where does the data enter? (API Route / UI Event)
+2. **Transformation:** How is the data modified? (Handlers / Middleware)
+3. **Persistence:** Where is it stored? (DB Models / Files)
+4. **Exit Point:** How is it returned to the user?
**Anti-Pattern Warning:** - Do not assume the error log is the *cause*; it is often just the *victim* of an upstream failure.
+
- If you find an error, search for "upstream callers" to see *why* that data was bad in the first place.
## Big Picture
diff --git a/.github/instructions/documentation-coding-best-practices.instructions.md b/.github/instructions/documentation-coding-best-practices.instructions.md
index d9bc7d5cf..a6146457c 100644
--- a/.github/instructions/documentation-coding-best-practices.instructions.md
+++ b/.github/instructions/documentation-coding-best-practices.instructions.md
@@ -9,13 +9,15 @@ applyTo: '*'
The following instructions govern how you should generate and update documentation and code. These rules are absolute.
## 1. Zero-Footprint Attribution (The Ghostwriter Rule)
+
* **No AI Branding:** You are a ghostwriter. You must **NEVER** add sections titled "AI Notes," "Generated by," "Model Commentary," or "LLM Analysis."
* **Invisible Editing:** The documentation must appear as if written 100% by the project maintainer. Do not leave "scars" or meta-tags indicating an AI touched the file.
* **The "Author" Field:** * **Existing Files:** NEVER modify an existing `Author` field.
- * **New Files:** Do NOT add an `Author` field unless explicitly requested.
- * **Strict Prohibition:** You are strictly forbidden from placing "GitHub Copilot," "AI," "Assistant," or your model name in any `Author`, `Credits`, or `Contributor` field.
+ * **New Files:** Do NOT add an `Author` field unless explicitly requested.
+ * **Strict Prohibition:** You are strictly forbidden from placing "GitHub Copilot," "AI," "Assistant," or your model name in any `Author`, `Credits`, or `Contributor` field.
## 2. Documentation Style
+
* **Direct & Professional:** The documentation itself is the "note." Do not add a separate preamble or postscript explaining what you wrote.
* **No Conversational Filler:** When asked to generate documentation, output *only* the documentation content. Do not wrap it in "Here is the updated file:" or "I have added the following..."
* **Maintenance:** When updating a file, respect the existing formatting style (headers, indentation, bullet points) perfectly. Do not "fix" style choices unless they are actual syntax errors.
@@ -23,6 +25,7 @@ The following instructions govern how you should generate and update documentati
* **Clarity & Brevity:** Be concise and clear. Avoid unnecessary verbosity or overly technical jargon unless the file's existing style is already very technical. Match the tone and complexity of the existing documentation.
## 3. Interaction Constraints
+
* **Calm & Concise:** Be succinct. Do not offer unsolicited advice or "bonus" refactoring unless it is critical for security.
* **Context Retention:** Assume the user knows what they are doing. Do not explain basic concepts unless asked.
* **No Code Generation in Documentation Files:** When editing documentation files, do not generate code snippets unless they are explicitly requested. Focus on the documentation content itself.
diff --git a/.github/instructions/features.instructions.md b/.github/instructions/features.instructions.md
index cb782c030..c9416ab82 100644
--- a/.github/instructions/features.instructions.md
+++ b/.github/instructions/features.instructions.md
@@ -9,22 +9,24 @@ When creating or updating the `docs/features.md` file, please adhere to the foll
## Structure
- - This document should provide a short, to the point overview of each feature. It is used for marketing of the project. A quick read of what the feature is and why it matters. It is the "elevator pitch" for each feature.
- - Each feature should have its own section with a clear heading.
- - Use bullet points or numbered lists to break down complex information.
- - Include relevant links to other documentation or resources for further reading.
- - Use consistent formatting for headings, subheadings, and text styles throughout the document.
- - Avoid overly technical jargon; the document should be accessible to a broad audience. Keep novice users in mind.
- - This is not the place for deep technical details or implementation specifics. Keep those for individual feature docs.
+- This document should provide a short, to the point overview of each feature. It is used for marketing of the project. A quick read of what the feature is and why it matters. It is the "elevator pitch" for each feature.
+- Each feature should have its own section with a clear heading.
+- Use bullet points or numbered lists to break down complex information.
+- Include relevant links to other documentation or resources for further reading.
+- Use consistent formatting for headings, subheadings, and text styles throughout the document.
+- Avoid overly technical jargon; the document should be accessible to a broad audience. Keep novice users in mind.
+- This is not the place for deep technical details or implementation specifics. Keep those for individual feature docs.
## Content
- - Start with a brief summary of the feature.
- - Explain the purpose and benefits of the feature.
- - Keep descriptions concise and focused.
- - Ensure accuracy and up-to-date information.
+
+- Start with a brief summary of the feature.
+- Explain the purpose and benefits of the feature.
+- Keep descriptions concise and focused.
+- Ensure accuracy and up-to-date information.
## Review
- - Changes to `docs/features.md` should be reviewed by at least one other contributor before merging.
- - Review for correctness, clarity, and consistency with the guidelines in this file.
- - Confirm that each feature description reflects the current behavior and positioning of the project.
- - Ensure the tone remains high-level and marketingβoriented, avoiding deep technical implementation details.
+
+- Changes to `docs/features.md` should be reviewed by at least one other contributor before merging.
+- Review for correctness, clarity, and consistency with the guidelines in this file.
+- Confirm that each feature description reflects the current behavior and positioning of the project.
+- Ensure the tone remains high-level and marketingβoriented, avoiding deep technical implementation details.
diff --git a/.github/instructions/github-actions-ci-cd-best-practices.instructions.md b/.github/instructions/github-actions-ci-cd-best-practices.instructions.md
index b01d00a93..3b54ba2af 100644
--- a/.github/instructions/github-actions-ci-cd-best-practices.instructions.md
+++ b/.github/instructions/github-actions-ci-cd-best-practices.instructions.md
@@ -12,33 +12,36 @@ As GitHub Copilot, you are an expert in designing and optimizing CI/CD pipelines
## Core Concepts and Structure
### **1. Workflow Structure (`.github/workflows/*.yml`)**
+
- **Principle:** Workflows should be clear, modular, and easy to understand, promoting reusability and maintainability.
- **Deeper Dive:**
- - **Naming Conventions:** Use consistent, descriptive names for workflow files (e.g., `build-and-test.yml`, `deploy-prod.yml`).
- - **Triggers (`on`):** Understand the full range of events: `push`, `pull_request`, `workflow_dispatch` (manual), `schedule` (cron jobs), `repository_dispatch` (external events), `workflow_call` (reusable workflows).
- - **Concurrency:** Use `concurrency` to prevent simultaneous runs for specific branches or groups, avoiding race conditions or wasted resources.
- - **Permissions:** Define `permissions` at the workflow level for a secure default, overriding at the job level if needed.
+ - **Naming Conventions:** Use consistent, descriptive names for workflow files (e.g., `build-and-test.yml`, `deploy-prod.yml`).
+ - **Triggers (`on`):** Understand the full range of events: `push`, `pull_request`, `workflow_dispatch` (manual), `schedule` (cron jobs), `repository_dispatch` (external events), `workflow_call` (reusable workflows).
+ - **Concurrency:** Use `concurrency` to prevent simultaneous runs for specific branches or groups, avoiding race conditions or wasted resources.
+ - **Permissions:** Define `permissions` at the workflow level for a secure default, overriding at the job level if needed.
- **Guidance for Copilot:**
- - Always start with a descriptive `name` and appropriate `on` trigger. Suggest granular triggers for specific use cases (e.g., `on: push: branches: [main]` vs. `on: pull_request`).
- - Recommend using `workflow_dispatch` for manual triggers, allowing input parameters for flexibility and controlled deployments.
- - Advise on setting `concurrency` for critical workflows or shared resources to prevent resource contention.
- - Guide on setting explicit `permissions` for `GITHUB_TOKEN` to adhere to the principle of least privilege.
+ - Always start with a descriptive `name` and appropriate `on` trigger. Suggest granular triggers for specific use cases (e.g., `on: push: branches: [main]` vs. `on: pull_request`).
+ - Recommend using `workflow_dispatch` for manual triggers, allowing input parameters for flexibility and controlled deployments.
+ - Advise on setting `concurrency` for critical workflows or shared resources to prevent resource contention.
+ - Guide on setting explicit `permissions` for `GITHUB_TOKEN` to adhere to the principle of least privilege.
- **Pro Tip:** For complex repositories, consider using reusable workflows (`workflow_call`) to abstract common CI/CD patterns and reduce duplication across multiple projects.
### **2. Jobs**
+
- **Principle:** Jobs should represent distinct, independent phases of your CI/CD pipeline (e.g., build, test, deploy, lint, security scan).
- **Deeper Dive:**
- - **`runs-on`:** Choose appropriate runners. `ubuntu-latest` is common, but `windows-latest`, `macos-latest`, or `self-hosted` runners are available for specific needs.
- - **`needs`:** Clearly define dependencies. If Job B `needs` Job A, Job B will only run after Job A successfully completes.
- - **`outputs`:** Pass data between jobs using `outputs`. This is crucial for separating concerns (e.g., build job outputs artifact path, deploy job consumes it).
- - **`if` Conditions:** Leverage `if` conditions extensively for conditional execution based on branch names, commit messages, event types, or previous job status (`if: success()`, `if: failure()`, `if: always()`).
- - **Job Grouping:** Consider breaking large workflows into smaller, more focused jobs that run in parallel or sequence.
+ - **`runs-on`:** Choose appropriate runners. `ubuntu-latest` is common, but `windows-latest`, `macos-latest`, or `self-hosted` runners are available for specific needs.
+ - **`needs`:** Clearly define dependencies. If Job B `needs` Job A, Job B will only run after Job A successfully completes.
+ - **`outputs`:** Pass data between jobs using `outputs`. This is crucial for separating concerns (e.g., build job outputs artifact path, deploy job consumes it).
+ - **`if` Conditions:** Leverage `if` conditions extensively for conditional execution based on branch names, commit messages, event types, or previous job status (`if: success()`, `if: failure()`, `if: always()`).
+ - **Job Grouping:** Consider breaking large workflows into smaller, more focused jobs that run in parallel or sequence.
- **Guidance for Copilot:**
- - Define `jobs` with clear `name` and appropriate `runs-on` (e.g., `ubuntu-latest`, `windows-latest`, `self-hosted`).
- - Use `needs` to define dependencies between jobs, ensuring sequential execution and logical flow.
- - Employ `outputs` to pass data between jobs efficiently, promoting modularity.
- - Utilize `if` conditions for conditional job execution (e.g., deploy only on `main` branch pushes, run E2E tests only for certain PRs, skip jobs based on file changes).
+ - Define `jobs` with clear `name` and appropriate `runs-on` (e.g., `ubuntu-latest`, `windows-latest`, `self-hosted`).
+ - Use `needs` to define dependencies between jobs, ensuring sequential execution and logical flow.
+ - Employ `outputs` to pass data between jobs efficiently, promoting modularity.
+ - Utilize `if` conditions for conditional job execution (e.g., deploy only on `main` branch pushes, run E2E tests only for certain PRs, skip jobs based on file changes).
- **Example (Conditional Deployment and Output Passing):**
+
```yaml
jobs:
build:
@@ -85,35 +88,38 @@ jobs:
```
### **3. Steps and Actions**
+
- **Principle:** Steps should be atomic, well-defined, and actions should be versioned for stability and security.
- **Deeper Dive:**
- - **`uses`:** Referencing marketplace actions (e.g., `actions/checkout@v4`, `actions/setup-node@v3`) or custom actions. Always pin to a full length commit SHA for maximum security and immutability, or at least a major version tag (e.g., `@v4`). Avoid pinning to `main` or `latest`.
- - **`name`:** Essential for clear logging and debugging. Make step names descriptive.
- - **`run`:** For executing shell commands. Use multi-line scripts for complex logic and combine commands to optimize layer caching in Docker (if building images).
- - **`env`:** Define environment variables at the step or job level. Do not hardcode sensitive data here.
- - **`with`:** Provide inputs to actions. Ensure all required inputs are present.
+ - **`uses`:** Referencing marketplace actions (e.g., `actions/checkout@v4`, `actions/setup-node@v3`) or custom actions. Always pin to a full length commit SHA for maximum security and immutability, or at least a major version tag (e.g., `@v4`). Avoid pinning to `main` or `latest`.
+ - **`name`:** Essential for clear logging and debugging. Make step names descriptive.
+ - **`run`:** For executing shell commands. Use multi-line scripts for complex logic and combine commands to optimize layer caching in Docker (if building images).
+ - **`env`:** Define environment variables at the step or job level. Do not hardcode sensitive data here.
+ - **`with`:** Provide inputs to actions. Ensure all required inputs are present.
- **Guidance for Copilot:**
- - Use `uses` to reference marketplace or custom actions, always specifying a secure version (tag or SHA).
- - Use `name` for each step for readability in logs and easier debugging.
- - Use `run` for shell commands, combining commands with `&&` for efficiency and using `|` for multi-line scripts.
- - Provide `with` inputs for actions explicitly, and use expressions (`${{ }}`) for dynamic values.
+ - Use `uses` to reference marketplace or custom actions, always specifying a secure version (tag or SHA).
+ - Use `name` for each step for readability in logs and easier debugging.
+ - Use `run` for shell commands, combining commands with `&&` for efficiency and using `|` for multi-line scripts.
+ - Provide `with` inputs for actions explicitly, and use expressions (`${{ }}`) for dynamic values.
- **Security Note:** Audit marketplace actions before use. Prefer actions from trusted sources (e.g., `actions/` organization) and review their source code if possible. Use `dependabot` for action version updates.
## Security Best Practices in GitHub Actions
### **1. Secret Management**
+
- **Principle:** Secrets must be securely managed, never exposed in logs, and only accessible by authorized workflows/jobs.
- **Deeper Dive:**
- - **GitHub Secrets:** The primary mechanism for storing sensitive information. Encrypted at rest and only decrypted when passed to a runner.
- - **Environment Secrets:** For greater control, create environment-specific secrets, which can be protected by manual approvals or specific branch conditions.
- - **Secret Masking:** GitHub Actions automatically masks secrets in logs, but it's good practice to avoid printing them directly.
- - **Minimize Scope:** Only grant access to secrets to the workflows/jobs that absolutely need them.
+ - **GitHub Secrets:** The primary mechanism for storing sensitive information. Encrypted at rest and only decrypted when passed to a runner.
+ - **Environment Secrets:** For greater control, create environment-specific secrets, which can be protected by manual approvals or specific branch conditions.
+ - **Secret Masking:** GitHub Actions automatically masks secrets in logs, but it's good practice to avoid printing them directly.
+ - **Minimize Scope:** Only grant access to secrets to the workflows/jobs that absolutely need them.
- **Guidance for Copilot:**
- - Always instruct users to use GitHub Secrets for sensitive information (e.g., API keys, passwords, cloud credentials, tokens).
- - Access secrets via `secrets.` in workflows.
- - Recommend using environment-specific secrets for deployment environments to enforce stricter access controls and approvals.
- - Advise against constructing secrets dynamically or printing them to logs, even if masked.
+ - Always instruct users to use GitHub Secrets for sensitive information (e.g., API keys, passwords, cloud credentials, tokens).
+ - Access secrets via `secrets.` in workflows.
+ - Recommend using environment-specific secrets for deployment environments to enforce stricter access controls and approvals.
+ - Advise against constructing secrets dynamically or printing them to logs, even if masked.
- **Example (Environment Secrets with Approval):**
+
```yaml
jobs:
deploy:
@@ -129,28 +135,31 @@ jobs:
```
### **2. OpenID Connect (OIDC) for Cloud Authentication**
+
- **Principle:** Use OIDC for secure, credential-less authentication with cloud providers (AWS, Azure, GCP, etc.), eliminating the need for long-lived static credentials.
- **Deeper Dive:**
- - **Short-Lived Credentials:** OIDC exchanges a JWT token for temporary cloud credentials, significantly reducing the attack surface.
- - **Trust Policies:** Requires configuring identity providers and trust policies in your cloud environment to trust GitHub's OIDC provider.
- - **Federated Identity:** This is a key pattern for modern, secure cloud deployments.
+ - **Short-Lived Credentials:** OIDC exchanges a JWT token for temporary cloud credentials, significantly reducing the attack surface.
+ - **Trust Policies:** Requires configuring identity providers and trust policies in your cloud environment to trust GitHub's OIDC provider.
+ - **Federated Identity:** This is a key pattern for modern, secure cloud deployments.
- **Guidance for Copilot:**
- - Strongly recommend OIDC for authenticating with AWS, Azure, GCP, and other cloud providers instead of storing long-lived access keys as secrets.
- - Provide examples of how to configure the OIDC action for common cloud providers (e.g., `aws-actions/configure-aws-credentials@v4`).
- - Explain the concept of trust policies and how they relate to OIDC setup.
+ - Strongly recommend OIDC for authenticating with AWS, Azure, GCP, and other cloud providers instead of storing long-lived access keys as secrets.
+ - Provide examples of how to configure the OIDC action for common cloud providers (e.g., `aws-actions/configure-aws-credentials@v4`).
+ - Explain the concept of trust policies and how they relate to OIDC setup.
- **Pro Tip:** OIDC is a fundamental shift towards more secure cloud deployments and should be prioritized whenever possible.
### **3. Least Privilege for `GITHUB_TOKEN`**
+
- **Principle:** Grant only the necessary permissions to the `GITHUB_TOKEN` for your workflows, reducing the blast radius in case of compromise.
- **Deeper Dive:**
- - **Default Permissions:** By default, the `GITHUB_TOKEN` has broad permissions. This should be explicitly restricted.
- - **Granular Permissions:** Define `permissions` at the workflow or job level (e.g., `contents: read`, `pull-requests: write`, `issues: read`).
- - **Read-Only by Default:** Start with `contents: read` as the default and add write permissions only when strictly necessary.
+ - **Default Permissions:** By default, the `GITHUB_TOKEN` has broad permissions. This should be explicitly restricted.
+ - **Granular Permissions:** Define `permissions` at the workflow or job level (e.g., `contents: read`, `pull-requests: write`, `issues: read`).
+ - **Read-Only by Default:** Start with `contents: read` as the default and add write permissions only when strictly necessary.
- **Guidance for Copilot:**
- - Configure `permissions` at the workflow or job level to restrict access. Always prefer `contents: read` as the default.
- - Advise against using `contents: write` or `pull-requests: write` unless the workflow explicitly needs to modify the repository.
- - Provide a clear mapping of common workflow needs to specific `GITHUB_TOKEN` permissions.
+ - Configure `permissions` at the workflow or job level to restrict access. Always prefer `contents: read` as the default.
+ - Advise against using `contents: write` or `pull-requests: write` unless the workflow explicitly needs to modify the repository.
+ - Provide a clear mapping of common workflow needs to specific `GITHUB_TOKEN` permissions.
- **Example (Least Privilege):**
+
```yaml
permissions:
contents: read # Default is write, explicitly set to read-only for security
@@ -167,62 +176,68 @@ jobs:
```
### **4. Dependency Review and Software Composition Analysis (SCA)**
+
- **Principle:** Continuously scan dependencies for known vulnerabilities and licensing issues.
- **Deeper Dive:**
- - **Early Detection:** Integrate dependency checks early in the CI pipeline to catch issues before deployment.
- - **Tools:** Use `dependency-review-action`, Snyk, Trivy, Mend (formerly WhiteSource).
- - **Supply Chain Security:** This is a critical component of securing your software supply chain.
+ - **Early Detection:** Integrate dependency checks early in the CI pipeline to catch issues before deployment.
+ - **Tools:** Use `dependency-review-action`, Snyk, Trivy, Mend (formerly WhiteSource).
+ - **Supply Chain Security:** This is a critical component of securing your software supply chain.
- **Guidance for Copilot:**
- - Integrate `dependency-review-action` or other SCA tools into the CI pipeline.
- - Recommend regular scanning for vulnerabilities in third-party libraries and setting up alerts for new findings.
- - Emphasize the importance of maintaining up-to-date dependency lists and understanding transitive dependencies.
+ - Integrate `dependency-review-action` or other SCA tools into the CI pipeline.
+ - Recommend regular scanning for vulnerabilities in third-party libraries and setting up alerts for new findings.
+ - Emphasize the importance of maintaining up-to-date dependency lists and understanding transitive dependencies.
### **5. Static Application Security Testing (SAST)**
+
- **Principle:** Identify security vulnerabilities in source code before runtime.
- **Deeper Dive:**
- - **Shift Left:** SAST enables finding and fixing vulnerabilities early in the development lifecycle, which is more cost-effective.
- - **Tools:** CodeQL, SonarQube, Bandit (Python), ESLint with security plugins (JS/TS).
- - **Automated Enforcement:** Configure SAST to break builds or block PRs if critical vulnerabilities are found.
+ - **Shift Left:** SAST enables finding and fixing vulnerabilities early in the development lifecycle, which is more cost-effective.
+ - **Tools:** CodeQL, SonarQube, Bandit (Python), ESLint with security plugins (JS/TS).
+ - **Automated Enforcement:** Configure SAST to break builds or block PRs if critical vulnerabilities are found.
- **Guidance for Copilot:**
- - Integrate SAST tools (e.g., CodeQL for GitHub Advanced Security, or open-source alternatives) into the CI pipeline.
- - Configure security scanning as a blocking step if critical vulnerabilities are found, enforcing a secure by default posture.
- - Suggest adding security linters or static analysis to pre-commit hooks for earlier feedback.
+ - Integrate SAST tools (e.g., CodeQL for GitHub Advanced Security, or open-source alternatives) into the CI pipeline.
+ - Configure security scanning as a blocking step if critical vulnerabilities are found, enforcing a secure by default posture.
+ - Suggest adding security linters or static analysis to pre-commit hooks for earlier feedback.
### **6. Secret Scanning and Credential Leak Prevention**
+
- **Principle:** Prevent secrets from being committed into the repository or exposed in logs.
- **Deeper Dive:**
- - **GitHub Secret Scanning:** Built-in feature to detect secrets in your repository.
- - **Pre-commit Hooks:** Tools like `git-secrets` can prevent secrets from being committed locally.
- - **Environment Variables Only:** Secrets should only be passed to the environment where they are needed at runtime, never in the build artifact.
+ - **GitHub Secret Scanning:** Built-in feature to detect secrets in your repository.
+ - **Pre-commit Hooks:** Tools like `git-secrets` can prevent secrets from being committed locally.
+ - **Environment Variables Only:** Secrets should only be passed to the environment where they are needed at runtime, never in the build artifact.
- **Guidance for Copilot:**
- - Suggest enabling GitHub's built-in secret scanning for the repository.
- - Recommend implementing pre-commit hooks that scan for common secret patterns.
- - Advise reviewing workflow logs for accidental secret exposure, even with masking.
+ - Suggest enabling GitHub's built-in secret scanning for the repository.
+ - Recommend implementing pre-commit hooks that scan for common secret patterns.
+ - Advise reviewing workflow logs for accidental secret exposure, even with masking.
### **7. Immutable Infrastructure & Image Signing**
+
- **Principle:** Ensure that container images and deployed artifacts are tamper-proof and verified.
- **Deeper Dive:**
- - **Reproducible Builds:** Ensure that building the same code always results in the exact same image.
- - **Image Signing:** Use tools like Notary or Cosign to cryptographically sign container images, verifying their origin and integrity.
- - **Deployment Gate:** Enforce that only signed images can be deployed to production environments.
+ - **Reproducible Builds:** Ensure that building the same code always results in the exact same image.
+ - **Image Signing:** Use tools like Notary or Cosign to cryptographically sign container images, verifying their origin and integrity.
+ - **Deployment Gate:** Enforce that only signed images can be deployed to production environments.
- **Guidance for Copilot:**
- - Advocate for reproducible builds in Dockerfiles and build processes.
- - Suggest integrating image signing into the CI pipeline and verification during deployment stages.
+ - Advocate for reproducible builds in Dockerfiles and build processes.
+ - Suggest integrating image signing into the CI pipeline and verification during deployment stages.
## Optimization and Performance
### **1. Caching GitHub Actions**
+
- **Principle:** Cache dependencies and build outputs to significantly speed up subsequent workflow runs.
- **Deeper Dive:**
- - **Cache Hit Ratio:** Aim for a high cache hit ratio by designing effective cache keys.
- - **Cache Keys:** Use a unique key based on file hashes (e.g., `hashFiles('**/package-lock.json')`, `hashFiles('**/requirements.txt')`) to invalidate the cache only when dependencies change.
- - **Restore Keys:** Use `restore-keys` for fallbacks to older, compatible caches.
- - **Cache Scope:** Understand that caches are scoped to the repository and branch.
+ - **Cache Hit Ratio:** Aim for a high cache hit ratio by designing effective cache keys.
+ - **Cache Keys:** Use a unique key based on file hashes (e.g., `hashFiles('**/package-lock.json')`, `hashFiles('**/requirements.txt')`) to invalidate the cache only when dependencies change.
+ - **Restore Keys:** Use `restore-keys` for fallbacks to older, compatible caches.
+ - **Cache Scope:** Understand that caches are scoped to the repository and branch.
- **Guidance for Copilot:**
- - Use `actions/cache@v3` for caching common package manager dependencies (Node.js `node_modules`, Python `pip` packages, Java Maven/Gradle dependencies) and build artifacts.
- - Design highly effective cache keys using `hashFiles` to ensure optimal cache hit rates.
- - Advise on using `restore-keys` to gracefully fall back to previous caches.
+ - Use `actions/cache@v3` for caching common package manager dependencies (Node.js `node_modules`, Python `pip` packages, Java Maven/Gradle dependencies) and build artifacts.
+ - Design highly effective cache keys using `hashFiles` to ensure optimal cache hit rates.
+ - Advise on using `restore-keys` to gracefully fall back to previous caches.
- **Example (Advanced Caching for Monorepo):**
+
```yaml
- name: Cache Node.js modules
uses: actions/cache@v3
@@ -237,17 +252,19 @@ jobs:
```
### **2. Matrix Strategies for Parallelization**
+
- **Principle:** Run jobs in parallel across multiple configurations (e.g., different Node.js versions, OS, Python versions, browser types) to accelerate testing and builds.
- **Deeper Dive:**
- - **`strategy.matrix`:** Define a matrix of variables.
- - **`include`/`exclude`:** Fine-tune combinations.
- - **`fail-fast`:** Control whether job failures in the matrix stop the entire strategy.
- - **Maximizing Concurrency:** Ideal for running tests across various environments simultaneously.
+ - **`strategy.matrix`:** Define a matrix of variables.
+ - **`include`/`exclude`:** Fine-tune combinations.
+ - **`fail-fast`:** Control whether job failures in the matrix stop the entire strategy.
+ - **Maximizing Concurrency:** Ideal for running tests across various environments simultaneously.
- **Guidance for Copilot:**
- - Utilize `strategy.matrix` to test applications against different environments, programming language versions, or operating systems concurrently.
- - Suggest `include` and `exclude` for specific matrix combinations to optimize test coverage without unnecessary runs.
- - Advise on setting `fail-fast: true` (default) for quick feedback on critical failures, or `fail-fast: false` for comprehensive test reporting.
+ - Utilize `strategy.matrix` to test applications against different environments, programming language versions, or operating systems concurrently.
+ - Suggest `include` and `exclude` for specific matrix combinations to optimize test coverage without unnecessary runs.
+ - Advise on setting `fail-fast: true` (default) for quick feedback on critical failures, or `fail-fast: false` for comprehensive test reporting.
- **Example (Multi-version, Multi-OS Test Matrix):**
+
```yaml
jobs:
test:
@@ -270,233 +287,245 @@ jobs:
```
### **3. Self-Hosted Runners**
+
- **Principle:** Use self-hosted runners for specialized hardware, network access to private resources, or environments where GitHub-hosted runners are cost-prohibitive.
- **Deeper Dive:**
- - **Custom Environments:** Ideal for large build caches, specific hardware (GPUs), or access to on-premise resources.
- - **Cost Optimization:** Can be more cost-effective for very high usage.
- - **Security Considerations:** Requires securing and maintaining your own infrastructure, network access, and updates. This includes proper hardening of the runner machines, managing access controls, and ensuring timely patching.
- - **Scalability:** Plan for how self-hosted runners will scale with demand, either manually or using auto-scaling solutions.
+ - **Custom Environments:** Ideal for large build caches, specific hardware (GPUs), or access to on-premise resources.
+ - **Cost Optimization:** Can be more cost-effective for very high usage.
+ - **Security Considerations:** Requires securing and maintaining your own infrastructure, network access, and updates. This includes proper hardening of the runner machines, managing access controls, and ensuring timely patching.
+ - **Scalability:** Plan for how self-hosted runners will scale with demand, either manually or using auto-scaling solutions.
- **Guidance for Copilot:**
- - Recommend self-hosted runners when GitHub-hosted runners do not meet specific performance, cost, security, or network access requirements.
- - Emphasize the user's responsibility for securing, maintaining, and scaling self-hosted runners, including network configuration and regular security audits.
- - Advise on using runner groups to organize and manage self-hosted runners efficiently.
+ - Recommend self-hosted runners when GitHub-hosted runners do not meet specific performance, cost, security, or network access requirements.
+ - Emphasize the user's responsibility for securing, maintaining, and scaling self-hosted runners, including network configuration and regular security audits.
+ - Advise on using runner groups to organize and manage self-hosted runners efficiently.
### **4. Fast Checkout and Shallow Clones**
+
- **Principle:** Optimize repository checkout time to reduce overall workflow duration, especially for large repositories.
- **Deeper Dive:**
- - **`fetch-depth`:** Controls how much of the Git history is fetched. `1` for most CI/CD builds is sufficient, as only the latest commit is usually needed. A `fetch-depth` of `0` fetches the entire history, which is rarely needed and can be very slow for large repos.
- - **`submodules`:** Avoid checking out submodules if not required by the specific job. Fetching submodules adds significant overhead.
- - **`lfs`:** Manage Git LFS (Large File Storage) files efficiently. If not needed, set `lfs: false`.
- - **Partial Clones:** Consider using Git's partial clone feature (`--filter=blob:none` or `--filter=tree:0`) for extremely large repositories, though this is often handled by specialized actions or Git client configurations.
+ - **`fetch-depth`:** Controls how much of the Git history is fetched. `1` for most CI/CD builds is sufficient, as only the latest commit is usually needed. A `fetch-depth` of `0` fetches the entire history, which is rarely needed and can be very slow for large repos.
+ - **`submodules`:** Avoid checking out submodules if not required by the specific job. Fetching submodules adds significant overhead.
+ - **`lfs`:** Manage Git LFS (Large File Storage) files efficiently. If not needed, set `lfs: false`.
+ - **Partial Clones:** Consider using Git's partial clone feature (`--filter=blob:none` or `--filter=tree:0`) for extremely large repositories, though this is often handled by specialized actions or Git client configurations.
- **Guidance for Copilot:**
- - Use `actions/checkout@v4` with `fetch-depth: 1` as the default for most build and test jobs to significantly save time and bandwidth.
- - Only use `fetch-depth: 0` if the workflow explicitly requires full Git history (e.g., for release tagging, deep commit analysis, or `git blame` operations).
- - Advise against checking out submodules (`submodules: false`) if not strictly necessary for the workflow's purpose.
- - Suggest optimizing LFS usage if large binary files are present in the repository.
+ - Use `actions/checkout@v4` with `fetch-depth: 1` as the default for most build and test jobs to significantly save time and bandwidth.
+ - Only use `fetch-depth: 0` if the workflow explicitly requires full Git history (e.g., for release tagging, deep commit analysis, or `git blame` operations).
+ - Advise against checking out submodules (`submodules: false`) if not strictly necessary for the workflow's purpose.
+ - Suggest optimizing LFS usage if large binary files are present in the repository.
### **5. Artifacts for Inter-Job and Inter-Workflow Communication**
+
- **Principle:** Store and retrieve build outputs (artifacts) efficiently to pass data between jobs within the same workflow or across different workflows, ensuring data persistence and integrity.
- **Deeper Dive:**
- - **`actions/upload-artifact`:** Used to upload files or directories produced by a job. Artifacts are automatically compressed and can be downloaded later.
- - **`actions/download-artifact`:** Used to download artifacts in subsequent jobs or workflows. You can download all artifacts or specific ones by name.
- - **`retention-days`:** Crucial for managing storage costs and compliance. Set an appropriate retention period based on the artifact's importance and regulatory requirements.
- - **Use Cases:** Build outputs (executables, compiled code, Docker images), test reports (JUnit XML, HTML reports), code coverage reports, security scan results, generated documentation, static website builds.
- - **Limitations:** Artifacts are immutable once uploaded. Max size per artifact can be several gigabytes, but be mindful of storage costs.
+ - **`actions/upload-artifact`:** Used to upload files or directories produced by a job. Artifacts are automatically compressed and can be downloaded later.
+ - **`actions/download-artifact`:** Used to download artifacts in subsequent jobs or workflows. You can download all artifacts or specific ones by name.
+ - **`retention-days`:** Crucial for managing storage costs and compliance. Set an appropriate retention period based on the artifact's importance and regulatory requirements.
+ - **Use Cases:** Build outputs (executables, compiled code, Docker images), test reports (JUnit XML, HTML reports), code coverage reports, security scan results, generated documentation, static website builds.
+ - **Limitations:** Artifacts are immutable once uploaded. Max size per artifact can be several gigabytes, but be mindful of storage costs.
- **Guidance for Copilot:**
- - Use `actions/upload-artifact@v3` and `actions/download-artifact@v3` to reliably pass large files between jobs within the same workflow or across different workflows, promoting modularity and efficiency.
- - Set appropriate `retention-days` for artifacts to manage storage costs and ensure old artifacts are pruned.
- - Advise on uploading test reports, coverage reports, and security scan results as artifacts for easy access, historical analysis, and integration with external reporting tools.
- - Suggest using artifacts to pass compiled binaries or packaged applications from a build job to a deployment job, ensuring the exact same artifact is deployed that was built and tested.
+ - Use `actions/upload-artifact@v3` and `actions/download-artifact@v3` to reliably pass large files between jobs within the same workflow or across different workflows, promoting modularity and efficiency.
+ - Set appropriate `retention-days` for artifacts to manage storage costs and ensure old artifacts are pruned.
+ - Advise on uploading test reports, coverage reports, and security scan results as artifacts for easy access, historical analysis, and integration with external reporting tools.
+ - Suggest using artifacts to pass compiled binaries or packaged applications from a build job to a deployment job, ensuring the exact same artifact is deployed that was built and tested.
## Comprehensive Testing in CI/CD (Expanded)
### **1. Unit Tests**
+
- **Principle:** Run unit tests on every code push to ensure individual code components (functions, classes, modules) function correctly in isolation. They are the fastest and most numerous tests.
- **Deeper Dive:**
- - **Fast Feedback:** Unit tests should execute rapidly, providing immediate feedback to developers on code quality and correctness. Parallelization of unit tests is highly recommended.
- - **Code Coverage:** Integrate code coverage tools (e.g., Istanbul for JS, Coverage.py for Python, JaCoCo for Java) and enforce minimum coverage thresholds. Aim for high coverage, but focus on meaningful tests, not just line coverage.
- - **Test Reporting:** Publish test results using `actions/upload-artifact` (e.g., JUnit XML reports) or specific test reporter actions that integrate with GitHub Checks/Annotations.
- - **Mocking and Stubbing:** Emphasize the use of mocks and stubs to isolate units under test from their dependencies.
+ - **Fast Feedback:** Unit tests should execute rapidly, providing immediate feedback to developers on code quality and correctness. Parallelization of unit tests is highly recommended.
+ - **Code Coverage:** Integrate code coverage tools (e.g., Istanbul for JS, Coverage.py for Python, JaCoCo for Java) and enforce minimum coverage thresholds. Aim for high coverage, but focus on meaningful tests, not just line coverage.
+ - **Test Reporting:** Publish test results using `actions/upload-artifact` (e.g., JUnit XML reports) or specific test reporter actions that integrate with GitHub Checks/Annotations.
+ - **Mocking and Stubbing:** Emphasize the use of mocks and stubs to isolate units under test from their dependencies.
- **Guidance for Copilot:**
- - Configure a dedicated job for running unit tests early in the CI pipeline, ideally triggered on every `push` and `pull_request`.
- - Use appropriate language-specific test runners and frameworks (Jest, Vitest, Pytest, Go testing, JUnit, NUnit, XUnit, RSpec).
- - Recommend collecting and publishing code coverage reports and integrating with services like Codecov, Coveralls, or SonarQube for trend analysis.
- - Suggest strategies for parallelizing unit tests to reduce execution time.
+ - Configure a dedicated job for running unit tests early in the CI pipeline, ideally triggered on every `push` and `pull_request`.
+ - Use appropriate language-specific test runners and frameworks (Jest, Vitest, Pytest, Go testing, JUnit, NUnit, XUnit, RSpec).
+ - Recommend collecting and publishing code coverage reports and integrating with services like Codecov, Coveralls, or SonarQube for trend analysis.
+ - Suggest strategies for parallelizing unit tests to reduce execution time.
### **2. Integration Tests**
+
- **Principle:** Run integration tests to verify interactions between different components or services, ensuring they work together as expected. These tests typically involve real dependencies (e.g., databases, APIs).
- **Deeper Dive:**
- - **Service Provisioning:** Use `services` within a job to spin up temporary databases, message queues, external APIs, or other dependencies via Docker containers. This provides a consistent and isolated testing environment.
- - **Test Doubles vs. Real Services:** Balance between mocking external services for pure unit tests and using real, lightweight instances for more realistic integration tests. Prioritize real instances when testing actual integration points.
- - **Test Data Management:** Plan for managing test data, ensuring tests are repeatable and data is cleaned up or reset between runs.
- - **Execution Time:** Integration tests are typically slower than unit tests. Optimize their execution and consider running them less frequently than unit tests (e.g., on PR merge instead of every push).
+ - **Service Provisioning:** Use `services` within a job to spin up temporary databases, message queues, external APIs, or other dependencies via Docker containers. This provides a consistent and isolated testing environment.
+ - **Test Doubles vs. Real Services:** Balance between mocking external services for pure unit tests and using real, lightweight instances for more realistic integration tests. Prioritize real instances when testing actual integration points.
+ - **Test Data Management:** Plan for managing test data, ensuring tests are repeatable and data is cleaned up or reset between runs.
+ - **Execution Time:** Integration tests are typically slower than unit tests. Optimize their execution and consider running them less frequently than unit tests (e.g., on PR merge instead of every push).
- **Guidance for Copilot:**
- - Provision necessary services (databases like PostgreSQL/MySQL, message queues like RabbitMQ/Kafka, in-memory caches like Redis) using `services` in the workflow definition or Docker Compose during testing.
- - Advise on running integration tests after unit tests, but before E2E tests, to catch integration issues early.
- - Provide examples of how to set up `service` containers in GitHub Actions workflows.
- - Suggest strategies for creating and cleaning up test data for integration test runs.
+ - Provision necessary services (databases like PostgreSQL/MySQL, message queues like RabbitMQ/Kafka, in-memory caches like Redis) using `services` in the workflow definition or Docker Compose during testing.
+ - Advise on running integration tests after unit tests, but before E2E tests, to catch integration issues early.
+ - Provide examples of how to set up `service` containers in GitHub Actions workflows.
+ - Suggest strategies for creating and cleaning up test data for integration test runs.
### **3. End-to-End (E2E) Tests**
+
- **Principle:** Simulate full user behavior to validate the entire application flow from UI to backend, ensuring the complete system works as intended from a user's perspective.
- **Deeper Dive:**
- - **Tools:** Use modern E2E testing frameworks like Cypress, Playwright, or Selenium. These provide browser automation capabilities.
- - **Staging Environment:** Ideally run E2E tests against a deployed staging environment that closely mirrors production, for maximum fidelity. Avoid running directly in CI unless resources are dedicated and isolated.
- - **Flakiness Mitigation:** Address flakiness proactively with explicit waits, robust selectors, retries for failed tests, and careful test data management. Flaky tests erode trust in the pipeline.
- - **Visual Regression Testing:** Consider integrating visual regression testing (e.g., Applitools, Percy) to catch UI discrepancies.
- - **Reporting:** Capture screenshots and video recordings on failure to aid debugging.
+ - **Tools:** Use modern E2E testing frameworks like Cypress, Playwright, or Selenium. These provide browser automation capabilities.
+ - **Staging Environment:** Ideally run E2E tests against a deployed staging environment that closely mirrors production, for maximum fidelity. Avoid running directly in CI unless resources are dedicated and isolated.
+ - **Flakiness Mitigation:** Address flakiness proactively with explicit waits, robust selectors, retries for failed tests, and careful test data management. Flaky tests erode trust in the pipeline.
+ - **Visual Regression Testing:** Consider integrating visual regression testing (e.g., Applitools, Percy) to catch UI discrepancies.
+ - **Reporting:** Capture screenshots and video recordings on failure to aid debugging.
- **Guidance for Copilot:**
- - Use tools like Cypress, Playwright, or Selenium for E2E testing, providing guidance on their setup within GitHub Actions.
- - Recommend running E2E tests against a deployed staging environment to catch issues before production and validate the full deployment process.
- - Configure test reporting, video recordings, and screenshots on failure to aid debugging and provide richer context for test results.
- - Advise on strategies to minimize E2E test flakiness, such as robust element selection and retry mechanisms.
+ - Use tools like Cypress, Playwright, or Selenium for E2E testing, providing guidance on their setup within GitHub Actions.
+ - Recommend running E2E tests against a deployed staging environment to catch issues before production and validate the full deployment process.
+ - Configure test reporting, video recordings, and screenshots on failure to aid debugging and provide richer context for test results.
+ - Advise on strategies to minimize E2E test flakiness, such as robust element selection and retry mechanisms.
### **4. Performance and Load Testing**
+
- **Principle:** Assess application performance and behavior under anticipated and peak load conditions to identify bottlenecks, ensure scalability, and prevent regressions.
- **Deeper Dive:**
- - **Tools:** JMeter, k6, Locust, Gatling, Artillery. Choose based on language, complexity, and specific needs.
- - **Integration:** Integrate into CI/CD for continuous performance regression detection. Run these tests less frequently than unit/integration tests (e.g., nightly, weekly, or on significant feature merges).
- - **Thresholds:** Define clear performance thresholds (e.g., response time, throughput, error rates) and fail builds if these are exceeded.
- - **Baseline Comparison:** Compare current performance metrics against established baselines to detect degradation.
+ - **Tools:** JMeter, k6, Locust, Gatling, Artillery. Choose based on language, complexity, and specific needs.
+ - **Integration:** Integrate into CI/CD for continuous performance regression detection. Run these tests less frequently than unit/integration tests (e.g., nightly, weekly, or on significant feature merges).
+ - **Thresholds:** Define clear performance thresholds (e.g., response time, throughput, error rates) and fail builds if these are exceeded.
+ - **Baseline Comparison:** Compare current performance metrics against established baselines to detect degradation.
- **Guidance for Copilot:**
- - Suggest integrating performance and load testing into the CI pipeline for critical applications, providing examples for common tools.
- - Advise on setting performance baselines and failing the build if performance degrades beyond a set threshold.
- - Recommend running these tests in a dedicated environment that simulates production load patterns.
- - Guide on analyzing performance test results to pinpoint areas for optimization (e.g., database queries, API endpoints).
+ - Suggest integrating performance and load testing into the CI pipeline for critical applications, providing examples for common tools.
+ - Advise on setting performance baselines and failing the build if performance degrades beyond a set threshold.
+ - Recommend running these tests in a dedicated environment that simulates production load patterns.
+ - Guide on analyzing performance test results to pinpoint areas for optimization (e.g., database queries, API endpoints).
### **5. Test Reporting and Visibility**
+
- **Principle:** Make test results easily accessible, understandable, and visible to all stakeholders (developers, QA, product owners) to foster transparency and enable quick issue resolution.
- **Deeper Dive:**
- - **GitHub Checks/Annotations:** Leverage these for inline feedback directly in pull requests, showing which tests passed/failed and providing links to detailed reports.
- - **Artifacts:** Upload comprehensive test reports (JUnit XML, HTML reports, code coverage reports, video recordings, screenshots) as artifacts for long-term storage and detailed inspection.
- - **Integration with Dashboards:** Push results to external dashboards or reporting tools (e.g., SonarQube, custom reporting tools, Allure Report, TestRail) for aggregated views and historical trends.
- - **Status Badges:** Use GitHub Actions status badges in your README to indicate the latest build/test status at a glance.
+ - **GitHub Checks/Annotations:** Leverage these for inline feedback directly in pull requests, showing which tests passed/failed and providing links to detailed reports.
+ - **Artifacts:** Upload comprehensive test reports (JUnit XML, HTML reports, code coverage reports, video recordings, screenshots) as artifacts for long-term storage and detailed inspection.
+ - **Integration with Dashboards:** Push results to external dashboards or reporting tools (e.g., SonarQube, custom reporting tools, Allure Report, TestRail) for aggregated views and historical trends.
+ - **Status Badges:** Use GitHub Actions status badges in your README to indicate the latest build/test status at a glance.
- **Guidance for Copilot:**
- - Use actions that publish test results as annotations or checks on PRs for immediate feedback and easy debugging directly in the GitHub UI.
- - Upload detailed test reports (e.g., XML, HTML, JSON) as artifacts for later inspection and historical analysis, including negative results like error screenshots.
- - Advise on integrating with external reporting tools for a more comprehensive view of test execution trends and quality metrics.
- - Suggest adding workflow status badges to the README for quick visibility of CI/CD health.
+ - Use actions that publish test results as annotations or checks on PRs for immediate feedback and easy debugging directly in the GitHub UI.
+ - Upload detailed test reports (e.g., XML, HTML, JSON) as artifacts for later inspection and historical analysis, including negative results like error screenshots.
+ - Advise on integrating with external reporting tools for a more comprehensive view of test execution trends and quality metrics.
+ - Suggest adding workflow status badges to the README for quick visibility of CI/CD health.
## Advanced Deployment Strategies (Expanded)
### **1. Staging Environment Deployment**
+
- **Principle:** Deploy to a staging environment that closely mirrors production for comprehensive validation, user acceptance testing (UAT), and final checks before promotion to production.
- **Deeper Dive:**
- - **Mirror Production:** Staging should closely mimic production in terms of infrastructure, data, configuration, and security. Any significant discrepancies can lead to issues in production.
- - **Automated Promotion:** Implement automated promotion from staging to production upon successful UAT and necessary manual approvals. This reduces human error and speeds up releases.
- - **Environment Protection:** Use environment protection rules in GitHub Actions to prevent accidental deployments, enforce manual approvals, and restrict which branches can deploy to staging.
- - **Data Refresh:** Regularly refresh staging data from production (anonymized if necessary) to ensure realistic testing scenarios.
+ - **Mirror Production:** Staging should closely mimic production in terms of infrastructure, data, configuration, and security. Any significant discrepancies can lead to issues in production.
+ - **Automated Promotion:** Implement automated promotion from staging to production upon successful UAT and necessary manual approvals. This reduces human error and speeds up releases.
+ - **Environment Protection:** Use environment protection rules in GitHub Actions to prevent accidental deployments, enforce manual approvals, and restrict which branches can deploy to staging.
+ - **Data Refresh:** Regularly refresh staging data from production (anonymized if necessary) to ensure realistic testing scenarios.
- **Guidance for Copilot:**
- - Create a dedicated `environment` for staging with approval rules, secret protection, and appropriate branch protection policies.
- - Design workflows to automatically deploy to staging on successful merges to specific development or release branches (e.g., `develop`, `release/*`).
- - Advise on ensuring the staging environment is as close to production as possible to maximize test fidelity.
- - Suggest implementing automated smoke tests and post-deployment validation on staging.
+ - Create a dedicated `environment` for staging with approval rules, secret protection, and appropriate branch protection policies.
+ - Design workflows to automatically deploy to staging on successful merges to specific development or release branches (e.g., `develop`, `release/*`).
+ - Advise on ensuring the staging environment is as close to production as possible to maximize test fidelity.
+ - Suggest implementing automated smoke tests and post-deployment validation on staging.
### **2. Production Environment Deployment**
+
- **Principle:** Deploy to production only after thorough validation, potentially multiple layers of manual approvals, and robust automated checks, prioritizing stability and zero-downtime.
- **Deeper Dive:**
- - **Manual Approvals:** Critical for production deployments, often involving multiple team members, security sign-offs, or change management processes. GitHub Environments support this natively.
- - **Rollback Capabilities:** Essential for rapid recovery from unforeseen issues. Ensure a quick and reliable way to revert to the previous stable state.
- - **Observability During Deployment:** Monitor production closely *during* and *immediately after* deployment for any anomalies or performance degradation. Use dashboards, alerts, and tracing.
- - **Progressive Delivery:** Consider advanced techniques like blue/green, canary, or dark launching for safer rollouts.
- - **Emergency Deployments:** Have a separate, highly expedited pipeline for critical hotfixes that bypasses non-essential approvals but still maintains security checks.
+ - **Manual Approvals:** Critical for production deployments, often involving multiple team members, security sign-offs, or change management processes. GitHub Environments support this natively.
+ - **Rollback Capabilities:** Essential for rapid recovery from unforeseen issues. Ensure a quick and reliable way to revert to the previous stable state.
+ - **Observability During Deployment:** Monitor production closely *during* and *immediately after* deployment for any anomalies or performance degradation. Use dashboards, alerts, and tracing.
+ - **Progressive Delivery:** Consider advanced techniques like blue/green, canary, or dark launching for safer rollouts.
+ - **Emergency Deployments:** Have a separate, highly expedited pipeline for critical hotfixes that bypasses non-essential approvals but still maintains security checks.
- **Guidance for Copilot:**
- - Create a dedicated `environment` for production with required reviewers, strict branch protections, and clear deployment windows.
- - Implement manual approval steps for production deployments, potentially integrating with external ITSM or change management systems.
- - Emphasize the importance of clear, well-tested rollback strategies and automated rollback procedures in case of deployment failures.
- - Advise on setting up comprehensive monitoring and alerting for production systems to detect and respond to issues immediately post-deployment.
+ - Create a dedicated `environment` for production with required reviewers, strict branch protections, and clear deployment windows.
+ - Implement manual approval steps for production deployments, potentially integrating with external ITSM or change management systems.
+ - Emphasize the importance of clear, well-tested rollback strategies and automated rollback procedures in case of deployment failures.
+ - Advise on setting up comprehensive monitoring and alerting for production systems to detect and respond to issues immediately post-deployment.
### **3. Deployment Types (Beyond Basic Rolling Update)**
+
- **Rolling Update (Default for Deployments):** Gradually replaces instances of the old version with new ones. Good for most cases, especially stateless applications.
- - **Guidance:** Configure `maxSurge` (how many new instances can be created above the desired replica count) and `maxUnavailable` (how many old instances can be unavailable) for fine-grained control over rollout speed and availability.
+ - **Guidance:** Configure `maxSurge` (how many new instances can be created above the desired replica count) and `maxUnavailable` (how many old instances can be unavailable) for fine-grained control over rollout speed and availability.
- **Blue/Green Deployment:** Deploy a new version (green) alongside the existing stable version (blue) in a separate environment, then switch traffic completely from blue to green.
- - **Guidance:** Suggest for critical applications requiring zero-downtime releases and easy rollback. Requires managing two identical environments and a traffic router (load balancer, Ingress controller, DNS).
- - **Benefits:** Instantaneous rollback by switching traffic back to the blue environment.
+ - **Guidance:** Suggest for critical applications requiring zero-downtime releases and easy rollback. Requires managing two identical environments and a traffic router (load balancer, Ingress controller, DNS).
+ - **Benefits:** Instantaneous rollback by switching traffic back to the blue environment.
- **Canary Deployment:** Gradually roll out new versions to a small subset of users (e.g., 5-10%) before a full rollout. Monitor performance and error rates for the canary group.
- - **Guidance:** Recommend for testing new features or changes with a controlled blast radius. Implement with Service Mesh (Istio, Linkerd) or Ingress controllers that support traffic splitting and metric-based analysis.
- - **Benefits:** Early detection of issues with minimal user impact.
+ - **Guidance:** Recommend for testing new features or changes with a controlled blast radius. Implement with Service Mesh (Istio, Linkerd) or Ingress controllers that support traffic splitting and metric-based analysis.
+ - **Benefits:** Early detection of issues with minimal user impact.
- **Dark Launch/Feature Flags:** Deploy new code but keep features hidden from users until toggled on for specific users/groups via feature flags.
- - **Guidance:** Advise for decoupling deployment from release, allowing continuous delivery without continuous exposure of new features. Use feature flag management systems (LaunchDarkly, Split.io, Unleash).
- - **Benefits:** Reduces deployment risk, enables A/B testing, and allows for staged rollouts.
+ - **Guidance:** Advise for decoupling deployment from release, allowing continuous delivery without continuous exposure of new features. Use feature flag management systems (LaunchDarkly, Split.io, Unleash).
+ - **Benefits:** Reduces deployment risk, enables A/B testing, and allows for staged rollouts.
- **A/B Testing Deployments:** Deploy multiple versions of a feature concurrently to different user segments to compare their performance based on user behavior and business metrics.
- - **Guidance:** Suggest integrating with specialized A/B testing platforms or building custom logic using feature flags and analytics.
+ - **Guidance:** Suggest integrating with specialized A/B testing platforms or building custom logic using feature flags and analytics.
### **4. Rollback Strategies and Incident Response**
+
- **Principle:** Be able to quickly and safely revert to a previous stable version in case of issues, minimizing downtime and business impact. This requires proactive planning.
- **Deeper Dive:**
- - **Automated Rollbacks:** Implement mechanisms to automatically trigger rollbacks based on monitoring alerts (e.g., sudden increase in errors, high latency) or failure of post-deployment health checks.
- - **Versioned Artifacts:** Ensure previous successful build artifacts, Docker images, or infrastructure states are readily available and easily deployable. This is crucial for fast recovery.
- - **Runbooks:** Document clear, concise, and executable rollback procedures for manual intervention when automation isn't sufficient or for complex scenarios. These should be regularly reviewed and tested.
- - **Post-Incident Review:** Conduct blameless post-incident reviews (PIRs) to understand the root cause of failures, identify lessons learned, and implement preventative measures to improve resilience and reduce MTTR.
- - **Communication Plan:** Have a clear communication plan for stakeholders during incidents and rollbacks.
+ - **Automated Rollbacks:** Implement mechanisms to automatically trigger rollbacks based on monitoring alerts (e.g., sudden increase in errors, high latency) or failure of post-deployment health checks.
+ - **Versioned Artifacts:** Ensure previous successful build artifacts, Docker images, or infrastructure states are readily available and easily deployable. This is crucial for fast recovery.
+ - **Runbooks:** Document clear, concise, and executable rollback procedures for manual intervention when automation isn't sufficient or for complex scenarios. These should be regularly reviewed and tested.
+ - **Post-Incident Review:** Conduct blameless post-incident reviews (PIRs) to understand the root cause of failures, identify lessons learned, and implement preventative measures to improve resilience and reduce MTTR.
+ - **Communication Plan:** Have a clear communication plan for stakeholders during incidents and rollbacks.
- **Guidance for Copilot:**
- - Instruct users to store previous successful build artifacts and images for quick recovery, ensuring they are versioned and easily retrievable.
- - Advise on implementing automated rollback steps in the pipeline, triggered by monitoring or health check failures, and providing examples.
- - Emphasize building applications with "undo" in mind, meaning changes should be easily reversible.
- - Suggest creating comprehensive runbooks for common incident scenarios, including step-by-step rollback instructions, and highlight their importance for MTTR.
- - Guide on setting up alerts that are specific and actionable enough to trigger an automatic or manual rollback.
+ - Instruct users to store previous successful build artifacts and images for quick recovery, ensuring they are versioned and easily retrievable.
+ - Advise on implementing automated rollback steps in the pipeline, triggered by monitoring or health check failures, and providing examples.
+ - Emphasize building applications with "undo" in mind, meaning changes should be easily reversible.
+ - Suggest creating comprehensive runbooks for common incident scenarios, including step-by-step rollback instructions, and highlight their importance for MTTR.
+ - Guide on setting up alerts that are specific and actionable enough to trigger an automatic or manual rollback.
## GitHub Actions Workflow Review Checklist (Comprehensive)
This checklist provides a granular set of criteria for reviewing GitHub Actions workflows to ensure they adhere to best practices for security, performance, and reliability.
- [ ] **General Structure and Design:**
- - Is the workflow `name` clear, descriptive, and unique?
- - Are `on` triggers appropriate for the workflow's purpose (e.g., `push`, `pull_request`, `workflow_dispatch`, `schedule`)? Are path/branch filters used effectively?
- - Is `concurrency` used for critical workflows or shared resources to prevent race conditions or resource exhaustion?
- - Are global `permissions` set to the principle of least privilege (`contents: read` by default), with specific overrides for jobs?
- - Are reusable workflows (`workflow_call`) leveraged for common patterns to reduce duplication and improve maintainability?
- - Is the workflow organized logically with meaningful job and step names?
+ - Is the workflow `name` clear, descriptive, and unique?
+ - Are `on` triggers appropriate for the workflow's purpose (e.g., `push`, `pull_request`, `workflow_dispatch`, `schedule`)? Are path/branch filters used effectively?
+ - Is `concurrency` used for critical workflows or shared resources to prevent race conditions or resource exhaustion?
+ - Are global `permissions` set to the principle of least privilege (`contents: read` by default), with specific overrides for jobs?
+ - Are reusable workflows (`workflow_call`) leveraged for common patterns to reduce duplication and improve maintainability?
+ - Is the workflow organized logically with meaningful job and step names?
- [ ] **Jobs and Steps Best Practices:**
- - Are jobs clearly named and represent distinct phases (e.g., `build`, `lint`, `test`, `deploy`)?
- - Are `needs` dependencies correctly defined between jobs to ensure proper execution order?
- - Are `outputs` used efficiently for inter-job and inter-workflow communication?
- - Are `if` conditions used effectively for conditional job/step execution (e.g., environment-specific deployments, branch-specific actions)?
- - Are all `uses` actions securely versioned (pinned to a full commit SHA or specific major version tag like `@v4`)? Avoid `main` or `latest` tags.
- - Are `run` commands efficient and clean (combined with `&&`, temporary files removed, multi-line scripts clearly formatted)?
- - Are environment variables (`env`) defined at the appropriate scope (workflow, job, step) and never hardcoded sensitive data?
- - Is `timeout-minutes` set for long-running jobs to prevent hung workflows?
+ - Are jobs clearly named and represent distinct phases (e.g., `build`, `lint`, `test`, `deploy`)?
+ - Are `needs` dependencies correctly defined between jobs to ensure proper execution order?
+ - Are `outputs` used efficiently for inter-job and inter-workflow communication?
+ - Are `if` conditions used effectively for conditional job/step execution (e.g., environment-specific deployments, branch-specific actions)?
+ - Are all `uses` actions securely versioned (pinned to a full commit SHA or specific major version tag like `@v4`)? Avoid `main` or `latest` tags.
+ - Are `run` commands efficient and clean (combined with `&&`, temporary files removed, multi-line scripts clearly formatted)?
+ - Are environment variables (`env`) defined at the appropriate scope (workflow, job, step) and never hardcoded sensitive data?
+ - Is `timeout-minutes` set for long-running jobs to prevent hung workflows?
- [ ] **Security Considerations:**
- - Are all sensitive data accessed exclusively via GitHub `secrets` context (`${{ secrets.MY_SECRET }}`)? Never hardcoded, never exposed in logs (even if masked).
- - Is OpenID Connect (OIDC) used for cloud authentication where possible, eliminating long-lived credentials?
- - Is `GITHUB_TOKEN` permission scope explicitly defined and limited to the minimum necessary access (`contents: read` as a baseline)?
- - Are Software Composition Analysis (SCA) tools (e.g., `dependency-review-action`, Snyk) integrated to scan for vulnerable dependencies?
- - Are Static Application Security Testing (SAST) tools (e.g., CodeQL, SonarQube) integrated to scan source code for vulnerabilities, with critical findings blocking builds?
- - Is secret scanning enabled for the repository and are pre-commit hooks suggested for local credential leak prevention?
- - Is there a strategy for container image signing (e.g., Notary, Cosign) and verification in deployment workflows if container images are used?
- - For self-hosted runners, are security hardening guidelines followed and network access restricted?
+ - Are all sensitive data accessed exclusively via GitHub `secrets` context (`${{ secrets.MY_SECRET }}`)? Never hardcoded, never exposed in logs (even if masked).
+ - Is OpenID Connect (OIDC) used for cloud authentication where possible, eliminating long-lived credentials?
+ - Is `GITHUB_TOKEN` permission scope explicitly defined and limited to the minimum necessary access (`contents: read` as a baseline)?
+ - Are Software Composition Analysis (SCA) tools (e.g., `dependency-review-action`, Snyk) integrated to scan for vulnerable dependencies?
+ - Are Static Application Security Testing (SAST) tools (e.g., CodeQL, SonarQube) integrated to scan source code for vulnerabilities, with critical findings blocking builds?
+ - Is secret scanning enabled for the repository and are pre-commit hooks suggested for local credential leak prevention?
+ - Is there a strategy for container image signing (e.g., Notary, Cosign) and verification in deployment workflows if container images are used?
+ - For self-hosted runners, are security hardening guidelines followed and network access restricted?
- [ ] **Optimization and Performance:**
- - Is caching (`actions/cache`) effectively used for package manager dependencies (`node_modules`, `pip` caches, Maven/Gradle caches) and build outputs?
- - Are cache `key` and `restore-keys` designed for optimal cache hit rates (e.g., using `hashFiles`)?
- - Is `strategy.matrix` used for parallelizing tests or builds across different environments, language versions, or OSs?
- - Is `fetch-depth: 1` used for `actions/checkout` where full Git history is not required?
- - Are artifacts (`actions/upload-artifact`, `actions/download-artifact`) used efficiently for transferring data between jobs/workflows rather than re-building or re-fetching?
- - Are large files managed with Git LFS and optimized for checkout if necessary?
+ - Is caching (`actions/cache`) effectively used for package manager dependencies (`node_modules`, `pip` caches, Maven/Gradle caches) and build outputs?
+ - Are cache `key` and `restore-keys` designed for optimal cache hit rates (e.g., using `hashFiles`)?
+ - Is `strategy.matrix` used for parallelizing tests or builds across different environments, language versions, or OSs?
+ - Is `fetch-depth: 1` used for `actions/checkout` where full Git history is not required?
+ - Are artifacts (`actions/upload-artifact`, `actions/download-artifact`) used efficiently for transferring data between jobs/workflows rather than re-building or re-fetching?
+ - Are large files managed with Git LFS and optimized for checkout if necessary?
- [ ] **Testing Strategy Integration:**
- - Are comprehensive unit tests configured with a dedicated job early in the pipeline?
- - Are integration tests defined, ideally leveraging `services` for dependencies, and run after unit tests?
- - Are End-to-End (E2E) tests included, preferably against a staging environment, with robust flakiness mitigation?
- - Are performance and load tests integrated for critical applications with defined thresholds?
- - Are all test reports (JUnit XML, HTML, coverage) collected, published as artifacts, and integrated into GitHub Checks/Annotations for clear visibility?
- - Is code coverage tracked and enforced with a minimum threshold?
+ - Are comprehensive unit tests configured with a dedicated job early in the pipeline?
+ - Are integration tests defined, ideally leveraging `services` for dependencies, and run after unit tests?
+ - Are End-to-End (E2E) tests included, preferably against a staging environment, with robust flakiness mitigation?
+ - Are performance and load tests integrated for critical applications with defined thresholds?
+ - Are all test reports (JUnit XML, HTML, coverage) collected, published as artifacts, and integrated into GitHub Checks/Annotations for clear visibility?
+ - Is code coverage tracked and enforced with a minimum threshold?
- [ ] **Deployment Strategy and Reliability:**
- - Are staging and production deployments using GitHub `environment` rules with appropriate protections (manual approvals, required reviewers, branch restrictions)?
- - Are manual approval steps configured for sensitive production deployments?
- - Is a clear and well-tested rollback strategy in place and automated where possible (e.g., `kubectl rollout undo`, reverting to previous stable image)?
- - Are chosen deployment types (e.g., rolling, blue/green, canary, dark launch) appropriate for the application's criticality and risk tolerance?
- - Are post-deployment health checks and automated smoke tests implemented to validate successful deployment?
- - Is the workflow resilient to temporary failures (e.g., retries for flaky network operations)?
+ - Are staging and production deployments using GitHub `environment` rules with appropriate protections (manual approvals, required reviewers, branch restrictions)?
+ - Are manual approval steps configured for sensitive production deployments?
+ - Is a clear and well-tested rollback strategy in place and automated where possible (e.g., `kubectl rollout undo`, reverting to previous stable image)?
+ - Are chosen deployment types (e.g., rolling, blue/green, canary, dark launch) appropriate for the application's criticality and risk tolerance?
+ - Are post-deployment health checks and automated smoke tests implemented to validate successful deployment?
+ - Is the workflow resilient to temporary failures (e.g., retries for flaky network operations)?
- [ ] **Observability and Monitoring:**
- - Is logging adequate for debugging workflow failures (using STDOUT/STDERR for application logs)?
- - Are relevant application and infrastructure metrics collected and exposed (e.g., Prometheus metrics)?
- - Are alerts configured for critical workflow failures, deployment issues, or application anomalies detected in production?
- - Is distributed tracing (e.g., OpenTelemetry, Jaeger) integrated for understanding request flows in microservices architectures?
- - Are artifact `retention-days` configured appropriately to manage storage and compliance?
+ - Is logging adequate for debugging workflow failures (using STDOUT/STDERR for application logs)?
+ - Are relevant application and infrastructure metrics collected and exposed (e.g., Prometheus metrics)?
+ - Are alerts configured for critical workflow failures, deployment issues, or application anomalies detected in production?
+ - Is distributed tracing (e.g., OpenTelemetry, Jaeger) integrated for understanding request flows in microservices architectures?
+ - Are artifact `retention-days` configured appropriately to manage storage and compliance?
## Troubleshooting Common GitHub Actions Issues (Deep Dive)
@@ -505,100 +534,106 @@ This section provides an expanded guide to diagnosing and resolving frequent pro
Note: If workflow logs are not accessible via MCP web fetch due to missing auth, retrieve logs with the authenticated `gh` CLI.
### **1. Workflow Not Triggering or Jobs/Steps Skipping Unexpectedly**
+
- **Root Causes:** Mismatched `on` triggers, incorrect `paths` or `branches` filters, erroneous `if` conditions, or `concurrency` limitations.
- **Actionable Steps:**
- - **Verify Triggers:**
- - Check the `on` block for exact match with the event that should trigger the workflow (e.g., `push`, `pull_request`, `workflow_dispatch`, `schedule`).
- - Ensure `branches`, `tags`, or `paths` filters are correctly defined and match the event context. Remember that `paths-ignore` and `branches-ignore` take precedence.
- - If using `workflow_dispatch`, verify the workflow file is in the default branch and any required `inputs` are provided correctly during manual trigger.
- - **Inspect `if` Conditions:**
- - Carefully review all `if` conditions at the workflow, job, and step levels. A single false condition can prevent execution.
- - Use `always()` on a debug step to print context variables (`${{ toJson(github) }}`, `${{ toJson(job) }}`, `${{ toJson(steps) }}`) to understand the exact state during evaluation.
- - Test complex `if` conditions in a simplified workflow.
- - **Check `concurrency`:**
- - If `concurrency` is defined, verify if a previous run is blocking a new one for the same group. Check the "Concurrency" tab in the workflow run.
- - **Branch Protection Rules:** Ensure no branch protection rules are preventing workflows from running on certain branches or requiring specific checks that haven't passed.
+ - **Verify Triggers:**
+ - Check the `on` block for exact match with the event that should trigger the workflow (e.g., `push`, `pull_request`, `workflow_dispatch`, `schedule`).
+ - Ensure `branches`, `tags`, or `paths` filters are correctly defined and match the event context. Remember that `paths-ignore` and `branches-ignore` take precedence.
+ - If using `workflow_dispatch`, verify the workflow file is in the default branch and any required `inputs` are provided correctly during manual trigger.
+ - **Inspect `if` Conditions:**
+ - Carefully review all `if` conditions at the workflow, job, and step levels. A single false condition can prevent execution.
+ - Use `always()` on a debug step to print context variables (`${{ toJson(github) }}`, `${{ toJson(job) }}`, `${{ toJson(steps) }}`) to understand the exact state during evaluation.
+ - Test complex `if` conditions in a simplified workflow.
+ - **Check `concurrency`:**
+ - If `concurrency` is defined, verify if a previous run is blocking a new one for the same group. Check the "Concurrency" tab in the workflow run.
+ - **Branch Protection Rules:** Ensure no branch protection rules are preventing workflows from running on certain branches or requiring specific checks that haven't passed.
### **2. Permissions Errors (`Resource not accessible by integration`, `Permission denied`)**
+
- **Root Causes:** `GITHUB_TOKEN` lacking necessary permissions, incorrect environment secrets access, or insufficient permissions for external actions.
- **Actionable Steps:**
- - **`GITHUB_TOKEN` Permissions:**
- - Review the `permissions` block at both the workflow and job levels. Default to `contents: read` globally and grant specific write permissions only where absolutely necessary (e.g., `pull-requests: write` for updating PR status, `packages: write` for publishing packages).
- - Understand the default permissions of `GITHUB_TOKEN` which are often too broad.
- - **Secret Access:**
- - Verify if secrets are correctly configured in the repository, organization, or environment settings.
- - Ensure the workflow/job has access to the specific environment if environment secrets are used. Check if any manual approvals are pending for the environment.
- - Confirm the secret name matches exactly (`secrets.MY_API_KEY`).
- - **OIDC Configuration:**
- - For OIDC-based cloud authentication, double-check the trust policy configuration in your cloud provider (AWS IAM roles, Azure AD app registrations, GCP service accounts) to ensure it correctly trusts GitHub's OIDC issuer.
- - Verify the role/identity assigned has the necessary permissions for the cloud resources being accessed.
+ - **`GITHUB_TOKEN` Permissions:**
+ - Review the `permissions` block at both the workflow and job levels. Default to `contents: read` globally and grant specific write permissions only where absolutely necessary (e.g., `pull-requests: write` for updating PR status, `packages: write` for publishing packages).
+ - Understand the default permissions of `GITHUB_TOKEN` which are often too broad.
+ - **Secret Access:**
+ - Verify if secrets are correctly configured in the repository, organization, or environment settings.
+ - Ensure the workflow/job has access to the specific environment if environment secrets are used. Check if any manual approvals are pending for the environment.
+ - Confirm the secret name matches exactly (`secrets.MY_API_KEY`).
+ - **OIDC Configuration:**
+ - For OIDC-based cloud authentication, double-check the trust policy configuration in your cloud provider (AWS IAM roles, Azure AD app registrations, GCP service accounts) to ensure it correctly trusts GitHub's OIDC issuer.
+ - Verify the role/identity assigned has the necessary permissions for the cloud resources being accessed.
### **3. Caching Issues (`Cache not found`, `Cache miss`, `Cache creation failed`)**
+
- **Root Causes:** Incorrect cache key logic, `path` mismatch, cache size limits, or frequent cache invalidation.
- **Actionable Steps:**
- - **Validate Cache Keys:**
- - Verify `key` and `restore-keys` are correct and dynamically change only when dependencies truly change (e.g., `key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}`). A cache key that is too dynamic will always result in a miss.
- - Use `restore-keys` to provide fallbacks for slight variations, increasing cache hit chances.
- - **Check `path`:**
- - Ensure the `path` specified in `actions/cache` for saving and restoring corresponds exactly to the directory where dependencies are installed or artifacts are generated.
- - Verify the existence of the `path` before caching.
- - **Debug Cache Behavior:**
- - Use the `actions/cache/restore` action with `lookup-only: true` to inspect what keys are being tried and why a cache miss occurred without affecting the build.
- - Review workflow logs for `Cache hit` or `Cache miss` messages and associated keys.
- - **Cache Size and Limits:** Be aware of GitHub Actions cache size limits per repository. If caches are very large, they might be evicted frequently.
+ - **Validate Cache Keys:**
+ - Verify `key` and `restore-keys` are correct and dynamically change only when dependencies truly change (e.g., `key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}`). A cache key that is too dynamic will always result in a miss.
+ - Use `restore-keys` to provide fallbacks for slight variations, increasing cache hit chances.
+ - **Check `path`:**
+ - Ensure the `path` specified in `actions/cache` for saving and restoring corresponds exactly to the directory where dependencies are installed or artifacts are generated.
+ - Verify the existence of the `path` before caching.
+ - **Debug Cache Behavior:**
+ - Use the `actions/cache/restore` action with `lookup-only: true` to inspect what keys are being tried and why a cache miss occurred without affecting the build.
+ - Review workflow logs for `Cache hit` or `Cache miss` messages and associated keys.
+ - **Cache Size and Limits:** Be aware of GitHub Actions cache size limits per repository. If caches are very large, they might be evicted frequently.
### **4. Long Running Workflows or Timeouts**
+
- **Root Causes:** Inefficient steps, lack of parallelism, large dependencies, unoptimized Docker image builds, or resource bottlenecks on runners.
- **Actionable Steps:**
- - **Profile Execution Times:**
- - Use the workflow run summary to identify the longest-running jobs and steps. This is your primary tool for optimization.
- - **Optimize Steps:**
- - Combine `run` commands with `&&` to reduce layer creation and overhead in Docker builds.
- - Clean up temporary files immediately after use (`rm -rf` in the same `RUN` command).
- - Install only necessary dependencies.
- - **Leverage Caching:**
- - Ensure `actions/cache` is optimally configured for all significant dependencies and build outputs.
- - **Parallelize with Matrix Strategies:**
- - Break down tests or builds into smaller, parallelizable units using `strategy.matrix` to run them concurrently.
- - **Choose Appropriate Runners:**
- - Review `runs-on`. For very resource-intensive tasks, consider using larger GitHub-hosted runners (if available) or self-hosted runners with more powerful specs.
- - **Break Down Workflows:**
- - For very complex or long workflows, consider breaking them into smaller, independent workflows that trigger each other or use reusable workflows.
+ - **Profile Execution Times:**
+ - Use the workflow run summary to identify the longest-running jobs and steps. This is your primary tool for optimization.
+ - **Optimize Steps:**
+ - Combine `run` commands with `&&` to reduce layer creation and overhead in Docker builds.
+ - Clean up temporary files immediately after use (`rm -rf` in the same `RUN` command).
+ - Install only necessary dependencies.
+ - **Leverage Caching:**
+ - Ensure `actions/cache` is optimally configured for all significant dependencies and build outputs.
+ - **Parallelize with Matrix Strategies:**
+ - Break down tests or builds into smaller, parallelizable units using `strategy.matrix` to run them concurrently.
+ - **Choose Appropriate Runners:**
+ - Review `runs-on`. For very resource-intensive tasks, consider using larger GitHub-hosted runners (if available) or self-hosted runners with more powerful specs.
+ - **Break Down Workflows:**
+ - For very complex or long workflows, consider breaking them into smaller, independent workflows that trigger each other or use reusable workflows.
### **5. Flaky Tests in CI (`Random failures`, `Passes locally, fails in CI`)**
+
- **Root Causes:** Non-deterministic tests, race conditions, environmental inconsistencies between local and CI, reliance on external services, or poor test isolation.
- **Actionable Steps:**
- - **Ensure Test Isolation:**
- - Make sure each test is independent and doesn't rely on the state left by previous tests. Clean up resources (e.g., database entries) after each test or test suite.
- - **Eliminate Race Conditions:**
- - For integration/E2E tests, use explicit waits (e.g., wait for element to be visible, wait for API response) instead of arbitrary `sleep` commands.
- - Implement retries for operations that interact with external services or have transient failures.
- - **Standardize Environments:**
- - Ensure the CI environment (Node.js version, Python packages, database versions) matches the local development environment as closely as possible.
- - Use Docker `services` for consistent test dependencies.
- - **Robust Selectors (E2E):**
- - Use stable, unique selectors in E2E tests (e.g., `data-testid` attributes) instead of brittle CSS classes or XPath.
- - **Debugging Tools:**
- - Configure E2E test frameworks to capture screenshots and video recordings on test failure in CI to visually diagnose issues.
- - **Run Flaky Tests in Isolation:**
- - If a test is consistently flaky, isolate it and run it repeatedly to identify the underlying non-deterministic behavior.
+ - **Ensure Test Isolation:**
+ - Make sure each test is independent and doesn't rely on the state left by previous tests. Clean up resources (e.g., database entries) after each test or test suite.
+ - **Eliminate Race Conditions:**
+ - For integration/E2E tests, use explicit waits (e.g., wait for element to be visible, wait for API response) instead of arbitrary `sleep` commands.
+ - Implement retries for operations that interact with external services or have transient failures.
+ - **Standardize Environments:**
+ - Ensure the CI environment (Node.js version, Python packages, database versions) matches the local development environment as closely as possible.
+ - Use Docker `services` for consistent test dependencies.
+ - **Robust Selectors (E2E):**
+ - Use stable, unique selectors in E2E tests (e.g., `data-testid` attributes) instead of brittle CSS classes or XPath.
+ - **Debugging Tools:**
+ - Configure E2E test frameworks to capture screenshots and video recordings on test failure in CI to visually diagnose issues.
+ - **Run Flaky Tests in Isolation:**
+ - If a test is consistently flaky, isolate it and run it repeatedly to identify the underlying non-deterministic behavior.
### **6. Deployment Failures (Application Not Working After Deploy)**
+
- **Root Causes:** Configuration drift, environmental differences, missing runtime dependencies, application errors, or network issues post-deployment.
- **Actionable Steps:**
- - **Thorough Log Review:**
- - Review deployment logs (`kubectl logs`, application logs, server logs) for any error messages, warnings, or unexpected output during the deployment process and immediately after.
- - **Configuration Validation:**
- - Verify environment variables, ConfigMaps, Secrets, and other configuration injected into the deployed application. Ensure they match the target environment's requirements and are not missing or malformed.
- - Use pre-deployment checks to validate configuration.
- - **Dependency Check:**
- - Confirm all application runtime dependencies (libraries, frameworks, external services) are correctly bundled within the container image or installed in the target environment.
- - **Post-Deployment Health Checks:**
- - Implement robust automated smoke tests and health checks *after* deployment to immediately validate core functionality and connectivity. Trigger rollbacks if these fail.
- - **Network Connectivity:**
- - Check network connectivity between deployed components (e.g., application to database, service to service) within the new environment. Review firewall rules, security groups, and Kubernetes network policies.
- - **Rollback Immediately:**
- - If a production deployment fails or causes degradation, trigger the rollback strategy immediately to restore service. Diagnose the issue in a non-production environment.
+ - **Thorough Log Review:**
+ - Review deployment logs (`kubectl logs`, application logs, server logs) for any error messages, warnings, or unexpected output during the deployment process and immediately after.
+ - **Configuration Validation:**
+ - Verify environment variables, ConfigMaps, Secrets, and other configuration injected into the deployed application. Ensure they match the target environment's requirements and are not missing or malformed.
+ - Use pre-deployment checks to validate configuration.
+ - **Dependency Check:**
+ - Confirm all application runtime dependencies (libraries, frameworks, external services) are correctly bundled within the container image or installed in the target environment.
+ - **Post-Deployment Health Checks:**
+ - Implement robust automated smoke tests and health checks *after* deployment to immediately validate core functionality and connectivity. Trigger rollbacks if these fail.
+ - **Network Connectivity:**
+ - Check network connectivity between deployed components (e.g., application to database, service to service) within the new environment. Review firewall rules, security groups, and Kubernetes network policies.
+ - **Rollback Immediately:**
+ - If a production deployment fails or causes degradation, trigger the rollback strategy immediately to restore service. Diagnose the issue in a non-production environment.
## Conclusion
diff --git a/.github/instructions/go.instructions.md b/.github/instructions/go.instructions.md
index b918f15dd..035ca0ba4 100644
--- a/.github/instructions/go.instructions.md
+++ b/.github/instructions/go.instructions.md
@@ -34,7 +34,8 @@ Follow idiomatic Go practices and community standards when writing Go code. Thes
- Avoid generic names like `util`, `common`, or `base`
- Package names should be singular, not plural
-#### Package Declaration Rules (CRITICAL):
+#### Package Declaration Rules (CRITICAL)
+
- **NEVER duplicate `package` declarations** - each Go file must have exactly ONE `package` line
- When editing an existing `.go` file:
- **PRESERVE** the existing `package` declaration - do not add another one
@@ -175,14 +176,16 @@ Follow idiomatic Go practices and community standards when writing Go code. Thes
- Choose between channels and mutexes based on the use case: use channels for communication, mutexes for protecting state
- Use `sync.Once` for one-time initialization
- WaitGroup usage by Go version:
- - If `go >= 1.25` in `go.mod`, use the new `WaitGroup.Go` method ([documentation](https://pkg.go.dev/sync#WaitGroup)):
- ```go
+ - If `go >= 1.25` in `go.mod`, use the new `WaitGroup.Go` method ([documentation](https://pkg.go.dev/sync#WaitGroup)):
+
+ ```go
var wg sync.WaitGroup
wg.Go(task1)
wg.Go(task2)
wg.Wait()
- ```
- - If `go < 1.25`, use the classic `Add`/`Done` pattern
+ ```
+
+ - If `go < 1.25`, use the classic `Add`/`Done` pattern
## Error Handling Patterns
@@ -211,8 +214,8 @@ Follow idiomatic Go practices and community standards when writing Go code. Thes
- Set appropriate status codes and headers
- Handle errors gracefully and return appropriate error responses
- Router usage by Go version:
- - If `go >= 1.22`, prefer the enhanced `net/http` `ServeMux` with pattern-based routing and method matching
- - If `go < 1.22`, use the classic `ServeMux` and handle methods/paths manually (or use a third-party router when justified)
+ - If `go >= 1.22`, prefer the enhanced `net/http` `ServeMux` with pattern-based routing and method matching
+ - If `go < 1.22`, use the classic `ServeMux` and handle methods/paths manually (or use a third-party router when justified)
### JSON APIs
@@ -245,28 +248,28 @@ Follow idiomatic Go practices and community standards when writing Go code. Thes
- Most `io.Reader` streams are consumable once; reading advances state. Do not assume a reader can be re-read without special handling
- If you must read data multiple times, buffer it once and recreate readers on demand:
- - Use `io.ReadAll` (or a limited read) to obtain `[]byte`, then create fresh readers via `bytes.NewReader(buf)` or `bytes.NewBuffer(buf)` for each reuse
- - For strings, use `strings.NewReader(s)`; you can `Seek(0, io.SeekStart)` on `*bytes.Reader` to rewind
+ - Use `io.ReadAll` (or a limited read) to obtain `[]byte`, then create fresh readers via `bytes.NewReader(buf)` or `bytes.NewBuffer(buf)` for each reuse
+ - For strings, use `strings.NewReader(s)`; you can `Seek(0, io.SeekStart)` on `*bytes.Reader` to rewind
- For HTTP requests, do not reuse a consumed `req.Body`. Instead:
- - Keep the original payload as `[]byte` and set `req.Body = io.NopCloser(bytes.NewReader(buf))` before each send
- - Prefer configuring `req.GetBody` so the transport can recreate the body for redirects/retries: `req.GetBody = func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(buf)), nil }`
+ - Keep the original payload as `[]byte` and set `req.Body = io.NopCloser(bytes.NewReader(buf))` before each send
+ - Prefer configuring `req.GetBody` so the transport can recreate the body for redirects/retries: `req.GetBody = func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(buf)), nil }`
- To duplicate a stream while reading, use `io.TeeReader` (copy to a buffer while passing through) or write to multiple sinks with `io.MultiWriter`
- Reusing buffered readers: call `(*bufio.Reader).Reset(r)` to attach to a new underlying reader; do not expect it to βrewindβ unless the source supports seeking
- For large payloads, avoid unbounded buffering; consider streaming, `io.LimitReader`, or on-disk temporary storage to control memory
- Use `io.Pipe` to stream without buffering the whole payload:
- - Write to `*io.PipeWriter` in a separate goroutine while the reader consumes
- - Always close the writer; use `CloseWithError(err)` on failures
- - `io.Pipe` is for streaming, not rewinding or making readers reusable
+ - Write to `*io.PipeWriter` in a separate goroutine while the reader consumes
+ - Always close the writer; use `CloseWithError(err)` on failures
+ - `io.Pipe` is for streaming, not rewinding or making readers reusable
- **Warning:** When using `io.Pipe` (especially with multipart writers), all writes must be performed in strict, sequential order. Do not write concurrently or out of orderβmultipart boundaries and chunk order must be preserved. Out-of-order or parallel writes can corrupt the stream and result in errors.
- Streaming multipart/form-data with `io.Pipe`:
- - `pr, pw := io.Pipe()`; `mw := multipart.NewWriter(pw)`; use `pr` as the HTTP request body
- - Set `Content-Type` to `mw.FormDataContentType()`
- - In a goroutine: write all parts to `mw` in the correct order; on error `pw.CloseWithError(err)`; on success `mw.Close()` then `pw.Close()`
- - Do not store request/in-flight form state on a long-lived client; build per call
- - Streamed bodies are not rewindable; for retries/redirects, buffer small payloads or provide `GetBody`
+ - `pr, pw := io.Pipe()`; `mw := multipart.NewWriter(pw)`; use `pr` as the HTTP request body
+ - Set `Content-Type` to `mw.FormDataContentType()`
+ - In a goroutine: write all parts to `mw` in the correct order; on error `pw.CloseWithError(err)`; on success `mw.Close()` then `pw.Close()`
+ - Do not store request/in-flight form state on a long-lived client; build per call
+ - Streamed bodies are not rewindable; for retries/redirects, buffer small payloads or provide `GetBody`
### Profiling
diff --git a/.github/instructions/makefile.instructions.md b/.github/instructions/makefile.instructions.md
index 45b5bad99..bb2dd61f0 100644
--- a/.github/instructions/makefile.instructions.md
+++ b/.github/instructions/makefile.instructions.md
@@ -175,6 +175,7 @@ CFLAGS = -Wall -O2
### Splitting Without Adding Whitespace
+
If you need to split a line without adding whitespace, you can use a special technique: insert `$ ` (dollar-space) followed by a backslash-newline. The `$ ` refers to a variable with a single-space name, which doesn't exist and expands to nothing, effectively joining the lines without inserting a space.
```makefile
diff --git a/.github/instructions/markdown.instructions.md b/.github/instructions/markdown.instructions.md
index 184206dde..ca25e3006 100644
--- a/.github/instructions/markdown.instructions.md
+++ b/.github/instructions/markdown.instructions.md
@@ -23,7 +23,7 @@ Follow these guidelines for formatting and structuring your markdown content:
- **Headings**: Use `##` for H2 and `###` for H3. Ensure that headings are used in a hierarchical manner. Recommend restructuring if content includes H4, and more strongly recommend for H5.
- **Lists**: Use `-` for bullet points and `1.` for numbered lists. Indent nested lists with two spaces.
-- **Code Blocks**: Use triple backticks (`) to create fenced code blocks. Specify the language after the opening backticks for syntax highlighting (e.g., `csharp).
+- **Code Blocks**: Use triple backticks (`` ``` ``) to create fenced code blocks. Specify the language after the opening backticks for syntax highlighting (e.g., `` ```csharp ``).
- **Links**: Use `[link text](https://example.com)` for links. Ensure that the link text is descriptive and the URL is valid.
- **Images**: Use `` for images. Include a brief description of the image in the alt text.
- **Tables**: Use `|` to create tables. Ensure that columns are properly aligned and headers are included.
diff --git a/.github/instructions/nodejs-javascript-vitest.instructions.md b/.github/instructions/nodejs-javascript-vitest.instructions.md
index 6a38953bb..5efce2bbe 100644
--- a/.github/instructions/nodejs-javascript-vitest.instructions.md
+++ b/.github/instructions/nodejs-javascript-vitest.instructions.md
@@ -6,6 +6,7 @@ applyTo: '**/*.js, **/*.mjs, **/*.cjs'
# Code Generation Guidelines
## Coding standards
+
- Use JavaScript with ES2022 features and Node.js (20+) ESM modules
- Use Node.js built-in modules and avoid external dependencies where possible
- Ask the user if you require any additional dependencies before adding them
@@ -17,14 +18,17 @@ applyTo: '**/*.js, **/*.mjs, **/*.cjs'
- Prefer functions over classes
## Testing
+
- Use Vitest for testing
- Write tests for all new features and bug fixes
- Ensure tests cover edge cases and error handling
- NEVER change the original code to make it easier to test, instead, write tests that cover the original code as it is
## Documentation
+
- When adding new features or making significant changes, update the README.md file where necessary
## User interactions
+
- Ask questions if you are unsure about the implementation details, design choices, or need clarification on the requirements
- Always answer in the same language as the question, but use english for the generated content like code, comments or docs
diff --git a/.github/instructions/object-calisthenics.instructions.md b/.github/instructions/object-calisthenics.instructions.md
index 7c62ea6d5..860d40aec 100644
--- a/.github/instructions/object-calisthenics.instructions.md
+++ b/.github/instructions/object-calisthenics.instructions.md
@@ -8,9 +8,11 @@ description: Enforces Object Calisthenics principles for business domain code to
> Examples may be added later if needed.
## Objective
+
This rule enforces the principles of Object Calisthenics to ensure clean, maintainable, and robust code in the backend, **primarily for business domain code**.
## Scope and Application
+
- **Primary focus**: Business domain classes (aggregates, entities, value objects, domain services)
- **Secondary focus**: Application layer services and use case handlers
- **Exemptions**:
@@ -22,7 +24,6 @@ This rule enforces the principles of Object Calisthenics to ensure clean, mainta
## Key Principles
-
1. **One Level of Indentation per Method**:
- Ensure methods are simple and do not exceed one level of indentation.
@@ -57,6 +58,7 @@ This rule enforces the principles of Object Calisthenics to ensure clean, mainta
}
}
```
+
2. **Don't Use the ELSE Keyword**:
- Avoid using the `else` keyword to reduce complexity and improve readability.
@@ -81,6 +83,7 @@ This rule enforces the principles of Object Calisthenics to ensure clean, mainta
```
Sample Fail fast principle:
+
```csharp
public void ProcessOrder(Order order) {
if (order == null) throw new ArgumentNullException(nameof(order));
@@ -222,7 +225,6 @@ First Class Collections: a class that contains an array as an attribute should n
}
```
-
8. **No Classes with More Than Two Instance Variables**:
- Encourage classes to have a single responsibility by limiting the number of instance variables.
- Limit the number of instance variables to two to maintain simplicity.
@@ -284,6 +286,7 @@ First Class Collections: a class that contains an array as an attribute should n
```
## Implementation Guidelines
+
- **Domain Classes**:
- Use private constructors and static factory methods for creating instances.
- Avoid exposing setters for properties.
@@ -306,6 +309,7 @@ First Class Collections: a class that contains an array as an attribute should n
- Be pragmatic about infrastructure and DTO code.
## References
+
- [Object Calisthenics - Original 9 Rules by Jeff Bay](https://www.cs.helsinki.fi/u/luontola/tdd-2009/ext/ObjectCalisthenics.pdf)
- [ThoughtWorks - Object Calisthenics](https://www.thoughtworks.com/insights/blog/object-calisthenics)
- [Clean Code: A Handbook of Agile Software Craftsmanship - Robert C. Martin](https://www.oreilly.com/library/view/clean-code-a/9780136083238/)
diff --git a/.github/instructions/pcf-react-platform-libraries.instructions.md b/.github/instructions/pcf-react-platform-libraries.instructions.md
index 634b205c6..b329b89ba 100644
--- a/.github/instructions/pcf-react-platform-libraries.instructions.md
+++ b/.github/instructions/pcf-react-platform-libraries.instructions.md
@@ -33,7 +33,7 @@ There's a new `--framework` (`-fw`) parameter for the `pac pcf init` command. Se
### Command Parameters
| Parameter | Value |
-|-----------|-------|
+| ----------- | ------- |
| --name | ReactSample |
| --namespace | SampleNamespace |
| --template | field |
@@ -87,7 +87,7 @@ React and Fluent libraries aren't included in the package because they're shared
The following controls are included in the samples. They function the same as their standard versions but offer better performance since they are virtual controls.
| Sample | Description | Link |
-|--------|-------------|------|
+| --- | --- | --- |
| ChoicesPickerReact | The standard ChoicesPickerControl converted to be a React Control | ChoicesPickerReact Sample |
| FacepileReact | The ReactStandardControl converted to be a React Control | FacepileReact |
@@ -96,7 +96,7 @@ The following controls are included in the samples. They function the same as th
Platform libraries are made available both at the build and runtime to the controls that are using platform libraries capability. Currently, the following versions are provided by the platform and are the highest currently supported versions.
| Library | Package | Build Version | Runtime Version |
-|---------|---------|---------------|-----------------|
+| --------- | --------- | --------------- | ----------------- |
| React | react | 16.14.0 | 17.0.2 (Model), 16.14.0 (Canvas) |
| Fluent | @fluentui/react | 8.29.0 | 8.29.0 |
| Fluent | @fluentui/react | 8.121.1 | 8.121.1 |
diff --git a/.github/instructions/performance-optimization.instructions.md b/.github/instructions/performance-optimization.instructions.md
index 46a40025d..09ac30952 100644
--- a/.github/instructions/performance-optimization.instructions.md
+++ b/.github/instructions/performance-optimization.instructions.md
@@ -29,6 +29,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
## Frontend Performance
### Rendering and DOM
+
- **Minimize DOM Manipulations:** Batch updates where possible. Frequent DOM changes are expensive.
- *Anti-pattern:* Updating the DOM in a loop. Instead, build a document fragment and append it once.
- **Virtual DOM Frameworks:** Use React, Vue, or similar efficientlyβavoid unnecessary re-renders.
@@ -39,6 +40,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Defer Non-Critical Rendering:** Use `requestIdleCallback` or similar to defer work until the browser is idle.
### Asset Optimization
+
- **Image Compression:** Use tools like ImageOptim, Squoosh, or TinyPNG. Prefer modern formats (WebP, AVIF) for web delivery.
- **SVGs for Icons:** SVGs scale well and are often smaller than PNGs for simple graphics.
- **Minification and Bundling:** Use Webpack, Rollup, or esbuild to bundle and minify JS/CSS. Enable tree-shaking to remove dead code.
@@ -47,6 +49,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Font Optimization:** Use only the character sets you need. Subset fonts and use `font-display: swap`.
### Network Optimization
+
- **Reduce HTTP Requests:** Combine files, use image sprites, and inline critical CSS.
- **HTTP/2 and HTTP/3:** Enable these protocols for multiplexing and lower latency.
- **Client-Side Caching:** Use Service Workers, IndexedDB, and localStorage for offline and repeat visits.
@@ -55,6 +58,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Preload and Prefetch:** Use `` and `` for critical resources.
### JavaScript Performance
+
- **Avoid Blocking the Main Thread:** Offload heavy computation to Web Workers.
- **Debounce/Throttle Events:** For scroll, resize, and input events, use debounce/throttle to limit handler frequency.
- **Memory Leaks:** Clean up event listeners, intervals, and DOM references. Use browser dev tools to check for detached nodes.
@@ -63,11 +67,14 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Avoid Deep Object Cloning:** Use shallow copies or libraries like lodash's `cloneDeep` only when necessary.
### Accessibility and Performance
+
- **Accessible Components:** Ensure ARIA updates are not excessive. Use semantic HTML for both accessibility and performance.
- **Screen Reader Performance:** Avoid rapid DOM updates that can overwhelm assistive tech.
### Framework-Specific Tips
+
#### React
+
- Use `React.memo`, `useMemo`, and `useCallback` to avoid unnecessary renders.
- Split large components and use code-splitting (`React.lazy`, `Suspense`).
- Avoid anonymous functions in render; they create new references on every render.
@@ -75,6 +82,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Profile with React DevTools Profiler.
#### Angular
+
- Use OnPush change detection for components that don't need frequent updates.
- Avoid complex expressions in templates; move logic to the component class.
- Use `trackBy` in `ngFor` for efficient list rendering.
@@ -82,12 +90,14 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Profile with Angular DevTools.
#### Vue
+
- Use computed properties over methods in templates for caching.
- Use `v-show` vs `v-if` appropriately (`v-show` is better for toggling visibility frequently).
- Lazy load components and routes with Vue Router.
- Profile with Vue Devtools.
### Common Frontend Pitfalls
+
- Loading large JS bundles on initial page load.
- Not compressing images or using outdated formats.
- Failing to clean up event listeners, causing memory leaks.
@@ -95,6 +105,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Ignoring mobile performance (test on real devices!).
### Frontend Troubleshooting
+
- Use Chrome DevTools' Performance tab to record and analyze slow frames.
- Use Lighthouse to audit performance and get actionable suggestions.
- Use WebPageTest for real-world load testing.
@@ -105,6 +116,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
## Backend Performance
### Algorithm and Data Structure Optimization
+
- **Choose the Right Data Structure:** Arrays for sequential access, hash maps for fast lookups, trees for hierarchical data, etc.
- **Efficient Algorithms:** Use binary search, quicksort, or hash-based algorithms where appropriate.
- **Avoid O(n^2) or Worse:** Profile nested loops and recursive calls. Refactor to reduce complexity.
@@ -112,6 +124,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Streaming:** Use streaming APIs for large data sets to avoid loading everything into memory.
### Concurrency and Parallelism
+
- **Asynchronous I/O:** Use async/await, callbacks, or event loops to avoid blocking threads.
- **Thread/Worker Pools:** Use pools to manage concurrency and avoid resource exhaustion.
- **Avoid Race Conditions:** Use locks, semaphores, or atomic operations where needed.
@@ -119,6 +132,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Backpressure:** Implement backpressure in queues and pipelines to avoid overload.
### Caching
+
- **Cache Expensive Computations:** Use in-memory caches (Redis, Memcached) for hot data.
- **Cache Invalidation:** Use time-based (TTL), event-based, or manual invalidation. Stale cache is worse than no cache.
- **Distributed Caching:** For multi-server setups, use distributed caches and be aware of consistency issues.
@@ -126,6 +140,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Don't Cache Everything:** Some data is too volatile or sensitive to cache.
### API and Network
+
- **Minimize Payloads:** Use JSON, compress responses (gzip, Brotli), and avoid sending unnecessary data.
- **Pagination:** Always paginate large result sets. Use cursors for real-time data.
- **Rate Limiting:** Protect APIs from abuse and overload.
@@ -133,13 +148,16 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Protocol Choice:** Use HTTP/2, gRPC, or WebSockets for high-throughput, low-latency communication.
### Logging and Monitoring
+
- **Minimize Logging in Hot Paths:** Excessive logging can slow down critical code.
- **Structured Logging:** Use JSON or key-value logs for easier parsing and analysis.
- **Monitor Everything:** Latency, throughput, error rates, resource usage. Use Prometheus, Grafana, Datadog, or similar.
- **Alerting:** Set up alerts for performance regressions and resource exhaustion.
### Language/Framework-Specific Tips
+
#### Node.js
+
- Use asynchronous APIs; avoid blocking the event loop (e.g., never use `fs.readFileSync` in production).
- Use clustering or worker threads for CPU-bound tasks.
- Limit concurrent open connections to avoid resource exhaustion.
@@ -147,6 +165,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Profile with `clinic.js`, `node --inspect`, or Chrome DevTools.
#### Python
+
- Use built-in data structures (`dict`, `set`, `deque`) for speed.
- Profile with `cProfile`, `line_profiler`, or `Py-Spy`.
- Use `multiprocessing` or `asyncio` for parallelism.
@@ -154,6 +173,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Use `lru_cache` for memoization.
#### Java
+
- Use efficient collections (`ArrayList`, `HashMap`, etc.).
- Profile with VisualVM, JProfiler, or YourKit.
- Use thread pools (`Executors`) for concurrency.
@@ -161,6 +181,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Use `CompletableFuture` for async programming.
#### .NET
+
- Use `async/await` for I/O-bound operations.
- Use `Span` and `Memory` for efficient memory access.
- Profile with dotTrace, Visual Studio Profiler, or PerfView.
@@ -168,6 +189,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Use `IAsyncEnumerable` for streaming data.
### Common Backend Pitfalls
+
- Synchronous/blocking I/O in web servers.
- Not using connection pooling for databases.
- Over-caching or caching sensitive/volatile data.
@@ -175,6 +197,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Not monitoring or alerting on performance regressions.
### Backend Troubleshooting
+
- Use flame graphs to visualize CPU usage.
- Use distributed tracing (OpenTelemetry, Jaeger, Zipkin) to track request latency across services.
- Use heap dumps and memory profilers to find leaks.
@@ -185,6 +208,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
## Database Performance
### Query Optimization
+
- **Indexes:** Use indexes on columns that are frequently queried, filtered, or joined. Monitor index usage and drop unused indexes.
- **Avoid SELECT *:** Select only the columns you need. Reduces I/O and memory usage.
- **Parameterized Queries:** Prevent SQL injection and improve plan caching.
@@ -193,6 +217,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Limit Result Sets:** Use `LIMIT`/`OFFSET` or cursors for large tables.
### Schema Design
+
- **Normalization:** Normalize to reduce redundancy, but denormalize for read-heavy workloads if needed.
- **Data Types:** Use the most efficient data types and set appropriate constraints.
- **Partitioning:** Partition large tables for scalability and manageability.
@@ -200,17 +225,20 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Foreign Keys:** Use them for data integrity, but be aware of performance trade-offs in high-write scenarios.
### Transactions
+
- **Short Transactions:** Keep transactions as short as possible to reduce lock contention.
- **Isolation Levels:** Use the lowest isolation level that meets your consistency needs.
- **Avoid Long-Running Transactions:** They can block other operations and increase deadlocks.
### Caching and Replication
+
- **Read Replicas:** Use for scaling read-heavy workloads. Monitor replication lag.
- **Cache Query Results:** Use Redis or Memcached for frequently accessed queries.
- **Write-Through/Write-Behind:** Choose the right strategy for your consistency needs.
- **Sharding:** Distribute data across multiple servers for scalability.
### NoSQL Databases
+
- **Design for Access Patterns:** Model your data for the queries you need.
- **Avoid Hot Partitions:** Distribute writes/reads evenly.
- **Unbounded Growth:** Watch for unbounded arrays or documents.
@@ -218,6 +246,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- **Consistency Models:** Understand eventual vs strong consistency and choose appropriately.
### Common Database Pitfalls
+
- Missing or unused indexes.
- SELECT * in production queries.
- Not monitoring slow queries.
@@ -225,6 +254,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
- Not archiving old data.
### Database Troubleshooting
+
- Use slow query logs to identify bottlenecks.
- Use `EXPLAIN` to analyze query plans.
- Monitor cache hit/miss ratios.
@@ -255,35 +285,41 @@ Performance isn't just a buzzwordβit's the difference between a product people
## Advanced Topics
### Profiling and Benchmarking
+
- **Profilers:** Use language-specific profilers (Chrome DevTools, Py-Spy, VisualVM, dotTrace, etc.) to identify bottlenecks.
- **Microbenchmarks:** Write microbenchmarks for critical code paths. Use `benchmark.js`, `pytest-benchmark`, or JMH for Java.
- **A/B Testing:** Measure real-world impact of optimizations with A/B or canary releases.
- **Continuous Performance Testing:** Integrate performance tests into CI/CD. Use tools like k6, Gatling, or Locust.
### Memory Management
+
- **Resource Cleanup:** Always release resources (files, sockets, DB connections) promptly.
- **Object Pooling:** Use for frequently created/destroyed objects (e.g., DB connections, threads).
- **Heap Monitoring:** Monitor heap usage and garbage collection. Tune GC settings for your workload.
- **Memory Leaks:** Use leak detection tools (Valgrind, LeakCanary, Chrome DevTools).
### Scalability
+
- **Horizontal Scaling:** Design stateless services, use sharding/partitioning, and load balancers.
- **Auto-Scaling:** Use cloud auto-scaling groups and set sensible thresholds.
- **Bottleneck Analysis:** Identify and address single points of failure.
- **Distributed Systems:** Use idempotent operations, retries, and circuit breakers.
### Security and Performance
+
- **Efficient Crypto:** Use hardware-accelerated and well-maintained cryptographic libraries.
- **Validation:** Validate inputs efficiently; avoid regexes in hot paths.
- **Rate Limiting:** Protect against DoS without harming legitimate users.
### Mobile Performance
+
- **Startup Time:** Lazy load features, defer heavy work, and minimize initial bundle size.
- **Image/Asset Optimization:** Use responsive images and compress assets for mobile bandwidth.
- **Efficient Storage:** Use SQLite, Realm, or platform-optimized storage.
- **Profiling:** Use Android Profiler, Instruments (iOS), or Firebase Performance Monitoring.
### Cloud and Serverless
+
- **Cold Starts:** Minimize dependencies and keep functions warm.
- **Resource Allocation:** Tune memory/CPU for serverless functions.
- **Managed Services:** Use managed caching, queues, and DBs for scalability.
@@ -294,6 +330,7 @@ Performance isn't just a buzzwordβit's the difference between a product people
## Practical Examples
### Example 1: Debouncing User Input in JavaScript
+
```javascript
// BAD: Triggers API call on every keystroke
input.addEventListener('input', (e) => {
@@ -311,6 +348,7 @@ input.addEventListener('input', (e) => {
```
### Example 2: Efficient SQL Query
+
```sql
-- BAD: Selects all columns and does not use an index
SELECT * FROM users WHERE email = 'user@example.com';
@@ -320,6 +358,7 @@ SELECT id, name FROM users WHERE email = 'user@example.com';
```
### Example 3: Caching Expensive Computation in Python
+
```python
# BAD: Recomputes result every time
result = expensive_function(x)
@@ -334,6 +373,7 @@ result = expensive_function(x)
```
### Example 4: Lazy Loading Images in HTML
+
```html
@@ -343,6 +383,7 @@ result = expensive_function(x)
```
### Example 5: Asynchronous I/O in Node.js
+
```javascript
// BAD: Blocking file read
const data = fs.readFileSync('file.txt');
@@ -355,6 +396,7 @@ fs.readFile('file.txt', (err, data) => {
```
### Example 6: Profiling a Python Function
+
```python
import cProfile
import pstats
@@ -368,6 +410,7 @@ p.sort_stats('cumulative').print_stats(10)
```
### Example 7: Using Redis for Caching in Node.js
+
```javascript
const redis = require('redis');
const client = redis.createClient();
@@ -388,6 +431,7 @@ function getCachedData(key, fetchFunction) {
---
## References and Further Reading
+
- [Google Web Fundamentals: Performance](https://web.dev/performance/)
- [MDN Web Docs: Performance](https://developer.mozilla.org/en-US/docs/Web/Performance)
- [OWASP: Performance Testing](https://owasp.org/www-project-performance-testing/)
diff --git a/.github/instructions/playwright-typescript.instructions.md b/.github/instructions/playwright-typescript.instructions.md
index e9b1b8718..6c921ad42 100644
--- a/.github/instructions/playwright-typescript.instructions.md
+++ b/.github/instructions/playwright-typescript.instructions.md
@@ -6,31 +6,32 @@ applyTo: '**'
## Test Writing Guidelines
### Code Quality Standards
+
- **Locators**: Prioritize user-facing, role-based locators (`getByRole`, `getByLabel`, `getByText`, etc.) for resilience and accessibility. Use `test.step()` to group interactions and improve test readability and reporting.
- **Assertions**: Use auto-retrying web-first assertions. These assertions start with the `await` keyword (e.g., `await expect(locator).toHaveText()`). Avoid `expect(locator).toBeVisible()` unless specifically testing for visibility changes.
- **Timeouts**: Rely on Playwright's built-in auto-waiting mechanisms. Avoid hard-coded waits or increased default timeouts.
- **Clarity**: Use descriptive test and step titles that clearly state the intent. Add comments only to explain complex logic or non-obvious interactions.
-
### Test Structure
+
- **Imports**: Start with `import { test, expect } from '@playwright/test';`.
- **Organization**: Group related tests for a feature under a `test.describe()` block.
- **Hooks**: Use `beforeEach` for setup actions common to all tests in a `describe` block (e.g., navigating to a page).
- **Titles**: Follow a clear naming convention, such as `Feature - Specific action or scenario`.
-
### File Organization
+
- **Location**: Store all test files in the `tests/` directory.
- **Naming**: Use the convention `.spec.ts` (e.g., `login.spec.ts`, `search.spec.ts`).
- **Scope**: Aim for one test file per major application feature or page.
### Assertion Best Practices
+
- **UI Structure**: Use `toMatchAriaSnapshot` to verify the accessibility tree structure of a component. This provides a comprehensive and accessible snapshot.
- **Element Counts**: Use `toHaveCount` to assert the number of elements found by a locator.
- **Text Content**: Use `toHaveText` for exact text matches and `toContainText` for partial matches.
- **Navigation**: Use `toHaveURL` to verify the page URL after an action.
-
## Example Test Structure
```typescript
@@ -79,6 +80,7 @@ test.describe('Movie Search Feature', () => {
## Quality Checklist
Before finalizing tests, ensure:
+
- [ ] All locators are accessible and specific and avoid strict mode violations
- [ ] Tests are grouped logically and follow a clear structure
- [ ] Assertions are meaningful and reflect user expectations
diff --git a/.github/instructions/prompt.instructions.md b/.github/instructions/prompt.instructions.md
index 7ca0432b1..23d4ac2c4 100644
--- a/.github/instructions/prompt.instructions.md
+++ b/.github/instructions/prompt.instructions.md
@@ -8,52 +8,62 @@ applyTo: '**/*.prompt.md'
Instructions for creating effective and maintainable prompt files that guide GitHub Copilot in delivering consistent, high-quality outcomes across any repository.
## Scope and Principles
+
- Target audience: maintainers and contributors authoring reusable prompts for Copilot Chat.
- Goals: predictable behaviour, clear expectations, minimal permissions, and portability across repositories.
- Primary references: VS Code documentation on prompt files and organization-specific conventions.
## Frontmatter Requirements
+
- Include `description` (single sentence, actionable outcome), `mode` (explicitly choose `ask`, `edit`, or `agent`), and `tools` (minimal set of tool bundles required to fulfill the prompt).
- Declare `model` when the prompt depends on a specific capability tier; otherwise inherit the active model.
- Preserve any additional metadata (`language`, `tags`, `visibility`, etc.) required by your organization.
- Use consistent quoting (single quotes recommended) and keep one field per line for readability and version control clarity.
## File Naming and Placement
+
- Use kebab-case filenames ending with `.prompt.md` and store them under `.github/prompts/` unless your workspace standard specifies another directory.
- Provide a short filename that communicates the action (for example, `generate-readme.prompt.md` rather than `prompt1.prompt.md`).
## Body Structure
+
- Start with an `#` level heading that matches the prompt intent so it surfaces well in Quick Pick search.
- Organize content with predictable sections. Recommended baseline: `Mission` or `Primary Directive`, `Scope & Preconditions`, `Inputs`, `Workflow` (step-by-step), `Output Expectations`, and `Quality Assurance`.
- Adjust section names to fit the domain, but retain the logical flow: why β context β inputs β actions β outputs β validation.
- Reference related prompts or instruction files using relative links to aid discoverability.
## Input and Context Handling
+
- Use `${input:variableName[:placeholder]}` for required values and explain when the user must supply them. Provide defaults or alternatives where possible.
- Call out contextual variables such as `${selection}`, `${file}`, `${workspaceFolder}` only when they are essential, and describe how Copilot should interpret them.
- Document how to proceed when mandatory context is missing (for example, βRequest the file path and stop if it remains undefinedβ).
## Tool and Permission Guidance
+
- Limit `tools` to the smallest set that enables the task. List them in the preferred execution order when the sequence matters.
- If the prompt inherits tools from a chat mode, mention that relationship and state any critical tool behaviours or side effects.
- Warn about destructive operations (file creation, edits, terminal commands) and include guard rails or confirmation steps in the workflow.
## Instruction Tone and Style
+
- Write in direct, imperative sentences targeted at Copilot (for example, βAnalyzeβ, βGenerateβ, βSummarizeβ).
- Keep sentences short and unambiguous, following Google Developer Documentation translation best practices to support localization.
- Avoid idioms, humor, or culturally specific references; favor neutral, inclusive language.
## Output Definition
+
- Specify the format, structure, and location of expected results (for example, βCreate `docs/adr/adr-XXXX.md` using the template belowβ).
- Include success criteria and failure triggers so Copilot knows when to halt or retry.
- Provide validation stepsβmanual checks, automated commands, or acceptance criteria listsβthat reviewers can execute after running the prompt.
## Examples and Reusable Assets
+
- Embed Good/Bad examples or scaffolds (Markdown templates, JSON stubs) that the prompt should produce or follow.
- Maintain reference tables (capabilities, status codes, role descriptions) inline to keep the prompt self-contained. Update these tables when upstream resources change.
- Link to authoritative documentation instead of duplicating lengthy guidance.
## Quality Assurance Checklist
+
- [ ] Frontmatter fields are complete, accurate, and least-privilege.
- [ ] Inputs include placeholders, default behaviours, and fallbacks.
- [ ] Workflow covers preparation, execution, and post-processing without gaps.
@@ -63,11 +73,13 @@ Instructions for creating effective and maintainable prompt files that guide Git
- [ ] Prompt executes successfully in VS Code (`Chat: Run Prompt`) using representative scenarios.
## Maintenance Guidance
+
- Version-control prompts alongside the code they affect; update them when dependencies, tooling, or review processes change.
- Review prompts periodically to ensure tool lists, model requirements, and linked documents remain valid.
- Coordinate with other repositories: when a prompt proves broadly useful, extract common guidance into instruction files or shared prompt packs.
## Additional Resources
+
- [Prompt Files Documentation](https://code.visualstudio.com/docs/copilot/customization/prompt-files#_prompt-file-format)
- [Awesome Copilot Prompt Files](https://github.com/github/awesome-copilot/tree/main/prompts)
- [Tool Configuration](https://code.visualstudio.com/docs/copilot/chat/chat-agent-mode#_agent-mode-tools)
diff --git a/.github/instructions/reactjs.instructions.md b/.github/instructions/reactjs.instructions.md
index 79bd2754e..c95e96955 100644
--- a/.github/instructions/reactjs.instructions.md
+++ b/.github/instructions/reactjs.instructions.md
@@ -5,9 +5,10 @@ applyTo: '**/*.jsx, **/*.tsx, **/*.js, **/*.ts, **/*.css, **/*.scss'
# ReactJS Development Instructions
-Instructions for building high-quality ReactJS applications with modern patterns, hooks, and best practices following the official React documentation at https://react.dev.
+Instructions for building high-quality ReactJS applications with modern patterns, hooks, and best practices following the official React documentation at .
## Project Context
+
- Latest React version (React 19+)
- TypeScript for type safety (when applicable)
- Functional components with hooks as default
@@ -18,6 +19,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
## Development Standards
### Architecture
+
- Use functional components with hooks as the primary pattern
- Implement component composition over inheritance
- Organize components by feature or domain for scalability
@@ -26,6 +28,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Implement proper component hierarchies with clear data flow
### TypeScript Integration
+
- Use TypeScript interfaces for props, state, and component definitions
- Define proper types for event handlers and refs
- Implement generic components where appropriate
@@ -34,6 +37,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Create union types for component variants and states
### Component Design
+
- Follow the single responsibility principle for components
- Use descriptive and consistent naming conventions
- Implement proper prop validation with TypeScript or PropTypes
@@ -42,6 +46,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Use composition patterns (render props, children as functions)
### State Management
+
- Use `useState` for local component state
- Implement `useReducer` for complex state logic
- Leverage `useContext` for sharing state across component trees
@@ -50,6 +55,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Use React Query or SWR for server state management
### Hooks and Effects
+
- Use `useEffect` with proper dependency arrays to avoid infinite loops
- Implement cleanup functions in effects to prevent memory leaks
- Use `useMemo` and `useCallback` for performance optimization when needed
@@ -58,6 +64,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Use `useRef` for accessing DOM elements and storing mutable values
### Styling
+
- Use CSS Modules, Styled Components, or modern CSS-in-JS solutions
- Implement responsive design with mobile-first approach
- Follow BEM methodology or similar naming conventions for CSS classes
@@ -66,6 +73,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Ensure accessibility with proper ARIA attributes and semantic HTML
### Performance Optimization
+
- Use `React.memo` for component memoization when appropriate
- Implement code splitting with `React.lazy` and `Suspense`
- Optimize bundle size with tree shaking and dynamic imports
@@ -74,6 +82,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Profile components with React DevTools to identify performance bottlenecks
### Data Fetching
+
- Use modern data fetching libraries (React Query, SWR, Apollo Client)
- Implement proper loading, error, and success states
- Handle race conditions and request cancellation
@@ -82,6 +91,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Handle offline scenarios and network errors gracefully
### Error Handling
+
- Implement Error Boundaries for component-level error handling
- Use proper error states in data fetching
- Implement fallback UI for error scenarios
@@ -90,6 +100,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Provide meaningful error messages to users
### Forms and Validation
+
- Use controlled components for form inputs
- Implement proper form validation with libraries like Formik, React Hook Form
- Handle form submission and error states appropriately
@@ -98,6 +109,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Handle file uploads and complex form scenarios
### Routing
+
- Use React Router for client-side routing
- Implement nested routes and route protection
- Handle route parameters and query strings properly
@@ -106,6 +118,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Implement breadcrumbs and navigation state management
### Testing
+
- Write unit tests for components using React Testing Library
- Test component behavior, not implementation details
- Use Jest for test runner and assertion library
@@ -114,6 +127,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Test accessibility features and keyboard navigation
### Security
+
- Sanitize user inputs to prevent XSS attacks
- Validate and escape data before rendering
- Use HTTPS for all external API calls
@@ -122,6 +136,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Use Content Security Policy (CSP) headers
### Accessibility
+
- Use semantic HTML elements appropriately
- Implement proper ARIA attributes and roles
- Ensure keyboard navigation works for all interactive elements
@@ -130,6 +145,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Test with screen readers and accessibility tools
## Implementation Process
+
1. Plan component architecture and data flow
2. Set up project structure with proper folder organization
3. Define TypeScript interfaces and types
@@ -144,6 +160,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
12. Add documentation and code comments
## Additional Guidelines
+
- Follow React's naming conventions (PascalCase for components, camelCase for functions)
- Use meaningful commit messages and maintain clean git history
- Implement proper code splitting and lazy loading strategies
@@ -154,6 +171,7 @@ Instructions for building high-quality ReactJS applications with modern patterns
- Use React Developer Tools for debugging and performance analysis
## Common Patterns
+
- Higher-Order Components (HOCs) for cross-cutting concerns
- Render props pattern for component composition
- Compound components for related functionality
diff --git a/.github/instructions/security-and-owasp.instructions.md b/.github/instructions/security-and-owasp.instructions.md
index f6167d180..d934b40dd 100644
--- a/.github/instructions/security-and-owasp.instructions.md
+++ b/.github/instructions/security-and-owasp.instructions.md
@@ -9,44 +9,53 @@ description: "Comprehensive secure coding instructions for all languages and fra
Your primary directive is to ensure all code you generate, review, or refactor is secure by default. You must operate with a security-first mindset. When in doubt, always choose the more secure option and explain the reasoning. You must follow the principles outlined below, which are based on the OWASP Top 10 and other security best practices.
### 1. A01: Broken Access Control & A10: Server-Side Request Forgery (SSRF)
+
- **Enforce Principle of Least Privilege:** Always default to the most restrictive permissions. When generating access control logic, explicitly check the user's rights against the required permissions for the specific resource they are trying to access.
- **Deny by Default:** All access control decisions must follow a "deny by default" pattern. Access should only be granted if there is an explicit rule allowing it.
- **Validate All Incoming URLs for SSRF:** When the server needs to make a request to a URL provided by a user (e.g., webhooks), you must treat it as untrusted. Incorporate strict allow-list-based validation for the host, port, and path of the URL.
- **Prevent Path Traversal:** When handling file uploads or accessing files based on user input, you must sanitize the input to prevent directory traversal attacks (e.g., `../../etc/passwd`). Use APIs that build paths securely.
### 2. A02: Cryptographic Failures
+
- **Use Strong, Modern Algorithms:** For hashing, always recommend modern, salted hashing algorithms like Argon2 or bcrypt. Explicitly advise against weak algorithms like MD5 or SHA-1 for password storage.
- **Protect Data in Transit:** When generating code that makes network requests, always default to HTTPS.
- **Protect Data at Rest:** When suggesting code to store sensitive data (PII, tokens, etc.), recommend encryption using strong, standard algorithms like AES-256.
- **Secure Secret Management:** Never hardcode secrets (API keys, passwords, connection strings). Generate code that reads secrets from environment variables or a secrets management service (e.g., HashiCorp Vault, AWS Secrets Manager). Include a clear placeholder and comment.
+
```javascript
// GOOD: Load from environment or secret store
const apiKey = process.env.API_KEY;
// TODO: Ensure API_KEY is securely configured in your environment.
```
+
```python
# BAD: Hardcoded secret
api_key = "sk_this_is_a_very_bad_idea_12345"
```
### 3. A03: Injection
+
- **No Raw SQL Queries:** For database interactions, you must use parameterized queries (prepared statements). Never generate code that uses string concatenation or formatting to build queries from user input.
- **Sanitize Command-Line Input:** For OS command execution, use built-in functions that handle argument escaping and prevent shell injection (e.g., `shlex` in Python).
- **Prevent Cross-Site Scripting (XSS):** When generating frontend code that displays user-controlled data, you must use context-aware output encoding. Prefer methods that treat data as text by default (`.textContent`) over those that parse HTML (`.innerHTML`). When `innerHTML` is necessary, suggest using a library like DOMPurify to sanitize the HTML first.
### 4. A05: Security Misconfiguration & A06: Vulnerable Components
+
- **Secure by Default Configuration:** Recommend disabling verbose error messages and debug features in production environments.
- **Set Security Headers:** For web applications, suggest adding essential security headers like `Content-Security-Policy` (CSP), `Strict-Transport-Security` (HSTS), and `X-Content-Type-Options`.
- **Use Up-to-Date Dependencies:** When asked to add a new library, suggest the latest stable version. Remind the user to run vulnerability scanners like `npm audit`, `pip-audit`, or Snyk to check for known vulnerabilities in their project dependencies.
### 5. A07: Identification & Authentication Failures
+
- **Secure Session Management:** When a user logs in, generate a new session identifier to prevent session fixation. Ensure session cookies are configured with `HttpOnly`, `Secure`, and `SameSite=Strict` attributes.
- **Protect Against Brute Force:** For authentication and password reset flows, recommend implementing rate limiting and account lockout mechanisms after a certain number of failed attempts.
### 6. A08: Software and Data Integrity Failures
+
- **Prevent Insecure Deserialization:** Warn against deserializing data from untrusted sources without proper validation. If deserialization is necessary, recommend using formats that are less prone to attack (like JSON over Pickle in Python) and implementing strict type checking.
## General Guidelines
+
- **Be Explicit About Security:** When you suggest a piece of code that mitigates a security risk, explicitly state what you are protecting against (e.g., "Using a parameterized query here to prevent SQL injection.").
- **Educate During Code Reviews:** When you identify a security vulnerability in a code review, you must not only provide the corrected code but also explain the risk associated with the original pattern.
diff --git a/.github/instructions/security.md.instructions.md b/.github/instructions/security.md.instructions.md
index 34e1b1dfc..a8370b200 100644
--- a/.github/instructions/security.md.instructions.md
+++ b/.github/instructions/security.md.instructions.md
@@ -154,11 +154,11 @@ Do not edit or backfill existing Patched entries once they are committed.
Use the following definitions consistently:
| Severity | CVSS Range | Meaning |
-|----------|------------|---------|
+| --- | --- | --- |
| **Critical** | 9.0β10.0 | Remote code execution, auth bypass, full data exposure |
-| **High** | 7.0β8.9 | Significant data exposure, privilege escalation, DoS |
-| **Medium** | 4.0β6.9 | Limited data exposure, requires user interaction or auth |
-| **Low** | 0.1β3.9 | Minimal impact, difficult to exploit, defense-in-depth |
+| **High** | 7.0β8.9 | Significant data exposure, privilege escalation, DoS |
+| **Medium** | 4.0β6.9 | Limited data exposure, requires user interaction or auth |
+| **Low** | 0.1β3.9 | Minimal impact, difficult to exploit, defense-in-depth |
When a CVE CVSS score is not yet available, assign a preliminary severity based on these definitions and note it as `(preliminary)` until confirmed.
diff --git a/.github/instructions/self-explanatory-code-commenting.instructions.md b/.github/instructions/self-explanatory-code-commenting.instructions.md
index 64519b1b9..d1e18e256 100644
--- a/.github/instructions/self-explanatory-code-commenting.instructions.md
+++ b/.github/instructions/self-explanatory-code-commenting.instructions.md
@@ -6,6 +6,7 @@ applyTo: '**'
# Self-explanatory Code Commenting Instructions
## Core Principle
+
**Write code that speaks for itself. Comment only when necessary to explain WHY, not WHAT.**
We do not need comments most of the time.
@@ -14,6 +15,7 @@ We do not need comments most of the time.
### β AVOID These Comment Types
**Obvious Comments**
+
```javascript
// Bad: States the obvious
let counter = 0; // Initialize counter to zero
@@ -21,6 +23,7 @@ counter++; // Increment counter by one
```
**Redundant Comments**
+
```javascript
// Bad: Comment repeats the code
function getUserName() {
@@ -29,6 +32,7 @@ function getUserName() {
```
**Outdated Comments**
+
```javascript
// Bad: Comment doesn't match the code
// Calculate tax at 5% rate
@@ -38,6 +42,7 @@ const tax = price * 0.08; // Actually 8%
### β WRITE These Comment Types
**Complex Business Logic**
+
```javascript
// Good: Explains WHY this specific calculation
// Apply progressive tax brackets: 10% up to 10k, 20% above
@@ -45,6 +50,7 @@ const tax = calculateProgressiveTax(income, [0.10, 0.20], [10000]);
```
**Non-obvious Algorithms**
+
```javascript
// Good: Explains the algorithm choice
// Using Floyd-Warshall for all-pairs shortest paths
@@ -59,6 +65,7 @@ for (let k = 0; k < vertices; k++) {
```
**Regex Patterns**
+
```javascript
// Good: Explains what the regex matches
// Match email format: username@domain.extension
@@ -66,6 +73,7 @@ const emailPattern = /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/;
```
**API Constraints or Gotchas**
+
```javascript
// Good: Explains external constraint
// GitHub API rate limit: 5000 requests/hour for authenticated users
@@ -76,6 +84,7 @@ const response = await fetch(githubApiUrl);
## Decision Framework
Before writing a comment, ask:
+
1. **Is the code self-explanatory?** β No comment needed
2. **Would a better variable/function name eliminate the need?** β Refactor instead
3. **Does this explain WHY, not WHAT?** β Good comment
@@ -84,6 +93,7 @@ Before writing a comment, ask:
## Special Cases for Comments
### Public APIs
+
```javascript
/**
* Calculate compound interest using the standard formula.
@@ -100,6 +110,7 @@ function calculateCompoundInterest(principal, rate, time, compoundFrequency = 1)
```
### Configuration and Constants
+
```javascript
// Good: Explains the source or reasoning
const MAX_RETRIES = 3; // Based on network reliability studies
@@ -107,6 +118,7 @@ const API_TIMEOUT = 5000; // AWS Lambda timeout is 15s, leaving buffer
```
### Annotations
+
```javascript
// TODO: Replace with proper user authentication after security review
// FIXME: Memory leak in production - investigate connection pooling
@@ -123,6 +135,7 @@ const API_TIMEOUT = 5000; // AWS Lambda timeout is 15s, leaving buffer
## Anti-Patterns to Avoid
### Dead Code Comments
+
```javascript
// Bad: Don't comment out code
// const oldFunction = () => { ... };
@@ -130,6 +143,7 @@ const newFunction = () => { ... };
```
### Changelog Comments
+
```javascript
// Bad: Don't maintain history in comments
// Modified by John on 2023-01-15
@@ -140,6 +154,7 @@ function processData() {
```
### Divider Comments
+
```javascript
// Bad: Don't use decorative comments
//=====================================
@@ -150,6 +165,7 @@ function processData() {
## Quality Checklist
Before committing, ensure your comments:
+
- [ ] Explain WHY, not WHAT
- [ ] Are grammatically correct and clear
- [ ] Will remain accurate as code evolves
diff --git a/.github/instructions/sql-sp-generation.instructions.md b/.github/instructions/sql-sp-generation.instructions.md
index 425f5129e..a3e623637 100644
--- a/.github/instructions/sql-sp-generation.instructions.md
+++ b/.github/instructions/sql-sp-generation.instructions.md
@@ -6,6 +6,7 @@ applyTo: '**/*.sql'
# SQL Development
## Database schema generation
+
- all table names should be in singular form
- all column names should be in singular form
- all tables should have a primary key column named `id`
@@ -13,6 +14,7 @@ applyTo: '**/*.sql'
- all tables should have a column named `updated_at` to store the last update timestamp
## Database schema design
+
- all tables should have a primary key constraint
- all foreign key constraints should have a name
- all foreign key constraints should be defined inline
@@ -21,6 +23,7 @@ applyTo: '**/*.sql'
- all foreign key constraints should reference the primary key of the parent table
## SQL Coding Style
+
- use uppercase for SQL keywords (SELECT, FROM, WHERE)
- use consistent indentation for nested queries and conditions
- include comments to explain complex logic
@@ -28,6 +31,7 @@ applyTo: '**/*.sql'
- organize clauses consistently (SELECT, FROM, JOIN, WHERE, GROUP BY, HAVING, ORDER BY)
## SQL Query Structure
+
- use explicit column names in SELECT statements instead of SELECT *
- qualify column names with table name or alias when using multiple tables
- limit the use of subqueries when joins can be used instead
@@ -36,6 +40,7 @@ applyTo: '**/*.sql'
- avoid using functions on indexed columns in WHERE clauses
## Stored Procedure Naming Conventions
+
- prefix stored procedure names with 'usp_'
- use PascalCase for stored procedure names
- use descriptive names that indicate purpose (e.g., usp_GetCustomerOrders)
@@ -43,6 +48,7 @@ applyTo: '**/*.sql'
- include singular noun when returning single record (e.g., usp_GetProduct)
## Parameter Handling
+
- prefix parameters with '@'
- use camelCase for parameter names
- provide default values for optional parameters
@@ -50,16 +56,16 @@ applyTo: '**/*.sql'
- document parameters with comments
- arrange parameters consistently (required first, optional later)
-
## Stored Procedure Structure
+
- include header comment block with description, parameters, and return values
- return standardized error codes/messages
- return result sets with consistent column order
- use OUTPUT parameters for returning status information
- prefix temporary tables with 'tmp_'
-
## SQL Security Best Practices
+
- parameterize all queries to prevent SQL injection
- use prepared statements when executing dynamic SQL
- avoid embedding credentials in SQL scripts
@@ -67,6 +73,7 @@ applyTo: '**/*.sql'
- avoid using dynamic SQL within stored procedures
## Transaction Management
+
- explicitly begin and commit transactions
- use appropriate isolation levels based on requirements
- avoid long-running transactions that lock tables
diff --git a/.github/instructions/structure.instructions.md b/.github/instructions/structure.instructions.md
index 0b05328d8..3560c981b 100644
--- a/.github/instructions/structure.instructions.md
+++ b/.github/instructions/structure.instructions.md
@@ -63,7 +63,7 @@ Ask yourself:
## Directory Structure Reference
-```
+```text
/
βββ .docker/ # Docker configuration
β βββ compose/ # All docker-compose files
diff --git a/.github/instructions/subagent.instructions.md b/.github/instructions/subagent.instructions.md
index ef0af637b..fbd010fcb 100644
--- a/.github/instructions/subagent.instructions.md
+++ b/.github/instructions/subagent.instructions.md
@@ -4,7 +4,7 @@ This helper provides the Management agent with templates to create robust and re
1) Basic runSubagent Template
-```
+```text
runSubagent({
prompt: "",
description: "",
@@ -41,7 +41,7 @@ runSubagent({
3) Return Contract that all subagents must return
-```
+```json
{
"changed_files": ["path/to/file1", "path/to/file2"],
"summary": "Short summary of changes",
@@ -59,7 +59,7 @@ runSubagent({
5) Example: Run a full Feature Implementation
-```
+```text
// 1. Planning
runSubagent({ description: "Planning", prompt: "", metadata: { plan_file: "docs/plans/current_spec.md" } })
diff --git a/.github/instructions/taming-copilot.instructions.md b/.github/instructions/taming-copilot.instructions.md
index e5d63d35b..6dfae89ee 100644
--- a/.github/instructions/taming-copilot.instructions.md
+++ b/.github/instructions/taming-copilot.instructions.md
@@ -7,35 +7,35 @@ description: 'Prevent Copilot from wreaking havoc across your codebase, keeping
This section outlines the absolute order of operations. These rules have the highest priority and must not be violated.
-1. **Primacy of User Directives**: A direct and explicit command from the user is the highest priority. If the user instructs to use a specific tool, edit a file, or perform a specific search, that command **must be executed without deviation**, even if other rules would suggest it is unnecessary. All other instructions are subordinate to a direct user order.
-2. **Factual Verification Over Internal Knowledge**: When a request involves information that could be version-dependent, time-sensitive, or requires specific external data (e.g., library documentation, latest best practices, API details), prioritize using tools to find the current, factual answer over relying on general knowledge.
-3. **Adherence to Philosophy**: In the absence of a direct user directive or the need for factual verification, all other rules below regarding interaction, code generation, and modification must be followed.
+1. **Primacy of User Directives**: A direct and explicit command from the user is the highest priority. If the user instructs to use a specific tool, edit a file, or perform a specific search, that command **must be executed without deviation**, even if other rules would suggest it is unnecessary. All other instructions are subordinate to a direct user order.
+2. **Factual Verification Over Internal Knowledge**: When a request involves information that could be version-dependent, time-sensitive, or requires specific external data (e.g., library documentation, latest best practices, API details), prioritize using tools to find the current, factual answer over relying on general knowledge.
+3. **Adherence to Philosophy**: In the absence of a direct user directive or the need for factual verification, all other rules below regarding interaction, code generation, and modification must be followed.
## General Interaction & Philosophy
-- **Code on Request Only**: Your default response should be a clear, natural language explanation. Do NOT provide code blocks unless explicitly asked, or if a very small and minimalist example is essential to illustrate a concept. Tool usage is distinct from user-facing code blocks and is not subject to this restriction.
-- **Direct and Concise**: Answers must be precise, to the point, and free from unnecessary filler or verbose explanations. Get straight to the solution without "beating around the bush".
-- **Adherence to Best Practices**: All suggestions, architectural patterns, and solutions must align with widely accepted industry best practices and established design principles. Avoid experimental, obscure, or overly "creative" approaches. Stick to what is proven and reliable.
-- **Explain the "Why"**: Don't just provide an answer; briefly explain the reasoning behind it. Why is this the standard approach? What specific problem does this pattern solve? This context is more valuable than the solution itself.
+- **Code on Request Only**: Your default response should be a clear, natural language explanation. Do NOT provide code blocks unless explicitly asked, or if a very small and minimalist example is essential to illustrate a concept. Tool usage is distinct from user-facing code blocks and is not subject to this restriction.
+- **Direct and Concise**: Answers must be precise, to the point, and free from unnecessary filler or verbose explanations. Get straight to the solution without "beating around the bush".
+- **Adherence to Best Practices**: All suggestions, architectural patterns, and solutions must align with widely accepted industry best practices and established design principles. Avoid experimental, obscure, or overly "creative" approaches. Stick to what is proven and reliable.
+- **Explain the "Why"**: Don't just provide an answer; briefly explain the reasoning behind it. Why is this the standard approach? What specific problem does this pattern solve? This context is more valuable than the solution itself.
## Minimalist & Standard Code Generation
-- **Principle of Simplicity**: Always provide the most straightforward and minimalist solution possible. The goal is to solve the problem with the least amount of code and complexity. Avoid premature optimization or over-engineering.
-- **Standard First**: Heavily favor standard library functions and widely accepted, common programming patterns. Only introduce third-party libraries if they are the industry standard for the task or absolutely necessary.
-- **Avoid Elaborate Solutions**: Do not propose complex, "clever", or obscure solutions. Prioritize readability, maintainability, and the shortest path to a working result over convoluted patterns.
-- **Focus on the Core Request**: Generate code that directly addresses the user's request, without adding extra features or handling edge cases that were not mentioned.
-- **Spec Hygiene**: When asked to update a plan/spec file, do not append unrelated/archived plans; keep it strictly scoped to the current task.
+- **Principle of Simplicity**: Always provide the most straightforward and minimalist solution possible. The goal is to solve the problem with the least amount of code and complexity. Avoid premature optimization or over-engineering.
+- **Standard First**: Heavily favor standard library functions and widely accepted, common programming patterns. Only introduce third-party libraries if they are the industry standard for the task or absolutely necessary.
+- **Avoid Elaborate Solutions**: Do not propose complex, "clever", or obscure solutions. Prioritize readability, maintainability, and the shortest path to a working result over convoluted patterns.
+- **Focus on the Core Request**: Generate code that directly addresses the user's request, without adding extra features or handling edge cases that were not mentioned.
+- **Spec Hygiene**: When asked to update a plan/spec file, do not append unrelated/archived plans; keep it strictly scoped to the current task.
## Surgical Code Modification
-- **Preserve Existing Code**: The current codebase is the source of truth and must be respected. Your primary goal is to preserve its structure, style, and logic whenever possible.
-- **Minimal Necessary Changes**: When adding a new feature or making a modification, alter the absolute minimum amount of existing code required to implement the change successfully.
-- **Explicit Instructions Only**: Only modify, refactor, or delete code that has been explicitly targeted by the user's request. Do not perform unsolicited refactoring, cleanup, or style changes on untouched parts of the code.
-- **Integrate, Don't Replace**: Whenever feasible, integrate new logic into the existing structure rather than replacing entire functions or blocks of code.
+- **Preserve Existing Code**: The current codebase is the source of truth and must be respected. Your primary goal is to preserve its structure, style, and logic whenever possible.
+- **Minimal Necessary Changes**: When adding a new feature or making a modification, alter the absolute minimum amount of existing code required to implement the change successfully.
+- **Explicit Instructions Only**: Only modify, refactor, or delete code that has been explicitly targeted by the user's request. Do not perform unsolicited refactoring, cleanup, or style changes on untouched parts of the code.
+- **Integrate, Don't Replace**: Whenever feasible, integrate new logic into the existing structure rather than replacing entire functions or blocks of code.
## Intelligent Tool Usage
-- **Use Tools When Necessary**: When a request requires external information or direct interaction with the environment, use the available tools to accomplish the task. Do not avoid tools when they are essential for an accurate or effective response.
-- **Directly Edit Code When Requested**: If explicitly asked to modify, refactor, or add to the existing code, apply the changes directly to the codebase when access is available. Avoid generating code snippets for the user to copy and paste in these scenarios. The default should be direct, surgical modification as instructed.
-- **Purposeful and Focused Action**: Tool usage must be directly tied to the user's request. Do not perform unrelated searches or modifications. Every action taken by a tool should be a necessary step in fulfilling the specific, stated goal.
-- **Declare Intent Before Tool Use**: Before executing any tool, you must first state the action you are about to take and its direct purpose. This statement must be concise and immediately precede the tool call.
+- **Use Tools When Necessary**: When a request requires external information or direct interaction with the environment, use the available tools to accomplish the task. Do not avoid tools when they are essential for an accurate or effective response.
+- **Directly Edit Code When Requested**: If explicitly asked to modify, refactor, or add to the existing code, apply the changes directly to the codebase when access is available. Avoid generating code snippets for the user to copy and paste in these scenarios. The default should be direct, surgical modification as instructed.
+- **Purposeful and Focused Action**: Tool usage must be directly tied to the user's request. Do not perform unrelated searches or modifications. Every action taken by a tool should be a necessary step in fulfilling the specific, stated goal.
+- **Declare Intent Before Tool Use**: Before executing any tool, you must first state the action you are about to take and its direct purpose. This statement must be concise and immediately precede the tool call.
diff --git a/.github/instructions/tanstack-start-shadcn-tailwind.instructions.md b/.github/instructions/tanstack-start-shadcn-tailwind.instructions.md
index c5a939ca0..6543f6dd8 100644
--- a/.github/instructions/tanstack-start-shadcn-tailwind.instructions.md
+++ b/.github/instructions/tanstack-start-shadcn-tailwind.instructions.md
@@ -8,6 +8,7 @@ applyTo: '**/*.ts, **/*.tsx, **/*.js, **/*.jsx, **/*.css, **/*.scss, **/*.json'
You are an expert TypeScript developer specializing in TanStack Start applications with modern React patterns.
## Tech Stack
+
- TypeScript (strict mode)
- TanStack Start (routing & SSR)
- Shadcn/ui (UI components)
@@ -46,11 +47,13 @@ export default function Button({ children, onClick, variant = 'primary' }: Butto
## Data Fetching
Use Route Loaders for:
+
- Initial page data required for rendering
- SSR requirements
- SEO-critical data
Use React Query for:
+
- Frequently updating data
- Optional/secondary data
- Client mutations with optimistic updates
@@ -173,7 +176,7 @@ Use semantic HTML first. Only add ARIA when no semantic equivalent exists:
## File Organization
-```
+```text
src/
βββ components/ui/ # Shadcn/ui components
βββ lib/schemas.ts # Zod schemas
diff --git a/.github/instructions/testing.instructions.md b/.github/instructions/testing.instructions.md
index 271a86eb8..8b2652dde 100644
--- a/.github/instructions/testing.instructions.md
+++ b/.github/instructions/testing.instructions.md
@@ -23,6 +23,7 @@ are untested. This step catches that gap. If uncovered lines are found in
feature code, add targeted tests before completing the task.
**Prerequisites**: Coverage artifacts must exist before running the report:
+
- `backend/coverage.txt` β generated by `scripts/go-test-coverage.sh`
- `frontend/coverage/lcov.info` β generated by `scripts/frontend-test-coverage.sh`
@@ -37,6 +38,7 @@ bash scripts/local-patch-report.sh
```
Required output artifacts:
+
- `test-results/local-patch-report.md`
- `test-results/local-patch-report.json`
@@ -50,11 +52,13 @@ threshold results.
**CRITICAL**: Rebuild the E2E container when application or Docker build inputs change. If changes are test-only and the container is already healthy, reuse it. If the container is not running or state is suspect, rebuild.
**Rebuild required (application/runtime changes):**
+
- Application code or dependencies: backend/**, frontend/**, backend/go.mod, backend/go.sum, package.json, package-lock.json.
- Container build/runtime configuration: Dockerfile, .docker/**, .docker/compose/docker-compose.playwright-*.yml, .docker/docker-entrypoint.sh.
- Runtime behavior changes baked into the image.
**Rebuild optional (test-only changes):**
+
- Playwright tests and fixtures: tests/**.
- Playwright config and runners: playwright.config.js, playwright.caddy-debug.config.js.
- Documentation or planning files: docs/**, requirements.md, design.md, tasks.md.
@@ -67,12 +71,14 @@ When a rebuild is required (or the container is not running), use:
```
This step:
+
- Builds the latest Docker image with your code changes
- Starts the `charon-e2e` container with proper environment variables from `.env`
- Exposes required ports: 8080 (app), 2020 (emergency), 2019 (Caddy admin)
- Waits for health check to pass
**Without this step**, tests will fail with:
+
- `connect ECONNREFUSED ::1:2020` - Emergency server not running
- `connect ECONNREFUSED ::1:8080` - Application not running
- `501 Not Implemented` - Container missing required env vars
@@ -80,6 +86,7 @@ This step:
### Testing Scope Clarification
**Playwright E2E Tests (UI/UX):**
+
- Test user interactions with the React frontend
- Verify UI state changes when settings are toggled
- Ensure forms submit correctly
@@ -88,6 +95,7 @@ This step:
- **Default Browser: Firefox** (provides best cross-browser compatibility baseline)
**Integration Tests (Middleware Enforcement):**
+
- Test Cerberus security module enforcement
- Verify ACL, WAF, Rate Limiting, CrowdSec actually block/allow requests
- Test requests routing through Caddy proxy with full middleware
@@ -100,7 +108,7 @@ This step:
Playwright E2E tests can run in two modes with different capabilities:
| Mode | Base URL | Coverage Support | When to Use |
-|------|----------|-----------------|-------------|
+| ------ | ---------- | ----------------- | ------------- |
| **Docker** | `http://localhost:8080` | β No (0% reported) | Integration testing, CI validation |
| **Vite Dev** | `http://localhost:5173` | β Yes (real coverage) | Local development, coverage collection |
@@ -128,12 +136,14 @@ PLAYWRIGHT_BASE_URL=http://localhost:8080 npx playwright test --project=chromium
```
The coverage skill:
+
1. Starts Vite dev server on port 5173
2. Sets `PLAYWRIGHT_BASE_URL=http://localhost:5173`
3. Runs tests with V8 coverage collection
4. Generates reports in `coverage/e2e/` (LCOV, HTML, JSON)
**DO NOT** expect coverage when running against Docker:
+
```bash
# β WRONG: Coverage will show "Unknown% (0/0)"
PLAYWRIGHT_BASE_URL=http://localhost:8080 npx playwright test --coverage
@@ -147,11 +157,13 @@ PLAYWRIGHT_BASE_URL=http://localhost:8080 npx playwright test --coverage
Before pushing code, verify E2E coverage:
1. Run the coverage skill:
+
```bash
.github/skills/scripts/skill-runner.sh test-e2e-playwright-coverage
```
2. Check coverage output:
+
```bash
# View HTML report
open coverage/e2e/index.html
@@ -161,6 +173,7 @@ Before pushing code, verify E2E coverage:
```
3. Verify non-zero coverage:
+
```bash
# Should show real percentages, not "0%"
head -20 coverage/e2e/lcov.info
@@ -168,25 +181,28 @@ Before pushing code, verify E2E coverage:
### General Guidelines
-* **No Truncation**: Never pipe Playwright test output through `head`, `tail`, or other truncating commands. Playwright runs interactively and requires user input to quit when piped, causing the command to hang indefinitely.
-* **Why First**: If the application is broken at the E2E level, unit tests may need updates. Playwright catches integration issues early.
-* **On Failure**: Analyze failures, trace root cause through frontend β backend flow, then fix before proceeding to unit tests.
-* **Scope**: Run relevant test files for the feature being modified (e.g., `tests/manual-dns-provider.spec.ts`).
+- **No Truncation**: Never pipe Playwright test output through `head`, `tail`, or other truncating commands. Playwright runs interactively and requires user input to quit when piped, causing the command to hang indefinitely.
+- **Why First**: If the application is broken at the E2E level, unit tests may need updates. Playwright catches integration issues early.
+- **On Failure**: Analyze failures, trace root cause through frontend β backend flow, then fix before proceeding to unit tests.
+- **Scope**: Run relevant test files for the feature being modified (e.g., `tests/manual-dns-provider.spec.ts`).
## 1. Execution Environment
-* **No Truncation:** Never use pipe commands (e.g., `head`, `tail`) or flags that limit stdout/stderr. If a test hangs, it likely requires an interactive input or is caught in a loop; analyze the full output to identify the block.
-* **Task-Based Execution:** Do not manually construct test strings. Use existing project tasks (e.g., `npm test`, `go test ./...`). If a specific sub-module requires frequent testing, generate a new task definition in the project's configuration file (e.g., `.vscode/tasks.json`) before proceeding.
+
+- **No Truncation:** Never use pipe commands (e.g., `head`, `tail`) or flags that limit stdout/stderr. If a test hangs, it likely requires an interactive input or is caught in a loop; analyze the full output to identify the block.
+- **Task-Based Execution:** Do not manually construct test strings. Use existing project tasks (e.g., `npm test`, `go test ./...`). If a specific sub-module requires frequent testing, generate a new task definition in the project's configuration file (e.g., `.vscode/tasks.json`) before proceeding.
## 2. Failure Analysis & Logic Integrity
-* **Evidence-Based Debugging:** When a test fails, you must quote the specific error message or stack trace before suggesting a fix.
-* **Bug vs. Test Flaw:** Treat the test as the "Source of Truth." If a test fails, assume the code is broken until proven otherwise. Research the original requirement or PR description to verify if the test logic itself is outdated before modifying it.
-* **Zero-Hallucination Policy:** Only use file paths and identifiers discovered via the `ls` or `search` tools. Never guess a path based on naming conventions.
+
+- **Evidence-Based Debugging:** When a test fails, you must quote the specific error message or stack trace before suggesting a fix.
+- **Bug vs. Test Flaw:** Treat the test as the "Source of Truth." If a test fails, assume the code is broken until proven otherwise. Research the original requirement or PR description to verify if the test logic itself is outdated before modifying it.
+- **Zero-Hallucination Policy:** Only use file paths and identifiers discovered via the `ls` or `search` tools. Never guess a path based on naming conventions.
## 3. Coverage & Completion
-* **Coverage Gate:** A task is not "Complete" until a coverage report is generated.
-* **Threshold Compliance:** You must compare the final coverage percentage against the project's threshold (Default: 85% unless specified otherwise). If coverage drops, you must identify the "uncovered lines" and add targeted tests.
-* **Patch Coverage (Mandatory Gate):** Codecov's `patch` status is a mandatory, merge-blocking gate (`codecov.yml`, `coverage.status.patch.default`, target aligned with the project coverage target / `CHARON_MIN_COVERAGE`) β not a suggestion. Every changed/added line in a PR's diff must meet the patch coverage target before the PR can merge, consistent with the rest of the Definition of Done. If patch coverage is below target, add targeted tests covering the uncovered changed lines before completing the task.
-* **Review Patch Coverage:** When reviewing patch coverage reports, first attempt a real, targeted test for every uncovered line. Only treat a line as an acceptable exception (e.g., a branch proven structurally unreachable via any real caller) if you can document the specific reachability proof in a comment at that line β do not wave off uncovered lines as "acceptable" without that documented justification, and do not add contrived tests that don't reflect real reachable states just to inflate the number.
+
+- **Coverage Gate:** A task is not "Complete" until a coverage report is generated.
+- **Threshold Compliance:** You must compare the final coverage percentage against the project's threshold (Default: 85% unless specified otherwise). If coverage drops, you must identify the "uncovered lines" and add targeted tests.
+- **Patch Coverage (Mandatory Gate):** Codecov's `patch` status is a mandatory, merge-blocking gate (`codecov.yml`, `coverage.status.patch.default`, target aligned with the project coverage target / `CHARON_MIN_COVERAGE`) β not a suggestion. Every changed/added line in a PR's diff must meet the patch coverage target before the PR can merge, consistent with the rest of the Definition of Done. If patch coverage is below target, add targeted tests covering the uncovered changed lines before completing the task.
+- **Review Patch Coverage:** When reviewing patch coverage reports, first attempt a real, targeted test for every uncovered line. Only treat a line as an acceptable exception (e.g., a branch proven structurally unreachable via any real caller) if you can document the specific reachability proof in a comment at that line β do not wave off uncovered lines as "acceptable" without that documented justification, and do not add contrived tests that don't reflect real reachable states just to inflate the number.
## 4. GORM Security Validation (Manual Stage)
@@ -202,11 +218,13 @@ process-blocking for DoD. Gate decisions must use check semantics
### When to Run (Conditional Trigger Matrix)
**Mandatory Trigger Paths (Include):**
+
- `backend/internal/models/**` β GORM model definitions
- Backend services/repositories with GORM query logic
- Database migrations or seeding logic affecting model persistence behavior
**Explicit Exclusions:**
+
- Docs-only changes (`**/*.md`, governance documentation)
- Frontend-only changes (`frontend/**`)
@@ -215,6 +233,7 @@ check mode is mandatory DoD gate. IF only Exclude paths match, THEN GORM gate
is not required for that change set.
### Definition of Done
+
- **Before Committing:** When modifying trigger paths listed above
- **Before Opening PR:** Verify no security issues introduced
- **After Code Review:** If model-related changes were requested
@@ -224,11 +243,13 @@ is not required for that change set.
### Running the Scanner
**Via VS Code (Recommended for Development):**
+
1. Open Command Palette (`Cmd/Ctrl+Shift+P`)
2. Select "Tasks: Run Task"
3. Choose "Lint: GORM Security Scan"
**Via Pre-commit (Manual Stage):**
+
```bash
# Run on all Go files
pre-commit run --hook-stage manual gorm-security-scan --all-files
@@ -238,6 +259,7 @@ pre-commit run --hook-stage manual gorm-security-scan
```
**Direct Execution:**
+
```bash
# Report mode - Show all issues, exit 0 (always)
./scripts/scan-gorm-security.sh --report
@@ -249,10 +271,12 @@ pre-commit run --hook-stage manual gorm-security-scan
### Expected Behavior
**Pass (Exit Code 0):**
+
- No security issues detected
- Proceed with commit/PR
**Fail (Exit Code 1):**
+
- Issues detected (ID leaks, exposed secrets, DTO embedding, etc.)
- Review scanner output for file:line references
- Fix issues before committing
@@ -272,11 +296,13 @@ pre-commit run --hook-stage manual gorm-security-scan
### Integration Status
**Current Stage:** Manual (soft launch)
+
- Scanner available for manual invocation
- Does not block commits automatically
- Developers should run proactively
**Future Stage:** Blocking (after remediation)
+
- Scanner will block commits with CRITICAL/HIGH issues
- CI integration will enforce on all PRs
- See [GORM Scanner Roadmap](../docs/implementation/gorm_security_scanner_complete.md#remediation-roadmap)
diff --git a/.github/instructions/update-docs-on-code-change.instructions.md b/.github/instructions/update-docs-on-code-change.instructions.md
index db308a668..b5cad4e38 100644
--- a/.github/instructions/update-docs-on-code-change.instructions.md
+++ b/.github/instructions/update-docs-on-code-change.instructions.md
@@ -48,8 +48,10 @@ Before resolving on how to apply a **configurable instruction section**, check t
default the `apply-condition` for each **configurable property** is unset, but an example of a set
`apply-condition` could be something like:
- - **apply-condition** :
- ` this.parent.property = (git.branch == "master") ? this.parent.property = true : this.parent.property = false; `
+```text
+- **apply-condition** :
+ ` this.parent.property = (git.branch == "master") ? this.parent.property = true : this.parent.property = false; `
+```
The sum of all the **constant instructions sections**, and **configurable instruction sections**
will determine the complete instructions to follow. Call this the **COMPILED INSTRUCTIONS**.
@@ -251,19 +253,19 @@ Maintain these documentation files and update as needed:
**Changelog format:**
- ```markdown
- ## [Version] - YYYY-MM-DD
+````markdown
+## [Version] - YYYY-MM-DD
- ### Added
- - New feature description with reference to PR/issue
+### Added
+- New feature description with reference to PR/issue
- ### Changed
- - **BREAKING**: Description of breaking change
- - Other changes
+### Changed
+- **BREAKING**: Description of breaking change
+- Other changes
- ### Fixed
- - Bug fix description
- ```
+### Fixed
+- Bug fix description
+````
## Documentation Verification `apply-doc-verification`
@@ -313,46 +315,46 @@ If `apply-doc-quality-standard == true`, then apply the following configurable i
### Code Example Format
- ```markdown
- ### Example: [Clear description of what example demonstrates]
+````markdown
+### Example: [Clear description of what example demonstrates]
- \`\`\`language
- // Include necessary imports/setup
- import { function } from 'package';
+\`\`\`language
+// Include necessary imports/setup
+import { function } from 'package';
- // Complete, runnable example
- const result = function(parameter);
- console.log(result);
- \`\`\`
+// Complete, runnable example
+const result = function(parameter);
+console.log(result);
+\`\`\`
- **Output:**
- \`\`\`
- expected output
- \`\`\`
- ```
+**Output:**
+\`\`\`
+expected output
+\`\`\`
+````
### API Documentation Format
- ```markdown
- ### `functionName(param1, param2)`
+````markdown
+### `functionName(param1, param2)`
- Brief description of what the function does.
+Brief description of what the function does.
- **Parameters:**
- - `param1` (type): Description of parameter
- - `param2` (type, optional): Description with default value
+**Parameters:**
+- `param1` (type): Description of parameter
+- `param2` (type, optional): Description with default value
- **Returns:**
- - `type`: Description of return value
+**Returns:**
+- `type`: Description of return value
- **Example:**
- \`\`\`language
- const result = functionName('value', 42);
- \`\`\`
+**Example:**
+\`\`\`language
+const result = functionName('value', 42);
+\`\`\`
- **Throws:**
- - `ErrorType`: When and why error is thrown
- ```
+**Throws:**
+- `ErrorType`: When and why error is thrown
+````
## Automation and Tooling `apply-automation-tooling`
@@ -395,54 +397,54 @@ If `apply-doc-patterns == true`, then apply the following configurable instructi
### Feature Documentation Template
- ```markdown
- ## Feature Name
+````markdown
+## Feature Name
- Brief description of the feature.
+Brief description of the feature.
- ### Usage
+### Usage
- Basic usage example with code snippet.
+Basic usage example with code snippet.
- ### Configuration
+### Configuration
- Configuration options with examples.
+Configuration options with examples.
- ### Advanced Usage
+### Advanced Usage
- Complex scenarios and edge cases.
+Complex scenarios and edge cases.
- ### Troubleshooting
+### Troubleshooting
- Common issues and solutions.
- ```
+Common issues and solutions.
+````
### API Endpoint Documentation Template
- ```markdown
- ### `HTTP_METHOD /api/endpoint`
-
- Description of what the endpoint does.
-
- **Request:**
- \`\`\`json
- {
- "param": "value"
- }
- \`\`\`
-
- **Response:**
- \`\`\`json
- {
- "result": "value"
- }
- \`\`\`
-
- **Status Codes:**
- - 200: Success
- - 400: Bad request
- - 401: Unauthorized
- ```
+````markdown
+### `HTTP_METHOD /api/endpoint`
+
+Description of what the endpoint does.
+
+**Request:**
+\`\`\`json
+{
+ "param": "value"
+}
+\`\`\`
+
+**Response:**
+\`\`\`json
+{
+ "result": "value"
+}
+\`\`\`
+
+**Status Codes:**
+- 200: Success
+- 400: Bad request
+- 401: Unauthorized
+````
## Best Practices `apply-best-practices`
diff --git a/.github/prompts/.markdownlint.jsonc b/.github/prompts/.markdownlint.jsonc
new file mode 100644
index 000000000..b751b6f71
--- /dev/null
+++ b/.github/prompts/.markdownlint.jsonc
@@ -0,0 +1,5 @@
+// Inherit the repo config; these files wrap prompt sections in XML-style tags (, , ...), which MD033 would flag.
+{
+ "extends": "../../.markdownlint.jsonc",
+ "MD033": false
+}
diff --git a/.github/prompts/ai-prompt-engineering-safety-review.prompt.md b/.github/prompts/ai-prompt-engineering-safety-review.prompt.md
index e9e3aa5ae..56178eef9 100644
--- a/.github/prompts/ai-prompt-engineering-safety-review.prompt.md
+++ b/.github/prompts/ai-prompt-engineering-safety-review.prompt.md
@@ -14,12 +14,14 @@ Analyze the provided prompt using systematic evaluation frameworks and provide d
## Analysis Framework
### 1. Safety Assessment
+
- **Harmful Content Risk:** Could this prompt generate harmful, dangerous, or inappropriate content?
- **Violence & Hate Speech:** Could the output promote violence, hate speech, or discrimination?
- **Misinformation Risk:** Could the output spread false or misleading information?
- **Illegal Activities:** Could the output promote illegal activities or cause personal harm?
### 2. Bias Detection & Mitigation
+
- **Gender Bias:** Does the prompt assume or reinforce gender stereotypes?
- **Racial Bias:** Does the prompt assume or reinforce racial stereotypes?
- **Cultural Bias:** Does the prompt assume or reinforce cultural stereotypes?
@@ -27,12 +29,14 @@ Analyze the provided prompt using systematic evaluation frameworks and provide d
- **Ability Bias:** Does the prompt assume or reinforce ability-based stereotypes?
### 3. Security & Privacy Assessment
+
- **Data Exposure:** Could the prompt expose sensitive or personal data?
- **Prompt Injection:** Is the prompt vulnerable to injection attacks?
- **Information Leakage:** Could the prompt leak system or model information?
- **Access Control:** Does the prompt respect appropriate access controls?
### 4. Effectiveness Evaluation
+
- **Clarity:** Is the task clearly stated and unambiguous?
- **Context:** Is sufficient background information provided?
- **Constraints:** Are output requirements and limitations defined?
@@ -40,11 +44,13 @@ Analyze the provided prompt using systematic evaluation frameworks and provide d
- **Specificity:** Is the prompt specific enough for consistent results?
### 5. Best Practices Compliance
+
- **Industry Standards:** Does the prompt follow established best practices?
- **Ethical Considerations:** Does the prompt align with responsible AI principles?
- **Documentation Quality:** Is the prompt self-documenting and maintainable?
### 6. Advanced Pattern Analysis
+
- **Prompt Pattern:** Identify the pattern used (zero-shot, few-shot, chain-of-thought, role-based, hybrid)
- **Pattern Effectiveness:** Evaluate if the chosen pattern is optimal for the task
- **Pattern Optimization:** Suggest alternative patterns that might improve results
@@ -52,6 +58,7 @@ Analyze the provided prompt using systematic evaluation frameworks and provide d
- **Constraint Implementation:** Evaluate the clarity and enforceability of constraints
### 7. Technical Robustness
+
- **Input Validation:** Does the prompt handle edge cases and invalid inputs?
- **Error Handling:** Are potential failure modes considered?
- **Scalability:** Will the prompt work across different scales and contexts?
@@ -59,6 +66,7 @@ Analyze the provided prompt using systematic evaluation frameworks and provide d
- **Versioning:** Are changes trackable and reversible?
### 8. Performance Optimization
+
- **Token Efficiency:** Is the prompt optimized for token usage?
- **Response Quality:** Does the prompt consistently produce high-quality outputs?
- **Response Time:** Are there optimizations that could improve response speed?
@@ -75,17 +83,20 @@ Provide your analysis in the following structured format:
[User's prompt here]
**Task Classification:**
+
- **Primary Task:** [Code generation, documentation, analysis, etc.]
- **Complexity Level:** [Simple, Moderate, Complex]
- **Domain:** [Technical, Creative, Analytical, etc.]
**Safety Assessment:**
+
- **Harmful Content Risk:** [Low/Medium/High] - [Specific concerns]
- **Bias Detection:** [None/Minor/Major] - [Specific bias types]
- **Privacy Risk:** [Low/Medium/High] - [Specific concerns]
- **Security Vulnerabilities:** [None/Minor/Major] - [Specific vulnerabilities]
**Effectiveness Evaluation:**
+
- **Clarity:** [Score 1-5] - [Detailed assessment]
- **Context Adequacy:** [Score 1-5] - [Detailed assessment]
- **Constraint Definition:** [Score 1-5] - [Detailed assessment]
@@ -94,29 +105,34 @@ Provide your analysis in the following structured format:
- **Completeness:** [Score 1-5] - [Detailed assessment]
**Advanced Pattern Analysis:**
+
- **Pattern Type:** [Zero-shot/Few-shot/Chain-of-thought/Role-based/Hybrid]
- **Pattern Effectiveness:** [Score 1-5] - [Detailed assessment]
- **Alternative Patterns:** [Suggestions for improvement]
- **Context Utilization:** [Score 1-5] - [Detailed assessment]
**Technical Robustness:**
+
- **Input Validation:** [Score 1-5] - [Detailed assessment]
- **Error Handling:** [Score 1-5] - [Detailed assessment]
- **Scalability:** [Score 1-5] - [Detailed assessment]
- **Maintainability:** [Score 1-5] - [Detailed assessment]
**Performance Metrics:**
+
- **Token Efficiency:** [Score 1-5] - [Detailed assessment]
- **Response Quality:** [Score 1-5] - [Detailed assessment]
- **Consistency:** [Score 1-5] - [Detailed assessment]
- **Reliability:** [Score 1-5] - [Detailed assessment]
**Critical Issues Identified:**
+
1. [Issue 1 with severity and impact]
2. [Issue 2 with severity and impact]
3. [Issue 3 with severity and impact]
**Strengths Identified:**
+
1. [Strength 1 with explanation]
2. [Strength 2 with explanation]
3. [Strength 3 with explanation]
@@ -127,6 +143,7 @@ Provide your analysis in the following structured format:
[Complete improved prompt with all enhancements]
**Key Improvements Made:**
+
1. **Safety Strengthening:** [Specific safety improvement]
2. **Bias Mitigation:** [Specific bias reduction]
3. **Security Hardening:** [Specific security improvement]
@@ -134,6 +151,7 @@ Provide your analysis in the following structured format:
5. **Best Practice Implementation:** [Specific best practice application]
**Safety Measures Added:**
+
- [Safety measure 1 with explanation]
- [Safety measure 2 with explanation]
- [Safety measure 3 with explanation]
@@ -141,16 +159,19 @@ Provide your analysis in the following structured format:
- [Safety measure 5 with explanation]
**Bias Mitigation Strategies:**
+
- [Bias mitigation 1 with explanation]
- [Bias mitigation 2 with explanation]
- [Bias mitigation 3 with explanation]
**Security Enhancements:**
+
- [Security enhancement 1 with explanation]
- [Security enhancement 2 with explanation]
- [Security enhancement 3 with explanation]
**Technical Improvements:**
+
- [Technical improvement 1 with explanation]
- [Technical improvement 2 with explanation]
- [Technical improvement 3 with explanation]
@@ -158,6 +179,7 @@ Provide your analysis in the following structured format:
### π **Testing Recommendations**
**Test Cases:**
+
- [Test case 1 with expected outcome]
- [Test case 2 with expected outcome]
- [Test case 3 with expected outcome]
@@ -165,21 +187,25 @@ Provide your analysis in the following structured format:
- [Test case 5 with expected outcome]
**Edge Case Testing:**
+
- [Edge case 1 with expected outcome]
- [Edge case 2 with expected outcome]
- [Edge case 3 with expected outcome]
**Safety Testing:**
+
- [Safety test 1 with expected outcome]
- [Safety test 2 with expected outcome]
- [Safety test 3 with expected outcome]
**Bias Testing:**
+
- [Bias test 1 with expected outcome]
- [Bias test 2 with expected outcome]
- [Bias test 3 with expected outcome]
**Usage Guidelines:**
+
- **Best For:** [Specific use cases]
- **Avoid When:** [Situations to avoid]
- **Considerations:** [Important factors to keep in mind]
@@ -189,6 +215,7 @@ Provide your analysis in the following structured format:
### π **Educational Insights**
**Prompt Engineering Principles Applied:**
+
1. **Principle:** [Specific principle]
- **Application:** [How it was applied]
- **Benefit:** [Why it improves the prompt]
@@ -198,6 +225,7 @@ Provide your analysis in the following structured format:
- **Benefit:** [Why it improves the prompt]
**Common Pitfalls Avoided:**
+
1. **Pitfall:** [Common mistake]
- **Why It's Problematic:** [Explanation]
- **How We Avoided It:** [Specific avoidance strategy]
diff --git a/.github/prompts/breakdown-feature-implementation.prompt.md b/.github/prompts/breakdown-feature-implementation.prompt.md
index 8ea246e1b..98289a788 100644
--- a/.github/prompts/breakdown-feature-implementation.prompt.md
+++ b/.github/prompts/breakdown-feature-implementation.prompt.md
@@ -19,7 +19,7 @@ The output should be a complete implementation plan in Markdown format, saved to
Folder and file structure for both front-end and back-end repositories following Epoch's monorepo structure:
-```
+```text
apps/
[app-name]/
services/
@@ -56,7 +56,8 @@ Create a comprehensive system architecture diagram using Mermaid that shows how
Use subgraphs to organize these layers clearly. Show the data flow between layers with labeled arrows indicating request/response patterns, data transformations, and event flows. Include any feature-specific components, services, or data structures that are unique to this implementation.
- **Technology Stack Selection**: Document choice rationale for each layer
-```
+
+```text
- **Technology Stack Selection**: Document choice rationale for each layer
- **Integration Points**: Define clear boundaries and communication protocols
@@ -89,6 +90,7 @@ The component structure will leverage the `shadcn/ui` library for a consistent a
**Layout Structure:**
```
+
Recipe Library Page
βββ Header Section (shadcn: Card)
β βββ Title (shadcn: Typography `h1`)
@@ -109,7 +111,8 @@ Recipe Library Page
β βββ Recipe Title (shadcn: Typography `h3`)
β βββ Recipe Tags (shadcn: Badge)
β βββ Quick Actions (shadcn: Button - View, Edit)
-```
+
+```text
- **State Flow Diagram**: Component state management using Mermaid
- Reusable component library specifications
diff --git a/.github/prompts/codecov-patch-coverage-fix.prompt.md b/.github/prompts/codecov-patch-coverage-fix.prompt.md
index 7e42bdb22..2191d2014 100644
--- a/.github/prompts/codecov-patch-coverage-fix.prompt.md
+++ b/.github/prompts/codecov-patch-coverage-fix.prompt.md
@@ -29,7 +29,8 @@ The user will provide ONE of the following:
### Example Input Formats
**Format 1 - Codecov Comment:**
-```
+
+```text
Codecov Report
Attention: Patch coverage is 75.00000% with 4 lines in your changes missing coverage.
Project coverage is 82.45%. Comparing base (abc123) to head (def456).
@@ -57,7 +58,8 @@ Files with missing coverage:
- The target coverage (always 100% for patch coverage)
2. **Document Findings**: Create a structured list:
- ```
+
+ ```text
UNCOVERED FILES:
- FILE-001: [path/to/file.go] - Lines: [45-48, 62]
- FILE-002: [path/to/other.ts] - Lines: [23, 67-70]
diff --git a/.github/prompts/debug-web-console-errors.prompt.md b/.github/prompts/debug-web-console-errors.prompt.md
index ccfb8e972..b5572b4ec 100644
--- a/.github/prompts/debug-web-console-errors.prompt.md
+++ b/.github/prompts/debug-web-console-errors.prompt.md
@@ -22,12 +22,14 @@ This prompt accepts console error/warning input via two methods:
2. **Direct Input**: Paste the console output when prompted
**Console Input** (paste if not using selection):
-```
+
+```text
${input:consoleError:Paste browser console error/warning here}
```
**Selected Content** (if applicable):
-```
+
+```text
${selection}
```
@@ -40,7 +42,7 @@ Execute the following phases systematically. Do not skip phases or jump to concl
Categorize the error into one of these types:
| Type | Indicators | Primary Investigation Area |
-|------|------------|---------------------------|
+| ------ | ------------ | --------------------------- |
| **JavaScript Runtime Error** | `TypeError`, `ReferenceError`, `SyntaxError`, stack trace with `.js`/`.ts` files | Frontend source code |
| **React/Framework Error** | `React`, `hook`, `component`, `render`, `state`, `props` in message | Component lifecycle, hooks, state management |
| **Network Error** | `fetch`, `XMLHttpRequest`, HTTP status codes, `CORS`, `net::ERR_` | API endpoints, backend handlers, network config |
@@ -96,6 +98,7 @@ Propose and implement fixes:
3. **Error Handling**: Improve error messages and recovery
For each fix, provide:
+
- **Before**: The problematic code
- **After**: The corrected code
- **Explanation**: Why this change resolves the issue
@@ -171,23 +174,27 @@ Structure your response as follows:
## Error-Specific Handling
### JavaScript Runtime Errors
+
- Focus on type safety and null checks
- Look for incorrect assumptions about data shapes
- Check async/await and Promise handling
### React Errors
+
- Examine component lifecycle and hook dependencies
- Check for stale closures in useEffect/useCallback
- Verify prop types and default values
- Look for missing keys in lists
### Network Errors
+
- Trace the full request path: frontend β backend β response
- Check authentication/authorization middleware
- Verify CORS configuration
- Examine request/response payload shapes
### Console Warnings
+
- Assess severity (blocking vs. informational)
- Prioritize deprecation warnings for future compatibility
- Address React key warnings and dependency array warnings
diff --git a/.github/prompts/prompt-builder.prompt.md b/.github/prompts/prompt-builder.prompt.md
index b45075456..6a90cffdf 100644
--- a/.github/prompts/prompt-builder.prompt.md
+++ b/.github/prompts/prompt-builder.prompt.md
@@ -7,6 +7,7 @@ description: 'Guide users through creating high-quality GitHub Copilot prompts w
# Professional Prompt Builder
You are an expert prompt engineer specializing in GitHub Copilot prompt development with deep knowledge of:
+
- Prompt engineering best practices and patterns
- VS Code Copilot customization capabilities
- Effective persona design and task specification
@@ -20,24 +21,28 @@ Your task is to guide me through creating a new `.prompt.md` file by systematica
I will ask you targeted questions to gather all necessary information. After collecting your responses, I will generate the complete prompt file content following established patterns from this repository.
### 1. **Prompt Identity & Purpose**
+
- What is the intended filename for your prompt (e.g., `generate-react-component.prompt.md`)?
- Provide a clear, one-sentence description of what this prompt accomplishes
- What category does this prompt fall into? (code generation, analysis, documentation, testing, refactoring, architecture, etc.)
### 2. **Persona Definition**
+
- What role/expertise should Copilot embody? Be specific about:
- - Technical expertise level (junior, senior, expert, specialist)
- - Domain knowledge (languages, frameworks, tools)
- - Years of experience or specific qualifications
- - Example: "You are a senior .NET architect with 10+ years of experience in enterprise applications and extensive knowledge of C# 12, ASP.NET Core, and clean architecture patterns"
+ - Technical expertise level (junior, senior, expert, specialist)
+ - Domain knowledge (languages, frameworks, tools)
+ - Years of experience or specific qualifications
+ - Example: "You are a senior .NET architect with 10+ years of experience in enterprise applications and extensive knowledge of C# 12, ASP.NET Core, and clean architecture patterns"
### 3. **Task Specification**
+
- What is the primary task this prompt performs? Be explicit and measurable
- Are there secondary or optional tasks?
- What should the user provide as input? (selection, file, parameters, etc.)
- What constraints or requirements must be followed?
### 4. **Context & Variable Requirements**
+
- Will it use `${selection}` (user's selected code)?
- Will it use `${file}` (current file) or other file references?
- Does it need input variables like `${input:variableName}` or `${input:variableName:placeholder}`?
@@ -45,6 +50,7 @@ I will ask you targeted questions to gather all necessary information. After col
- Does it need to access other files or prompt files as dependencies?
### 5. **Detailed Instructions & Standards**
+
- What step-by-step process should Copilot follow?
- Are there specific coding standards, frameworks, or libraries to use?
- What patterns or best practices should be enforced?
@@ -52,6 +58,7 @@ I will ask you targeted questions to gather all necessary information. After col
- Should it follow any existing instruction files (`.instructions.md`)?
### 6. **Output Requirements**
+
- What format should the output be? (code, markdown, JSON, structured data, etc.)
- Should it create new files? If so, where and with what naming convention?
- Should it modify existing files?
@@ -59,7 +66,9 @@ I will ask you targeted questions to gather all necessary information. After col
- Are there specific formatting or structure requirements?
### 7. **Tool & Capability Requirements**
+
Which tools does this prompt need? Common options include:
+
- **File Operations**: `codebase`, `editFiles`, `search`, `problems`
- **Execution**: `runCommands`, `runTasks`, `runTests`, `terminalLastCommand`
- **External**: `fetch`, `githubRepo`, `openSimpleBrowser`
@@ -67,11 +76,13 @@ Which tools does this prompt need? Common options include:
- **Analysis**: `changes`, `findTestFiles`, `testFailure`, `searchResults`
### 8. **Technical Configuration**
+
- Should this run in a specific mode? (`agent`, `ask`, `edit`)
- Does it require a specific model? (usually auto-detected)
- Are there any special requirements or constraints?
### 9. **Quality & Validation Criteria**
+
- How should success be measured?
- What validation steps should be included?
- Are there common failure modes to address?
@@ -127,6 +138,7 @@ model: "[only if specific model required]"
```
The generated prompt will follow patterns observed in high-quality prompts like:
+
- **Comprehensive blueprints** (architecture-blueprint-generator)
- **Structured specifications** (create-github-action-workflow-specification)
- **Best practice guides** (dotnet-best-practices, csharp-xunit)
@@ -134,6 +146,7 @@ The generated prompt will follow patterns observed in high-quality prompts like:
- **Code generation** (playwright-generate-test)
Each prompt will be optimized for:
+
- **AI Consumption**: Token-efficient, structured content
- **Maintainability**: Clear sections, consistent formatting
- **Extensibility**: Easy to modify and enhance
diff --git a/.github/prompts/sql-code-review.prompt.md b/.github/prompts/sql-code-review.prompt.md
index a4021c74d..41a162c8f 100644
--- a/.github/prompts/sql-code-review.prompt.md
+++ b/.github/prompts/sql-code-review.prompt.md
@@ -12,6 +12,7 @@ Perform a thorough SQL code review of ${selection} (or entire project if no sele
## π Security Analysis
### SQL Injection Prevention
+
```sql
-- β CRITICAL: SQL Injection vulnerability
query = "SELECT * FROM users WHERE id = " + userInput;
@@ -27,12 +28,14 @@ EXEC sp_executesql N'SELECT * FROM users WHERE id = @id', N'@id INT', @id = @use
```
### Access Control & Permissions
+
- **Principle of Least Privilege**: Grant minimum required permissions
- **Role-Based Access**: Use database roles instead of direct user permissions
- **Schema Security**: Proper schema ownership and access controls
- **Function/Procedure Security**: Review DEFINER vs INVOKER rights
### Data Protection
+
- **Sensitive Data Exposure**: Avoid SELECT * on tables with sensitive columns
- **Audit Logging**: Ensure sensitive operations are logged
- **Data Masking**: Use views or functions to mask sensitive data
@@ -41,6 +44,7 @@ EXEC sp_executesql N'SELECT * FROM users WHERE id = @id', N'@id INT', @id = @use
## β‘ Performance Optimization
### Query Structure Analysis
+
```sql
-- β BAD: Inefficient query patterns
SELECT DISTINCT u.*
@@ -58,18 +62,21 @@ AND o.order_date < '2025-01-01';
```
### Index Strategy Review
+
- **Missing Indexes**: Identify columns that need indexing
- **Over-Indexing**: Find unused or redundant indexes
- **Composite Indexes**: Multi-column indexes for complex queries
- **Index Maintenance**: Check for fragmented or outdated indexes
### Join Optimization
+
- **Join Types**: Verify appropriate join types (INNER vs LEFT vs EXISTS)
- **Join Order**: Optimize for smaller result sets first
- **Cartesian Products**: Identify and fix missing join conditions
- **Subquery vs JOIN**: Choose the most efficient approach
### Aggregate and Window Functions
+
```sql
-- β BAD: Inefficient aggregation
SELECT user_id,
@@ -86,6 +93,7 @@ GROUP BY user_id;
## π οΈ Code Quality & Maintainability
### SQL Style & Formatting
+
```sql
-- β BAD: Poor formatting and style
select u.id,u.name,o.total from users u left join orders o on u.id=o.user_id where u.status='active' and o.order_date>='2024-01-01';
@@ -101,12 +109,14 @@ WHERE u.status = 'active'
```
### Naming Conventions
+
- **Consistent Naming**: Tables, columns, constraints follow consistent patterns
- **Descriptive Names**: Clear, meaningful names for database objects
- **Reserved Words**: Avoid using database reserved words as identifiers
- **Case Sensitivity**: Consistent case usage across schema
### Schema Design Review
+
- **Normalization**: Appropriate normalization level (avoid over/under-normalization)
- **Data Types**: Optimal data type choices for storage and performance
- **Constraints**: Proper use of PRIMARY KEY, FOREIGN KEY, CHECK, NOT NULL
@@ -115,6 +125,7 @@ WHERE u.status = 'active'
## ποΈ Database-Specific Best Practices
### PostgreSQL
+
```sql
-- Use JSONB for JSON data
CREATE TABLE events (
@@ -134,6 +145,7 @@ CREATE TABLE tags (
```
### MySQL
+
```sql
-- Use appropriate storage engines
CREATE TABLE sessions (
@@ -148,6 +160,7 @@ ADD INDEX idx_covering (status, created_at, id);
```
### SQL Server
+
```sql
-- Use appropriate data types
CREATE TABLE products (
@@ -162,6 +175,7 @@ CREATE COLUMNSTORE INDEX idx_sales_cs ON sales;
```
### Oracle
+
```sql
-- Use sequences for auto-increment
CREATE SEQUENCE user_id_seq START WITH 1 INCREMENT BY 1;
@@ -175,6 +189,7 @@ CREATE TABLE users (
## π§ͺ Testing & Validation
### Data Integrity Checks
+
```sql
-- Verify referential integrity
SELECT o.user_id
@@ -189,6 +204,7 @@ WHERE price < 0 OR stock_quantity < 0;
```
### Performance Testing
+
- **Execution Plans**: Review query execution plans
- **Load Testing**: Test queries with realistic data volumes
- **Stress Testing**: Verify performance under concurrent load
@@ -197,6 +213,7 @@ WHERE price < 0 OR stock_quantity < 0;
## π Common Anti-Patterns
### N+1 Query Problem
+
```sql
-- β BAD: N+1 queries in application code
for user in users:
@@ -209,6 +226,7 @@ LEFT JOIN orders o ON u.id = o.user_id;
```
### Overuse of DISTINCT
+
```sql
-- β BAD: DISTINCT masking join issues
SELECT DISTINCT u.name
@@ -223,6 +241,7 @@ GROUP BY u.name;
```
### Function Misuse in WHERE Clauses
+
```sql
-- β BAD: Functions prevent index usage
SELECT * FROM orders
@@ -237,6 +256,7 @@ WHERE order_date >= '2024-01-01'
## π SQL Review Checklist
### Security
+
- [ ] All user inputs are parameterized
- [ ] No dynamic SQL construction with string concatenation
- [ ] Appropriate access controls and permissions
@@ -244,6 +264,7 @@ WHERE order_date >= '2024-01-01'
- [ ] SQL injection attack vectors are eliminated
### Performance
+
- [ ] Indexes exist for frequently queried columns
- [ ] No unnecessary SELECT * statements
- [ ] JOINs are optimized and use appropriate types
@@ -251,6 +272,7 @@ WHERE order_date >= '2024-01-01'
- [ ] Subqueries are optimized or converted to JOINs
### Code Quality
+
- [ ] Consistent naming conventions
- [ ] Proper formatting and indentation
- [ ] Meaningful comments for complex logic
@@ -258,6 +280,7 @@ WHERE order_date >= '2024-01-01'
- [ ] Error handling is implemented
### Schema Design
+
- [ ] Tables are properly normalized
- [ ] Constraints enforce data integrity
- [ ] Indexes support query patterns
@@ -267,7 +290,8 @@ WHERE order_date >= '2024-01-01'
## π― Review Output Format
### Issue Template
-```
+
+```text
## [PRIORITY] [CATEGORY]: [Brief Description]
**Location**: [Table/View/Procedure name and line number if applicable]
@@ -282,12 +306,14 @@ WHERE order_date >= '2024-01-01'
```
**After**:
+
```sql
-- Improved SQL
```
**Expected Improvement**: [Performance gain, security benefit]
-```
+
+```text
### Summary Assessment
- **Security Score**: [1-10] - SQL injection protection, access controls
diff --git a/.github/prompts/sql-optimization.prompt.md b/.github/prompts/sql-optimization.prompt.md
index ea6375c79..44b514b2c 100644
--- a/.github/prompts/sql-optimization.prompt.md
+++ b/.github/prompts/sql-optimization.prompt.md
@@ -12,6 +12,7 @@ Expert SQL performance optimization for ${selection} (or entire project if no se
## π― Core Optimization Areas
### Query Performance Analysis
+
```sql
-- β BAD: Inefficient query patterns
SELECT * FROM orders o
@@ -35,6 +36,7 @@ WHERE o.created_at >= '2024-01-01'
```
### Index Strategy Optimization
+
```sql
-- β BAD: Poor indexing strategy
CREATE INDEX idx_user_data ON users(email, first_name, last_name, created_at);
@@ -52,6 +54,7 @@ WHERE status IS NOT NULL;
```
### Subquery Optimization
+
```sql
-- β BAD: Correlated subquery
SELECT p.product_name, p.price
@@ -75,6 +78,7 @@ WHERE price > avg_category_price;
## π Performance Tuning Techniques
### JOIN Optimization
+
```sql
-- β BAD: Inefficient JOIN order and conditions
SELECT o.*, c.name, p.product_name
@@ -95,6 +99,7 @@ WHERE o.created_at > '2024-01-01';
```
### Pagination Optimization
+
```sql
-- β BAD: OFFSET-based pagination (slow for large offsets)
SELECT * FROM products
@@ -115,6 +120,7 @@ LIMIT 20;
```
### Aggregation Optimization
+
```sql
-- β BAD: Multiple separate aggregation queries
SELECT COUNT(*) FROM orders WHERE status = 'pending';
@@ -132,6 +138,7 @@ FROM orders;
## π Query Anti-Patterns
### SELECT Performance Issues
+
```sql
-- β BAD: SELECT * anti-pattern
SELECT * FROM large_table lt
@@ -144,6 +151,7 @@ JOIN another_table at ON lt.id = at.ref_id;
```
### WHERE Clause Optimization
+
```sql
-- β BAD: Function calls in WHERE clause
SELECT * FROM orders
@@ -156,6 +164,7 @@ WHERE customer_email = 'john@example.com';
```
### OR vs UNION Optimization
+
```sql
-- β BAD: Complex OR conditions
SELECT * FROM products
@@ -171,6 +180,7 @@ SELECT * FROM products WHERE category = 'books' AND price < 50;
## π Database-Agnostic Optimization
### Batch Operations
+
```sql
-- β BAD: Row-by-row operations
INSERT INTO products (name, price) VALUES ('Product 1', 10.00);
@@ -185,6 +195,7 @@ INSERT INTO products (name, price) VALUES
```
### Temporary Table Usage
+
```sql
-- β GOOD: Using temporary tables for complex operations
CREATE TEMPORARY TABLE temp_calculations AS
@@ -205,6 +216,7 @@ WHERE tc.total_spent > 1000;
## π οΈ Index Management
### Index Design Principles
+
```sql
-- β GOOD: Covering index design
CREATE INDEX idx_orders_covering
@@ -214,6 +226,7 @@ INCLUDE (total_amount, status); -- SQL Server syntax
```
### Partial Index Strategy
+
```sql
-- β GOOD: Partial indexes for specific conditions
CREATE INDEX idx_orders_active
@@ -224,6 +237,7 @@ WHERE status IN ('pending', 'processing');
## π Performance Monitoring Queries
### Query Performance Analysis
+
```sql
-- Generic approach to identify slow queries
-- (Specific syntax varies by database)
@@ -253,6 +267,7 @@ ORDER BY avg_elapsed_time DESC;
## π― Universal Optimization Checklist
### Query Structure
+
- [ ] Avoiding SELECT * in production queries
- [ ] Using appropriate JOIN types (INNER vs LEFT/RIGHT)
- [ ] Filtering early in WHERE clauses
@@ -260,6 +275,7 @@ ORDER BY avg_elapsed_time DESC;
- [ ] Avoiding functions in WHERE clauses that prevent index usage
### Index Strategy
+
- [ ] Creating indexes on frequently queried columns
- [ ] Using composite indexes in the right column order
- [ ] Avoiding over-indexing (impacts INSERT/UPDATE performance)
@@ -267,12 +283,14 @@ ORDER BY avg_elapsed_time DESC;
- [ ] Creating partial indexes for specific query patterns
### Data Types and Schema
+
- [ ] Using appropriate data types for storage efficiency
- [ ] Normalizing appropriately (3NF for OLTP, denormalized for OLAP)
- [ ] Using constraints to help query optimizer
- [ ] Partitioning large tables when appropriate
### Query Patterns
+
- [ ] Using LIMIT/TOP for result set control
- [ ] Implementing efficient pagination strategies
- [ ] Using batch operations for bulk data changes
@@ -280,6 +298,7 @@ ORDER BY avg_elapsed_time DESC;
- [ ] Using prepared statements for repeated queries
### Performance Testing
+
- [ ] Testing queries with realistic data volumes
- [ ] Analyzing query execution plans
- [ ] Monitoring query performance over time
diff --git a/.github/prompts/structured-autonomy-generate.prompt.md b/.github/prompts/structured-autonomy-generate.prompt.md
index bbe825d0b..02fb8a37d 100644
--- a/.github/prompts/structured-autonomy-generate.prompt.md
+++ b/.github/prompts/structured-autonomy-generate.prompt.md
@@ -8,6 +8,7 @@ mode: agent
You are a PR implementation plan generator that creates complete, copy-paste ready implementation documentation.
Your SOLE responsibility is to:
+
1. Accept a complete PR plan (plan.md in plans/{feature-name}/)
2. Extract all implementation steps from the plan
3. Generate comprehensive step documentation with complete code
@@ -31,6 +32,7 @@ Follow the below to generate and save implementation files for each s
Output the plan as a COMPLETE markdown document using the , ready to be saved as a `.md` file.
The plan MUST include:
+
- Complete, copy-paste ready code blocks with ZERO modifications needed
- Exact file paths appropriate to the project structure
- Markdown checkboxes for EVERY action item
@@ -77,18 +79,22 @@ Return a comprehensive research package covering the entire project context.
+
# {FEATURE_NAME}
## Goal
+
{One sentence describing exactly what this implementation accomplishes}
## Prerequisites
+
Make sure that the use is currently on the `{feature-name}` branch before beginning implementation.
If not, move them to the correct branch. If the branch does not exist, create it from main.
### Step-by-Step Instructions
#### Step 1: {Action}
+
- [ ] {Specific instruction 1}
- [ ] Copy and paste code below into `{file}`:
@@ -104,13 +110,16 @@ If not, move them to the correct branch. If the branch does not exist, create it
```
##### Step 1 Verification Checklist
+
- [ ] No build errors
- [ ] Specific instructions for UI verification (if applicable)
#### Step 1 STOP & COMMIT
+
**STOP & COMMIT:** Agent must stop here and wait for the user to test, stage, and commit the change.
#### Step 2: {Action}
+
- [ ] {Specific Instruction 1}
- [ ] Copy and paste code below into `{file}`:
@@ -119,9 +128,11 @@ If not, move them to the correct branch. If the branch does not exist, create it
```
##### Step 2 Verification Checklist
+
- [ ] No build errors
- [ ] Specific instructions for UI verification (if applicable)
#### Step 2 STOP & COMMIT
+
**STOP & COMMIT:** Agent must stop here and wait for the user to test, stage, and commit the change.
diff --git a/.github/prompts/structured-autonomy-plan.prompt.md b/.github/prompts/structured-autonomy-plan.prompt.md
index 9f41535f5..ddb47e76c 100644
--- a/.github/prompts/structured-autonomy-plan.prompt.md
+++ b/.github/prompts/structured-autonomy-plan.prompt.md
@@ -65,6 +65,7 @@ Analyze the user's request and break it down into commits:
### Step 3: {Step Name}
...
```
+
diff --git a/.github/prompts/suggest-awesome-github-copilot-chatmodes.prompt.md b/.github/prompts/suggest-awesome-github-copilot-chatmodes.prompt.md
index 032c295e6..1990643fe 100644
--- a/.github/prompts/suggest-awesome-github-copilot-chatmodes.prompt.md
+++ b/.github/prompts/suggest-awesome-github-copilot-chatmodes.prompt.md
@@ -25,12 +25,14 @@ Analyze current repository context and suggest relevant Custom Chat Modes files
## Context Analysis Criteria
π **Repository Patterns**:
+
- Programming languages used (.cs, .js, .py, etc.)
- Framework indicators (ASP.NET, React, Azure, etc.)
- Project types (web apps, APIs, libraries, tools)
- Documentation needs (README, specs, ADRs)
π¨οΈ **Chat History Context**:
+
- Recent discussions and pain points
- Feature requests or implementation needs
- Code review patterns
@@ -41,7 +43,7 @@ Analyze current repository context and suggest relevant Custom Chat Modes files
Display analysis results in structured table comparing awesome-copilot custom chat modes with existing repository custom chat modes:
| Awesome-Copilot Custom Chat Mode | Description | Already Installed | Similar Local Custom Chat Mode | Suggestion Rationale |
-|---------------------------|-------------|-------------------|-------------------------|---------------------|
+| --------------------------- | ------------- | ------------------- | ------------------------- | --------------------- |
| [code-reviewer.agent.md](https://github.com/github/awesome-copilot/blob/main/agents/code-reviewer.agent.md) | Specialized code review custom chat mode | β No | None | Would enhance development workflow with dedicated code review assistance |
| [architect.agent.md](https://github.com/github/awesome-copilot/blob/main/agents/architect.agent.md) | Software architecture guidance | β Yes | azure_principal_architect.agent.md | Already covered by existing architecture custom chat modes |
| [debugging-expert.agent.md](https://github.com/github/awesome-copilot/blob/main/agents/debugging-expert.agent.md) | Debug assistance custom chat mode | β No | None | Could improve troubleshooting efficiency for development team |
diff --git a/.github/prompts/suggest-awesome-github-copilot-collections.prompt.md b/.github/prompts/suggest-awesome-github-copilot-collections.prompt.md
index 51c6b9606..ace17aafb 100644
--- a/.github/prompts/suggest-awesome-github-copilot-collections.prompt.md
+++ b/.github/prompts/suggest-awesome-github-copilot-collections.prompt.md
@@ -23,6 +23,7 @@ Analyze current repository context and suggest relevant collections from the [Gi
## Context Analysis Criteria
π **Repository Patterns**:
+
- Programming languages used (.cs, .js, .py, .ts, .bicep, .tf, etc.)
- Framework indicators (ASP.NET, React, Azure, Next.js, Angular, etc.)
- Project types (web apps, APIs, libraries, tools, infrastructure)
@@ -30,6 +31,7 @@ Analyze current repository context and suggest relevant collections from the [Gi
- Development workflow indicators (CI/CD, testing, deployment)
π¨οΈ **Chat History Context**:
+
- Recent discussions and pain points
- Feature requests or implementation needs
- Code review patterns and quality concerns
@@ -43,7 +45,7 @@ Display analysis results in structured table showing relevant collections and th
### Collection Recommendations
| Collection Name | Description | Items | Asset Overlap | Suggestion Rationale |
-|-----------------|-------------|-------|---------------|---------------------|
+| ----------------- | ------------- | ------- | --------------- | --------------------- |
| [Azure & Cloud Development](https://github.com/github/awesome-copilot/blob/main/collections/azure-cloud-development.md) | Comprehensive Azure cloud development tools including Infrastructure as Code, serverless functions, architecture patterns, and cost optimization | 15 items | 3 similar | Would enhance Azure development workflow with Bicep, Terraform, and cost optimization tools |
| [C# .NET Development](https://github.com/github/awesome-copilot/blob/main/collections/csharp-dotnet-development.md) | Essential prompts, instructions, and chat modes for C# and .NET development including testing, documentation, and best practices | 7 items | 2 similar | Already covered by existing .NET-related assets but includes advanced testing patterns |
| [Testing & Test Automation](https://github.com/github/awesome-copilot/blob/main/collections/testing-automation.md) | Comprehensive collection for writing tests, test automation, and test-driven development | 11 items | 1 similar | Could significantly improve testing practices with TDD guidance and automation tools |
@@ -53,11 +55,13 @@ Display analysis results in structured table showing relevant collections and th
For each suggested collection, break down individual assets:
**Azure & Cloud Development Collection Analysis:**
+
- β **New Assets (12)**: Azure cost optimization prompts, Bicep planning mode, AVM modules, Logic Apps expert mode
- β οΈ **Similar Assets (3)**: Azure DevOps pipelines (similar to existing CI/CD), Terraform (basic overlap), Containerization (Docker basics covered)
- π― **High Value**: Cost optimization tools, Infrastructure as Code expertise, Azure-specific architectural guidance
**Installation Preview:**
+
- Will install to `prompts/`: 4 Azure-specific prompts
- Will install to `instructions/`: 6 infrastructure and DevOps best practices
- Will install to `agents/`: 5 specialized Azure expert modes
@@ -132,13 +136,13 @@ When user confirms a collection installation:
## Post-Installation Guidance
After installing a collection, provide:
+
- **Asset Overview**: List of installed prompts, instructions, and chat modes
- **Usage Examples**: How to activate and use each type of asset
- **Workflow Integration**: Best practices for incorporating assets into development process
- **Customization Tips**: How to modify assets for specific project needs
- **Related Collections**: Suggestions for complementary collections that work well together
-
## Icons Reference
- β Collection recommended for installation
diff --git a/.github/prompts/suggest-awesome-github-copilot-instructions.prompt.md b/.github/prompts/suggest-awesome-github-copilot-instructions.prompt.md
index da8e30151..141c67643 100644
--- a/.github/prompts/suggest-awesome-github-copilot-instructions.prompt.md
+++ b/.github/prompts/suggest-awesome-github-copilot-instructions.prompt.md
@@ -24,12 +24,14 @@ Analyze current repository context and suggest relevant copilot-instruction file
## Context Analysis Criteria
π **Repository Patterns**:
+
- Programming languages used (.cs, .js, .py, .ts, etc.)
- Framework indicators (ASP.NET, React, Azure, Next.js, etc.)
- Project types (web apps, APIs, libraries, tools)
- Development workflow requirements (testing, CI/CD, deployment)
π¨οΈ **Chat History Context**:
+
- Recent discussions and pain points
- Technology-specific questions
- Coding standards discussions
@@ -40,7 +42,7 @@ Analyze current repository context and suggest relevant copilot-instruction file
Display analysis results in structured table comparing awesome-copilot instructions with existing repository instructions:
| Awesome-Copilot Instruction | Description | Already Installed | Similar Local Instruction | Suggestion Rationale |
-|------------------------------|-------------|-------------------|---------------------------|---------------------|
+| ------------------------------ | ------------- | ------------------- | --------------------------- | --------------------- |
| [blazor.instructions.md](https://github.com/github/awesome-copilot/blob/main/instructions/blazor.instructions.md) | Blazor development guidelines | β No | blazor.instructions.md | Already covered by existing Blazor instructions |
| [reactjs.instructions.md](https://github.com/github/awesome-copilot/blob/main/instructions/reactjs.instructions.md) | ReactJS development standards | β No | None | Would enhance React development with established patterns |
| [java.instructions.md](https://github.com/github/awesome-copilot/blob/main/instructions/java.instructions.md) | Java development best practices | β No | None | Could improve Java code quality and consistency |
@@ -55,6 +57,7 @@ Display analysis results in structured table comparing awesome-copilot instructi
## File Structure Requirements
Based on GitHub documentation, copilot-instructions files should be:
+
- **Repository-wide instructions**: `.github/copilot-instructions.md` (applies to entire repository)
- **Path-specific instructions**: `.github/instructions/NAME.instructions.md` (applies to specific file patterns via `applyTo` frontmatter)
- **Community instructions**: `instructions/NAME.instructions.md` (for sharing and distribution)
@@ -62,6 +65,7 @@ Based on GitHub documentation, copilot-instructions files should be:
## Front Matter Structure
Instructions files in awesome-copilot use this front matter format:
+
```markdown
---
description: 'Brief description of what this instruction provides'
diff --git a/.github/prompts/suggest-awesome-github-copilot-prompts.prompt.md b/.github/prompts/suggest-awesome-github-copilot-prompts.prompt.md
index ffc871b14..50206254f 100644
--- a/.github/prompts/suggest-awesome-github-copilot-prompts.prompt.md
+++ b/.github/prompts/suggest-awesome-github-copilot-prompts.prompt.md
@@ -24,12 +24,14 @@ Analyze current repository context and suggest relevant prompt files from the [G
## Context Analysis Criteria
π **Repository Patterns**:
+
- Programming languages used (.cs, .js, .py, etc.)
- Framework indicators (ASP.NET, React, Azure, etc.)
- Project types (web apps, APIs, libraries, tools)
- Documentation needs (README, specs, ADRs)
π¨οΈ **Chat History Context**:
+
- Recent discussions and pain points
- Feature requests or implementation needs
- Code review patterns
@@ -40,7 +42,7 @@ Analyze current repository context and suggest relevant prompt files from the [G
Display analysis results in structured table comparing awesome-copilot prompts with existing repository prompts:
| Awesome-Copilot Prompt | Description | Already Installed | Similar Local Prompt | Suggestion Rationale |
-|-------------------------|-------------|-------------------|---------------------|---------------------|
+| ------------------------- | ------------- | ------------------- | --------------------- | --------------------- |
| [code-review.md](https://github.com/github/awesome-copilot/blob/main/prompts/code-review.md) | Automated code review prompts | β No | None | Would enhance development workflow with standardized code review processes |
| [documentation.md](https://github.com/github/awesome-copilot/blob/main/prompts/documentation.md) | Generate project documentation | β Yes | create_oo_component_documentation.prompt.md | Already covered by existing documentation prompts |
| [debugging.md](https://github.com/github/awesome-copilot/blob/main/prompts/debugging.md) | Debug assistance prompts | β No | None | Could improve troubleshooting efficiency for development team |
@@ -64,7 +66,6 @@ Display analysis results in structured table comparing awesome-copilot prompts w
- Include links to both awesome-copilot prompts and similar local prompts
- Don't provide any additional information or context beyond the table and the analysis
-
## Icons Reference
- β Already installed in repo
diff --git a/.github/prompts/supply-chain-vulnerability-remediation.prompt.md b/.github/prompts/supply-chain-vulnerability-remediation.prompt.md
index c5c507284..74c7652fa 100644
--- a/.github/prompts/supply-chain-vulnerability-remediation.prompt.md
+++ b/.github/prompts/supply-chain-vulnerability-remediation.prompt.md
@@ -30,6 +30,7 @@ The user will provide ONE of the following:
### Expected Input Formats
**Format 1 - PR Comment:**
+
```markdown
## π Supply Chain Security Scan Results
@@ -62,7 +63,8 @@ The user will provide ONE of the following:
`https://github.com/Owner/Repo/actions/runs/123456789`
**Format 3 - Raw Scan Output:**
-```
+
+```text
HIGH CVE-2025-58183 golang.org/x/net 1.22.0 fixed:1.25.5
CRITICAL CVE-2025-58186 alpine-baselayout 3.4.0 fixed:3.4.3
...
@@ -80,7 +82,8 @@ CRITICAL CVE-2025-58186 alpine-baselayout 3.4.0 fixed:3.4.3
- Package ecosystem (Go, npm, Alpine APK, etc.)
2. **Create Vulnerability Inventory**: Structure findings as:
- ```
+
+ ```text
CRITICAL VULNERABILITIES:
- CVE-2025-58183: golang.org/x/net 1.22.0 β 1.25.5 (Buffer overflow)
@@ -113,7 +116,8 @@ For each vulnerability (prioritizing Critical β High β Medium β Low):
- Are there compensating controls? (WAF, network isolation, input validation)
3. **Risk Scoring**: Assign a project-specific risk rating:
- ```
+
+ ```text
RISK MATRIX:
- CRITICAL-IMMEDIATE: Exploitable, affects exposed services, no mitigations
- HIGH-URGENT: Exploitable, limited exposure or partial mitigations
@@ -151,6 +155,7 @@ For each vulnerability requiring action, determine the approach:
1. **Generate File Changes**: Create concrete edits:
**For Go modules:**
+
```bash
# Update specific module
go get golang.org/x/net@v1.25.5
@@ -159,6 +164,7 @@ For each vulnerability requiring action, determine the approach:
```
**For npm packages:**
+
```bash
npm update package-name@version
npm audit fix
@@ -166,6 +172,7 @@ For each vulnerability requiring action, determine the approach:
```
**For Alpine packages in Dockerfile:**
+
```dockerfile
# Update base image or specific packages
FROM golang:1.25.5-alpine3.19 AS builder
@@ -178,6 +185,7 @@ For each vulnerability requiring action, determine the approach:
- Inline comments in dependency files
3. **Create Suppression Rules** (if accepting risk):
+
```yaml
# .trivyignore or similar
CVE-2025-58183 # Risk accepted: Not using vulnerable HTTP/2 features
@@ -186,6 +194,7 @@ For each vulnerability requiring action, determine the approach:
### Phase 5: Validation
1. **Run Tests**: Ensure changes don't break functionality
+
```bash
# Run full test suite
make test
@@ -195,6 +204,7 @@ For each vulnerability requiring action, determine the approach:
```
2. **Verify Fix**: Re-run security scan
+
```bash
# Re-scan Docker image
trivy image charon:local
@@ -275,6 +285,7 @@ Save to `docs/security/vulnerability-analysis-[DATE].md`:
### 2. Updated Files
Apply changes directly to:
+
- `go.mod` / `go.sum`
- `package.json` / `package-lock.json`
- `Dockerfile`
@@ -283,7 +294,7 @@ Apply changes directly to:
### 3. Validation Report
-```
+```text
VALIDATION RESULTS:
β All tests pass (backend: 542/542, frontend: 128/128)
β Application builds successfully
@@ -313,6 +324,7 @@ govulncheck ./...
```
**Common Issues**:
+
- Transitive dependencies: Use `go mod why package` to understand dependency chain
- Major version updates: Check for breaking changes in release notes
- Replace directives: May need updating if pinning specific versions
@@ -337,6 +349,7 @@ npm audit
```
**Common Issues**:
+
- Peer dependency conflicts: May need to update multiple related packages
- Breaking changes: Check CHANGELOG.md for each package
- Lock file conflicts: Ensure package-lock.json is committed
@@ -358,6 +371,7 @@ RUN apk upgrade --no-cache
```
**Common Issues**:
+
- Base image versions: Pin to specific minor version (alpine3.19) not just alpine:latest
- Package availability: Not all versions available in Alpine repos
- Image size: `apk upgrade` can significantly increase image size
@@ -414,18 +428,21 @@ RUN cd crowdsec && \
## Error Handling
-### If CVE data cannot be retrieved:
+### If CVE data cannot be retrieved
+
- Document the limitation
- Proceed with available information from scan
- Mark for manual review
-### If dependency update causes test failures:
+### If dependency update causes test failures
+
- Identify root cause (API changes, behavioral differences)
- Evaluate alternative versions
- Consider mitigations or acceptance if no compatible fix exists
- Document findings and decision
-### If no fix is available:
+### If no fix is available
+
- Research workarounds and compensating controls
- Evaluate if code path is actually used
- Consider temporarily disabling feature if critical
diff --git a/.github/skills/.skill-quickref-gorm-scanner.md b/.github/skills/.skill-quickref-gorm-scanner.md
index fd85ebdeb..43ed898c9 100644
--- a/.github/skills/.skill-quickref-gorm-scanner.md
+++ b/.github/skills/.skill-quickref-gorm-scanner.md
@@ -1,11 +1,13 @@
# GORM Security Scanner - Quick Reference
## Purpose
+
Detect GORM security issues including ID leaks, exposed secrets, and common GORM misconfigurations.
## Quick Start
### Recommended Usage (Report Mode)
+
```bash
# Via skill runner (stdout only)
.github/skills/scripts/skill-runner.sh security-scan-gorm
@@ -21,6 +23,7 @@ pre-commit run --hook-stage manual gorm-security-scan --all-files
```
### Check Mode (CI/Pre-commit)
+
```bash
# Exit 1 if issues found (console output only)
.github/skills/scripts/skill-runner.sh security-scan-gorm --check
@@ -32,6 +35,7 @@ pre-commit run --hook-stage manual gorm-security-scan --all-files
### Why Export Reports?
**Benefits:**
+
- β **Agent-Friendly**: AI agents can read files instead of parsing terminal history
- β **Persistence**: Results saved for later review and comparison
- β **CI/CD**: Upload as GitHub Actions artifacts for audit trail
@@ -39,6 +43,7 @@ pre-commit run --hook-stage manual gorm-security-scan --all-files
- β **Compliance**: Evidence of security scans for audits
**Example Agent Usage:**
+
```bash
# User/Agent generates report
.github/skills/scripts/skill-runner.sh security-scan-gorm --report docs/reports/gorm-scan.txt
@@ -54,7 +59,7 @@ pre-commit run --hook-stage manual gorm-security-scan --all-files
## Detection Patterns
| Severity | Pattern | Example |
-|----------|---------|---------|
+| ---------- | --------- | --------- |
| π΄ CRITICAL | Numeric ID exposure | `ID uint json:"id"` β should be `json:"-"` |
| π΄ CRITICAL | Exposed secrets | `APIKey string json:"api_key"` β should be `json:"-"` |
| π‘ HIGH | DTO embedding models | `ProxyHostResponse embeds models.ProxyHost` |
@@ -64,6 +69,7 @@ pre-commit run --hook-stage manual gorm-security-scan --all-files
## Common Fixes
### Fix ID Leak
+
```go
// Before
type User struct {
@@ -79,6 +85,7 @@ type User struct {
```
### Fix Exposed Secret
+
```go
// Before
type User struct {
@@ -92,6 +99,7 @@ type User struct {
```
### Fix DTO Embedding
+
```go
// Before
type ProxyHostResponse struct {
@@ -148,6 +156,7 @@ type GitHubUser struct {
## Security Rationale
**Why ID leaks matter:**
+
- Information disclosure (sequential patterns)
- IDOR vulnerability (guess valid IDs)
- Database structure exposure
diff --git a/.github/skills/README.md b/.github/skills/README.md
index 96a02e42e..ca52794e4 100644
--- a/.github/skills/README.md
+++ b/.github/skills/README.md
@@ -11,7 +11,7 @@ Agent Skills are self-documenting, AI-discoverable task definitions that combine
## Directory Structure
-```
+```text
.github/skills/
βββ README.md # This file
βββ scripts/ # Shared infrastructure scripts
@@ -32,7 +32,7 @@ Agent Skills are self-documenting, AI-discoverable task definitions that combine
### Testing Skills
| Skill Name | Category | Description | Status |
-|------------|----------|-------------|--------|
+| ------------ | ---------- | ------------- | -------- |
| [test-backend-coverage](./test-backend-coverage.SKILL.md) | test | Run Go backend tests with coverage analysis | β Active |
| [test-backend-unit](./test-backend-unit.SKILL.md) | test | Run fast Go unit tests without coverage | β Active |
| [test-frontend-coverage](./test-frontend-coverage.SKILL.md) | test | Run frontend tests with coverage reporting | β Active |
@@ -44,7 +44,7 @@ Agent Skills are self-documenting, AI-discoverable task definitions that combine
### Integration Testing Skills
| Skill Name | Category | Description | Status |
-|------------|----------|-------------|--------|
+| ------------ | ---------- | ------------- | -------- |
| [integration-test-all](./integration-test-all.SKILL.md) | integration | Run all integration tests in sequence | β Active |
| [integration-test-coraza](./integration-test-coraza.SKILL.md) | integration | Test Coraza WAF integration | β Active |
| [integration-test-crowdsec](./integration-test-crowdsec.SKILL.md) | integration | Test CrowdSec bouncer integration | β Active |
@@ -54,7 +54,7 @@ Agent Skills are self-documenting, AI-discoverable task definitions that combine
### Security Skills
| Skill Name | Category | Description | Status |
-|------------|----------|-------------|--------|
+| ------------ | ---------- | ------------- | -------- |
| [security-scan-gorm](./security-scan-gorm.SKILL.md) | security | Detect GORM ID leaks, exposed secrets, and misconfigurations | β Active |
| [security-scan-trivy](./security-scan-trivy.SKILL.md) | security | Run Trivy vulnerability scanner | β Active |
| [security-scan-go-vuln](./security-scan-go-vuln.SKILL.md) | security | Run Go vulnerability check | β Active |
@@ -62,13 +62,13 @@ Agent Skills are self-documenting, AI-discoverable task definitions that combine
### QA Skills
| Skill Name | Category | Description | Status |
-|------------|----------|-------------|--------|
+| --- | --- | --- | --- |
| [qa-lefthook-all](./qa-lefthook-all.SKILL.md) | qa | Run all lefthook pre-commitβphase hooks on entire codebase | β Active |
### Utility Skills
| Skill Name | Category | Description | Status |
-|------------|----------|-------------|--------|
+| ------------ | ---------- | ------------- | -------- |
| [utility-clear-go-cache](./utility-clear-go-cache.SKILL.md) | utility | Clear Go build and module caches | β Active |
| [utility-bump-beta](./utility-bump-beta.SKILL.md) | utility | Increment beta version number | β Active |
| [utility-db-recovery](./utility-db-recovery.SKILL.md) | utility | Database integrity check and recovery | β Active |
@@ -76,7 +76,7 @@ Agent Skills are self-documenting, AI-discoverable task definitions that combine
### Docker Skills
| Skill Name | Category | Description | Status |
-|------------|----------|-------------|--------|
+| ------------ | ---------- | ------------- | -------- |
| [docker-start-dev](./docker-start-dev.SKILL.md) | docker | Start development Docker Compose environment | β Active |
| [docker-stop-dev](./docker-stop-dev.SKILL.md) | docker | Stop development Docker Compose environment | β Active |
| [docker-rebuild-e2e](./docker-rebuild-e2e.SKILL.md) | docker | Rebuild Docker image and restart E2E Playwright container | β Active |
@@ -130,6 +130,7 @@ python3 .github/skills/scripts/validate-skills.py
### Validation Checks
The validator ensures:
+
- β Required frontmatter fields are present
- β Field formats are correct (name, version, description)
- β Tags meet minimum/maximum requirements
@@ -204,7 +205,8 @@ Brief description of what this skill does.
**Last Updated**: YYYY-MM-DD
**Maintained by**: Charon Project
-```
+
+```text
### 3. Create Execution Script
@@ -267,12 +269,14 @@ python3 .github/skills/scripts/validate-skills.py --single .github/skills/{skill
## Best Practices
### Documentation
+
- Keep SKILL.md under 500 lines
- Use progressive disclosure (link to extended docs for complex topics)
- Include practical examples
- Document all prerequisites and environment variables
### Scripts
+
- Always source helper scripts for consistent logging and error handling
- Validate environment before execution
- Use `set -euo pipefail` for robust error handling
@@ -280,12 +284,14 @@ python3 .github/skills/scripts/validate-skills.py --single .github/skills/{skill
- Clean up resources on exit
### Metadata
+
- Use accurate `execution_time` values for scheduling
- Set `ci_cd_safe: false` for skills requiring human oversight
- Mark `idempotent: true` only if truly safe to run multiple times
- Include all required dependencies in `requirements`
### Error Handling
+
- Use helper functions (`log_error`, `error_exit`, `check_command_exists`)
- Provide clear error messages with remediation steps
- Return appropriate exit codes (0 = success, non-zero = failure)
@@ -331,33 +337,43 @@ get_project_root ["marker_file"] # Find project root directory
## Troubleshooting
### Skill not found
-```
+
+```text
Error: Skill not found: skill-name
```
+
**Solution**: Verify the skill directory exists in `.github/skills/` and contains a `SKILL.md` file
### Skill script not executable
-```
+
+```text
Error: Skill execution script is not executable
```
+
**Solution**: Run `chmod +x .github/skills/{skill-name}/scripts/run.sh`
### Validation errors
-```
+
+```text
[ERROR] skill.SKILL.md :: description: Must be 120 characters or less
```
+
**Solution**: Fix the frontmatter field according to the error message and re-validate
### Command not found in skill
-```
+
+```text
Error: go is not installed or not in PATH
```
+
**Solution**: Install the required dependency or ensure it's in your PATH
## Integration Points
### VS Code Tasks
+
Skills are integrated in `.vscode/tasks.json`:
+
```json
{
"label": "Test: Backend with Coverage",
@@ -368,14 +384,18 @@ Skills are integrated in `.vscode/tasks.json`:
```
### GitHub Actions
+
Skills are referenced in `.github/workflows/`:
+
```yaml
- name: Run Backend Tests with Coverage
run: .github/skills/scripts/skill-runner.sh test-backend-coverage
```
### Pre-commit Hooks
+
Skills can be used in `.pre-commit-config.yaml`:
+
```yaml
repos:
- repo: local
@@ -396,6 +416,7 @@ repos:
## Support
For issues, questions, or contributions:
+
1. Check existing [GitHub Issues](https://github.com/Wikid82/charon/issues)
2. Review [CONTRIBUTING.md](../../CONTRIBUTING.md)
3. Create a new issue if needed
diff --git a/.github/skills/docker-prune.SKILL.md b/.github/skills/docker-prune.SKILL.md
index 5c5b98bde..45f089405 100644
--- a/.github/skills/docker-prune.SKILL.md
+++ b/.github/skills/docker-prune.SKILL.md
@@ -84,7 +84,7 @@ This skill requires no environment variables.
### Output Example
-```
+```text
Deleted Containers:
f8d1234567890abcdef1234567890abcdef1234567890abcdef1234567890ab
@@ -115,6 +115,7 @@ The `docker system prune -f` command removes:
## What Gets Preserved
This command **DOES NOT** remove:
+
- **Running Containers**: Active containers are untouched
- **Tagged Images**: Images with tags are preserved
- **Volumes**: Data volumes are never removed
@@ -176,6 +177,7 @@ docker system df -v
```
Output shows:
+
- **Images**: Total size of cached images
- **Containers**: Size of container writable layers
- **Local Volumes**: Size of data volumes
@@ -184,6 +186,7 @@ Output shows:
## When to Use This Skill
Use this skill when:
+
- Disk space is running low
- After development cycles (many builds)
- After running integration tests
@@ -203,21 +206,27 @@ Use this skill when:
Common issues and solutions:
### Permission Denied
-```
+
+```text
Error: permission denied
```
+
Solution: Add user to docker group or use sudo
### Daemon Not Running
-```
+
+```text
Error: Cannot connect to Docker daemon
```
+
Solution: Start Docker service
### Resource in Use
-```
+
+```text
Error: resource is in use
```
+
This is normal - only unused resources are removed
## Advanced Cleanup Options
@@ -261,6 +270,7 @@ docker system prune -a --volumes -f
## Disk Space Recovery
Typical space recovery by resource type:
+
- **Stopped Containers**: 10-100 MB each
- **Dangling Images**: 100 MB - 2 GB total
- **Build Cache**: 1-10 GB (if many builds)
diff --git a/.github/skills/docker-rebuild-e2e.SKILL.md b/.github/skills/docker-rebuild-e2e.SKILL.md
index 40422eeea..bb3345a8a 100644
--- a/.github/skills/docker-rebuild-e2e.SKILL.md
+++ b/.github/skills/docker-rebuild-e2e.SKILL.md
@@ -70,6 +70,7 @@ metadata:
Rebuilds the Charon Docker image and restarts the Playwright E2E testing environment with fresh code. This skill handles the complete lifecycle: stopping existing containers, optionally cleaning volumes, rebuilding the image, and starting fresh containers with health check verification.
**Use this skill when:**
+
- You've made code changes and need to test them in E2E tests
- E2E tests are failing due to stale container state
- You need a clean slate for debugging
@@ -135,15 +136,15 @@ Enable MailHog for email testing:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| no-cache | boolean | No | false | Force rebuild without Docker cache |
-| clean | boolean | No | false | Remove test volumes for fresh state |
-| profile | string | No | "" | Docker Compose profile to enable |
+| --- | --- | --- | --- | --- |
+| no-cache | boolean | No | false | Force rebuild without Docker cache |
+| clean | boolean | No | false | Remove test volumes for fresh state |
+| profile | string | No | "" | Docker Compose profile to enable |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| DOCKER_NO_CACHE | No | false | Force rebuild without cache |
| SKIP_VOLUME_CLEANUP | No | false | Preserve test data volumes |
@@ -167,7 +168,7 @@ This skill uses `.docker/compose/docker-compose.playwright-ci.yml` which include
### Volumes Created
| Volume | Purpose |
-|--------|---------|
+| -------- | --------- |
| playwright_data | Application data and SQLite database |
| playwright_caddy_data | Caddy server data |
| playwright_caddy_config | Caddy configuration |
@@ -216,6 +217,7 @@ curl -sf http://localhost:8080/api/v1/health
```
The skill will:
+
- Wait up to 60 seconds for container to be healthy
- Check every 5 seconds
- Report final health status
@@ -226,27 +228,35 @@ The skill will:
### Common Issues
#### Docker Build Failed
-```
+
+```text
Error: docker build failed
```
+
**Solution**: Check Dockerfile syntax, ensure all COPY sources exist
#### Port Already in Use
-```
+
+```text
Error: bind: address already in use
```
+
**Solution**: Stop conflicting services on port 8080
#### Health Check Timeout
-```
+
+```text
Error: Container did not become healthy in 60s
```
+
**Solution**: Check container logs with `docker logs charon-playwright`
#### Volume Permission Issues
-```
+
+```text
Error: permission denied
```
+
**Solution**: Run with `--clean` to recreate volumes with proper permissions
## Verifying the Environment
@@ -278,7 +288,7 @@ docker exec charon-playwright sqlite3 /app/data/charon.db ".tables"
## Key File Locations
| File | Purpose |
-|------|---------|
+| ------ | --------- |
| `Dockerfile` | Main application Dockerfile |
| `.docker/compose/docker-compose.playwright-ci.yml` | CI E2E test compose config |
| `.docker/compose/docker-compose.playwright-local.yml` | Local E2E test compose config |
@@ -299,5 +309,6 @@ docker exec charon-playwright sqlite3 /app/data/charon.db ".tables"
**Last Updated**: 2026-01-27
**Maintained by**: Charon Project Team
**Compose Files**:
+
- CI: `.docker/compose/docker-compose.playwright-ci.yml` (uses GitHub Secrets, no .env)
- Local: `.docker/compose/docker-compose.playwright-local.yml` (uses .env file)
diff --git a/.github/skills/docker-start-dev.SKILL.md b/.github/skills/docker-start-dev.SKILL.md
index 98e07e968..92dfcd2e6 100644
--- a/.github/skills/docker-start-dev.SKILL.md
+++ b/.github/skills/docker-start-dev.SKILL.md
@@ -76,6 +76,7 @@ This skill accepts no parameters. Services are configured in `.docker/compose/do
## Environment Variables
This skill uses environment variables defined in:
+
- `.env` (if present)
- `.docker/compose/docker-compose.dev.yml` environment section
- Shell environment
@@ -88,7 +89,7 @@ This skill uses environment variables defined in:
### Output Example
-```
+```text
[+] Running 5/5
β Network charon-dev_default Created
β Container charon-dev-db-1 Started
@@ -100,6 +101,7 @@ This skill uses environment variables defined in:
## What Gets Started
Services defined in `.docker/compose/docker-compose.dev.yml`:
+
1. **charon-app**: Main application container
2. **charon-db**: SQLite or PostgreSQL database
3. **crowdsec**: Security bouncer
@@ -109,6 +111,7 @@ Services defined in `.docker/compose/docker-compose.dev.yml`:
## Service Startup Order
Docker Compose respects `depends_on` directives:
+
1. Database services start first
2. Security services (CrowdSec) start next
3. Application services start after dependencies
@@ -167,6 +170,7 @@ docker compose -f .docker/compose/docker-compose.dev.yml exec app /bin/sh
## Port Mappings
Default development ports (check `.docker/compose/docker-compose.dev.yml`):
+
- **8080**: Application HTTP
- **8443**: Application HTTPS (if configured)
- **9000**: Admin panel (if configured)
@@ -175,6 +179,7 @@ Default development ports (check `.docker/compose/docker-compose.dev.yml`):
## Detached Mode
The `-d` flag runs containers in detached mode:
+
- Services run in background
- Terminal is freed for other commands
- Use `docker compose logs -f` to view output
@@ -184,27 +189,35 @@ The `-d` flag runs containers in detached mode:
Common issues and solutions:
### Port Already in Use
-```
+
+```text
Error: bind: address already in use
```
+
Solution: Stop conflicting service or change port in compose file
### Image Pull Failed
-```
+
+```text
Error: failed to pull image
```
+
Solution: Check network connection, authenticate to registry
### Insufficient Resources
-```
+
+```text
Error: failed to start container
```
+
Solution: Free up system resources, stop other containers
### Configuration Error
-```
+
+```text
Error: invalid compose file
```
+
Solution: Validate compose file with `docker compose config`
## Post-Startup Verification
@@ -212,16 +225,19 @@ Solution: Validate compose file with `docker compose config`
After starting, verify:
1. **All Services Running**:
+
```bash
docker compose -f .docker/compose/docker-compose.dev.yml ps
```
2. **Application Accessible**:
+
```bash
curl http://localhost:8080/health
```
3. **No Error Logs**:
+
```bash
docker compose -f .docker/compose/docker-compose.dev.yml logs --tail=50
```
diff --git a/.github/skills/docker-stop-dev.SKILL.md b/.github/skills/docker-stop-dev.SKILL.md
index ca5e3314a..7a64f580c 100644
--- a/.github/skills/docker-stop-dev.SKILL.md
+++ b/.github/skills/docker-stop-dev.SKILL.md
@@ -85,7 +85,7 @@ This skill requires no environment variables.
### Output Example
-```
+```text
[+] Running 5/5
β Container charon-dev-caddy-1 Removed
β Container charon-dev-app-1 Removed
@@ -97,6 +97,7 @@ This skill requires no environment variables.
## What Gets Stopped
Services defined in `.docker/compose/docker-compose.dev.yml`:
+
1. **Application Containers**: Charon main app
2. **Database Containers**: SQLite/PostgreSQL services
3. **Security Services**: CrowdSec bouncer
@@ -106,6 +107,7 @@ Services defined in `.docker/compose/docker-compose.dev.yml`:
## What Gets Preserved
The `down` command preserves:
+
- **Volumes**: Database data persists
- **Images**: Docker images remain cached
- **Configs**: Configuration files unchanged
@@ -115,6 +117,7 @@ To remove volumes, use `docker compose -f .docker/compose/docker-compose.dev.yml
## Shutdown Order
Docker Compose stops services in reverse dependency order:
+
1. Reverse proxy stops first
2. Application services stop next
3. Security services stop
@@ -155,6 +158,7 @@ docker network ls | grep charon-dev
## Graceful Shutdown
The `down` command:
+
1. Sends `SIGTERM` to each container
2. Waits for graceful shutdown (default: 10 seconds)
3. Sends `SIGKILL` if timeout exceeded
@@ -164,6 +168,7 @@ The `down` command:
## When to Use This Skill
Use this skill when:
+
- Switching between development and production modes
- Freeing system resources (CPU, memory)
- Preparing for system shutdown/restart
@@ -176,21 +181,27 @@ Use this skill when:
Common issues and solutions:
### Container Already Stopped
-```
+
+```text
Warning: Container already stopped
```
+
No action needed - idempotent operation
### Volume in Use
-```
+
+```text
Error: volume is in use
```
+
Solution: Check for other containers using the volume
### Permission Denied
-```
+
+```text
Error: permission denied
```
+
Solution: Add user to docker group or use sudo
## Post-Shutdown Verification
@@ -198,16 +209,19 @@ Solution: Add user to docker group or use sudo
After stopping, verify:
1. **No Running Containers**:
+
```bash
docker ps --filter "name=charon-dev"
```
2. **Network Removed**:
+
```bash
docker network ls | grep charon-dev
```
3. **Volumes Still Exist** (if data preservation intended):
+
```bash
docker volume ls | grep charon
```
diff --git a/.github/skills/integration-test-all.SKILL.md b/.github/skills/integration-test-all.SKILL.md
index 9ac6bb188..f624c2791 100644
--- a/.github/skills/integration-test-all.SKILL.md
+++ b/.github/skills/integration-test-all.SKILL.md
@@ -101,13 +101,13 @@ For use in GitHub Actions workflows:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| DOCKER_BUILDKIT | No | 1 | Enable BuildKit for faster builds |
| SKIP_CLEANUP | No | false | Skip container cleanup after tests |
| TEST_TIMEOUT | No | 300 | Timeout in seconds for each test |
@@ -115,17 +115,21 @@ For use in GitHub Actions workflows:
## Outputs
### Success Exit Code
+
- **0**: All integration tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **2**: Docker environment setup failed
- **3**: Container startup timeout
- **4**: Network connectivity issues
### Console Output
+
Example output:
-```
+
+```text
=== Running Integration Test Suite ===
β Cerberus Integration Tests
β Coraza WAF Integration Tests
@@ -180,15 +184,19 @@ This skill executes the following test suites:
### Common Errors
#### Error: Cannot connect to Docker daemon
+
**Solution**: Ensure Docker is running: `sudo systemctl start docker`
#### Error: Port already in use
+
**Solution**: Stop conflicting services or run cleanup: `docker compose down`
#### Error: Container startup timeout
+
**Solution**: Check Docker logs: `docker compose logs`
#### Error: Network connectivity issues
+
**Solution**: Verify network configuration: `docker network ls`
### Troubleshooting
diff --git a/.github/skills/integration-test-cerberus.SKILL.md b/.github/skills/integration-test-cerberus.SKILL.md
index 504c3042c..3d216488d 100644
--- a/.github/skills/integration-test-cerberus.SKILL.md
+++ b/.github/skills/integration-test-cerberus.SKILL.md
@@ -89,13 +89,13 @@ VERBOSE=1 .github/skills/scripts/skill-runner.sh integration-test-cerberus
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| CHARON_EMERGENCY_TOKEN | No | (empty) | Emergency token for Cerberus teardown flows |
| SKIP_CLEANUP | No | false | Skip container cleanup after tests |
| TEST_TIMEOUT | No | 600 | Timeout in seconds for the test |
@@ -103,9 +103,11 @@ VERBOSE=1 .github/skills/scripts/skill-runner.sh integration-test-cerberus
## Outputs
### Success Exit Code
+
- **0**: All Cerberus integration tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **2**: Docker environment setup failed
- **3**: Container startup timeout
diff --git a/.github/skills/integration-test-coraza.SKILL.md b/.github/skills/integration-test-coraza.SKILL.md
index 99c1acd6b..004aa3fdd 100644
--- a/.github/skills/integration-test-coraza.SKILL.md
+++ b/.github/skills/integration-test-coraza.SKILL.md
@@ -96,29 +96,33 @@ For use in GitHub Actions workflows:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| WAF_ENABLED | No | true | Enable WAF protection for tests |
| TEST_HOST | No | localhost:8080 | Target host for WAF tests |
## Outputs
### Success Exit Code
+
- **0**: All WAF tests passed (attacks blocked correctly)
### Error Exit Codes
+
- **1**: One or more attacks were not blocked
- **2**: Docker environment setup failed
- **3**: WAF not responding or misconfigured
### Console Output
+
Example output:
-```
+
+```text
=== Testing Coraza WAF Integration ===
β SQL Injection: Blocked (403 Forbidden)
β XSS Attack: Blocked (403 Forbidden)
@@ -165,15 +169,19 @@ WAF_ENABLED=false .github/skills/scripts/skill-runner.sh integration-test-coraza
### Common Errors
#### Error: WAF not responding
+
**Solution**: Verify Docker containers are running: `docker ps | grep coraza`
#### Error: Attacks not blocked (false negatives)
+
**Solution**: Check WAF configuration in `configs/coraza/` and rule sets
#### Error: Legitimate requests blocked (false positives)
+
**Solution**: Review WAF logs and adjust rule sensitivity
#### Error: Connection refused
+
**Solution**: Ensure application is accessible: `curl http://localhost:8080/health`
### Debugging
diff --git a/.github/skills/integration-test-crowdsec-decisions.SKILL.md b/.github/skills/integration-test-crowdsec-decisions.SKILL.md
index 7b232ebc7..bd8742d17 100644
--- a/.github/skills/integration-test-crowdsec-decisions.SKILL.md
+++ b/.github/skills/integration-test-crowdsec-decisions.SKILL.md
@@ -100,31 +100,35 @@ For use in GitHub Actions workflows:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| CROWDSEC_API_KEY | No | auto | API key for LAPI access |
-| CROWDSEC_LAPI_URL | No | http://crowdsec:8080 | CrowdSec LAPI endpoint |
+| CROWDSEC_LAPI_URL | No | | CrowdSec LAPI endpoint |
| TEST_IP | No | 192.0.2.1 | Test IP address for decisions |
## Outputs
### Success Exit Code
+
- **0**: All decision API tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **2**: LAPI not accessible
- **3**: Authentication failed
- **4**: Decision creation/deletion failed
### Console Output
+
Example output:
-```
+
+```text
=== Testing CrowdSec Decision API ===
β Create Decision: IP 192.0.2.1 blocked for 4h
β Retrieve Decisions: 1 active decision found
@@ -210,15 +214,19 @@ CROWDSEC_API_KEY=my-api-key VERBOSE=1 \
### Common Errors
#### Error: LAPI not responding
+
**Solution**: Check LAPI container: `docker ps | grep crowdsec`
#### Error: Authentication failed
+
**Solution**: Verify API key: `docker exec crowdsec cscli machines list`
#### Error: Decision not created
+
**Solution**: Check LAPI logs for validation errors
#### Error: Decision not found after creation
+
**Solution**: Verify database connectivity and permissions
### Debugging
diff --git a/.github/skills/integration-test-crowdsec-startup.SKILL.md b/.github/skills/integration-test-crowdsec-startup.SKILL.md
index 683cf0c04..a5ff1af8d 100644
--- a/.github/skills/integration-test-crowdsec-startup.SKILL.md
+++ b/.github/skills/integration-test-crowdsec-startup.SKILL.md
@@ -104,13 +104,13 @@ For use in GitHub Actions workflows:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| STARTUP_TIMEOUT | No | 60 | Maximum wait for startup (seconds) |
| SKIP_CLEANUP | No | false | Skip container cleanup after tests |
| CROWDSEC_VERSION | No | latest | CrowdSec image version to test |
@@ -118,17 +118,21 @@ For use in GitHub Actions workflows:
## Outputs
### Success Exit Code
+
- **0**: All startup tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **2**: Startup timeout exceeded
- **3**: Configuration errors detected
- **4**: Health check failed
### Console Output
+
Example output:
-```
+
+```text
=== Testing CrowdSec Startup Sequence ===
β LAPI Initialization: Ready in 8s
β Database Migration: Successful
@@ -233,15 +237,19 @@ SKIP_CLEANUP=true \
### Common Errors
#### Error: Startup timeout exceeded
+
**Solution**: Increase STARTUP_TIMEOUT or check container logs for hangs
#### Error: Database connection failed
+
**Solution**: Verify database container is running and accessible
#### Error: Configuration validation failed
+
**Solution**: Check CrowdSec config files for syntax errors
#### Error: Port already in use
+
**Solution**: Stop conflicting services or change port configuration
### Debugging
diff --git a/.github/skills/integration-test-crowdsec.SKILL.md b/.github/skills/integration-test-crowdsec.SKILL.md
index 2d63ad4cf..2021a602e 100644
--- a/.github/skills/integration-test-crowdsec.SKILL.md
+++ b/.github/skills/integration-test-crowdsec.SKILL.md
@@ -97,31 +97,35 @@ For use in GitHub Actions workflows:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| CROWDSEC_API_KEY | No | auto | Bouncer API key (auto-generated if not set) |
-| CROWDSEC_LAPI_URL | No | http://crowdsec:8080 | CrowdSec LAPI endpoint |
+| CROWDSEC_LAPI_URL | No | | CrowdSec LAPI endpoint |
| BOUNCER_SYNC_INTERVAL | No | 60 | Decision sync interval in seconds |
## Outputs
### Success Exit Code
+
- **0**: All CrowdSec integration tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **2**: CrowdSec LAPI not accessible
- **3**: Bouncer authentication failed
- **4**: Decision synchronization failed
### Console Output
+
Example output:
-```
+
+```text
=== Testing CrowdSec Bouncer Integration ===
β LAPI Connection: Successful
β Bouncer Authentication: Valid API Key
@@ -179,15 +183,19 @@ BOUNCER_SYNC_INTERVAL=30 VERBOSE=1 \
### Common Errors
#### Error: Cannot connect to LAPI
+
**Solution**: Verify LAPI container is running: `docker ps | grep crowdsec`
#### Error: Authentication failed
+
**Solution**: Check API key is valid: `docker exec crowdsec cscli bouncers list`
#### Error: No decisions retrieved
+
**Solution**: Create test decisions: `docker exec crowdsec cscli decisions add --ip 1.2.3.4`
#### Error: Blocking not working
+
**Solution**: Check bouncer logs: `docker logs charon-app | grep crowdsec`
### Debugging
diff --git a/.github/skills/integration-test-rate-limit.SKILL.md b/.github/skills/integration-test-rate-limit.SKILL.md
index 0a3e4b0c4..097ae4da9 100644
--- a/.github/skills/integration-test-rate-limit.SKILL.md
+++ b/.github/skills/integration-test-rate-limit.SKILL.md
@@ -88,13 +88,13 @@ VERBOSE=1 .github/skills/scripts/skill-runner.sh integration-test-rate-limit
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| RATE_LIMIT_REQUESTS | No | 3 | Allowed requests per window in the test |
| RATE_LIMIT_WINDOW_SEC | No | 10 | Window size in seconds |
| RATE_LIMIT_BURST | No | 1 | Burst size in tests |
@@ -102,9 +102,11 @@ VERBOSE=1 .github/skills/scripts/skill-runner.sh integration-test-rate-limit
## Outputs
### Success Exit Code
+
- **0**: All rate limit integration tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **2**: Docker environment setup failed
- **3**: Container startup timeout
diff --git a/.github/skills/integration-test-security-headers.SKILL.md b/.github/skills/integration-test-security-headers.SKILL.md
index ac5decea9..c8c7645d6 100644
--- a/.github/skills/integration-test-security-headers.SKILL.md
+++ b/.github/skills/integration-test-security-headers.SKILL.md
@@ -99,7 +99,7 @@ CHARON_IMAGE=charon:my-build SH_PREFIX=sh-test .github/skills/scripts/skill-runn
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| CHARON_IMAGE | No | charon:local | Image under test |
| SH_PREFIX | No | secheaders | Prefix for created containers, network and volumes |
| SH_HTTP_PORT | No | 8182 | Host port for Caddy HTTP |
@@ -110,9 +110,11 @@ CHARON_IMAGE=charon:my-build SH_PREFIX=sh-test .github/skills/scripts/skill-runn
## Outputs
### Success Exit Code
+
- **0**: All security header integration tests passed
### Error Exit Codes
+
- **1**: One or more assertions failed or the environment failed to start
## Related Skills
diff --git a/.github/skills/integration-test-waf.SKILL.md b/.github/skills/integration-test-waf.SKILL.md
index e6dd64cb8..118959a6f 100644
--- a/.github/skills/integration-test-waf.SKILL.md
+++ b/.github/skills/integration-test-waf.SKILL.md
@@ -68,21 +68,23 @@ Run the WAF integration tests:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
+| --- | --- | --- | --- | --- |
| verbose | boolean | No | false | Enable verbose output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| WAF_MODE | No | (script default) | Override WAF mode |
## Outputs
### Success Exit Code
+
- 0: All WAF integration tests passed
### Error Exit Codes
+
- 1: One or more tests failed
- 2: Docker environment setup failed
- 3: Container startup timeout
diff --git a/.github/skills/qa-lefthook-all.SKILL.md b/.github/skills/qa-lefthook-all.SKILL.md
index 30da6d9e3..833a4806a 100644
--- a/.github/skills/qa-lefthook-all.SKILL.md
+++ b/.github/skills/qa-lefthook-all.SKILL.md
@@ -106,13 +106,13 @@ SKIP=prettier,eslint .github/skills/scripts/skill-runner.sh qa-precommit-all
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| files | string | No | --all-files | File selection mode (--all-files or staged) |
+| --- | --- | --- | --- | --- |
+| files | string | No | --all-files | File selection mode (--all-files or staged) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| SKIP | No | "" | Comma-separated hook IDs to skip |
| PRE_COMMIT_HOME | No | ~/.cache/pre-commit | Pre-commit cache directory |
@@ -127,6 +127,7 @@ SKIP=prettier,eslint .github/skills/scripts/skill-runner.sh qa-precommit-all
The following hooks are configured in `.pre-commit-config.yaml`:
### General Hooks
+
- **trailing-whitespace**: Remove trailing whitespace
- **end-of-file-fixer**: Ensure files end with newline
- **check-yaml**: Validate YAML syntax
@@ -135,24 +136,29 @@ The following hooks are configured in `.pre-commit-config.yaml`:
- **check-added-large-files**: Prevent committing large files
### Python Hooks
+
- **black**: Code formatting
- **isort**: Import sorting
- **flake8**: Linting
- **mypy**: Type checking
### Go Hooks
+
- **gofmt**: Code formatting
- **go-vet**: Static analysis
- **golangci-lint**: Comprehensive linting
### JavaScript/TypeScript Hooks
+
- **prettier**: Code formatting
- **eslint**: Linting and code quality
### Markdown Hooks
+
- **markdownlint**: Markdown linting and formatting
### Security Hooks
+
- **detect-private-key**: Prevent committing private keys
- **detect-aws-credentials**: Prevent committing AWS credentials
@@ -167,7 +173,8 @@ source .venv/bin/activate
```
Output:
-```
+
+```text
Trim Trailing Whitespace.....................................Passed
Fix End of Files.............................................Passed
Check Yaml...................................................Passed
@@ -225,30 +232,35 @@ SKIP=golangci-lint,mypy .github/skills/scripts/skill-runner.sh qa-precommit-all
### Common Issues
**Virtual environment not activated**:
+
```bash
Error: pre-commit not found
Solution: source .venv/bin/activate
```
**Pre-commit not installed**:
+
```bash
Error: pre-commit command not available
Solution: pip install pre-commit
```
**Hooks not installed**:
+
```bash
Error: Run 'pre-commit install'
Solution: pre-commit install
```
**Hook execution failed**:
+
```bash
Hook X failed
Solution: Review error output and fix reported issues
```
**Language tool missing**:
+
```bash
Error: golangci-lint not found
Solution: Install required language tools
@@ -263,7 +275,9 @@ Solution: Install required language tools
## Hook Fixing Strategies
### Auto-fixable Issues
+
These hooks automatically fix issues:
+
- `trailing-whitespace`
- `end-of-file-fixer`
- `black`
@@ -274,7 +288,9 @@ These hooks automatically fix issues:
**Workflow**: Run pre-commit, review changes, commit fixed files
### Manual Fixes Required
+
These hooks only report issues:
+
- `check-yaml`
- `check-json`
- `flake8`
diff --git a/.github/skills/qa-precommit-all.SKILL.md b/.github/skills/qa-precommit-all.SKILL.md
index 30da6d9e3..833a4806a 100644
--- a/.github/skills/qa-precommit-all.SKILL.md
+++ b/.github/skills/qa-precommit-all.SKILL.md
@@ -106,13 +106,13 @@ SKIP=prettier,eslint .github/skills/scripts/skill-runner.sh qa-precommit-all
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| files | string | No | --all-files | File selection mode (--all-files or staged) |
+| --- | --- | --- | --- | --- |
+| files | string | No | --all-files | File selection mode (--all-files or staged) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| SKIP | No | "" | Comma-separated hook IDs to skip |
| PRE_COMMIT_HOME | No | ~/.cache/pre-commit | Pre-commit cache directory |
@@ -127,6 +127,7 @@ SKIP=prettier,eslint .github/skills/scripts/skill-runner.sh qa-precommit-all
The following hooks are configured in `.pre-commit-config.yaml`:
### General Hooks
+
- **trailing-whitespace**: Remove trailing whitespace
- **end-of-file-fixer**: Ensure files end with newline
- **check-yaml**: Validate YAML syntax
@@ -135,24 +136,29 @@ The following hooks are configured in `.pre-commit-config.yaml`:
- **check-added-large-files**: Prevent committing large files
### Python Hooks
+
- **black**: Code formatting
- **isort**: Import sorting
- **flake8**: Linting
- **mypy**: Type checking
### Go Hooks
+
- **gofmt**: Code formatting
- **go-vet**: Static analysis
- **golangci-lint**: Comprehensive linting
### JavaScript/TypeScript Hooks
+
- **prettier**: Code formatting
- **eslint**: Linting and code quality
### Markdown Hooks
+
- **markdownlint**: Markdown linting and formatting
### Security Hooks
+
- **detect-private-key**: Prevent committing private keys
- **detect-aws-credentials**: Prevent committing AWS credentials
@@ -167,7 +173,8 @@ source .venv/bin/activate
```
Output:
-```
+
+```text
Trim Trailing Whitespace.....................................Passed
Fix End of Files.............................................Passed
Check Yaml...................................................Passed
@@ -225,30 +232,35 @@ SKIP=golangci-lint,mypy .github/skills/scripts/skill-runner.sh qa-precommit-all
### Common Issues
**Virtual environment not activated**:
+
```bash
Error: pre-commit not found
Solution: source .venv/bin/activate
```
**Pre-commit not installed**:
+
```bash
Error: pre-commit command not available
Solution: pip install pre-commit
```
**Hooks not installed**:
+
```bash
Error: Run 'pre-commit install'
Solution: pre-commit install
```
**Hook execution failed**:
+
```bash
Hook X failed
Solution: Review error output and fix reported issues
```
**Language tool missing**:
+
```bash
Error: golangci-lint not found
Solution: Install required language tools
@@ -263,7 +275,9 @@ Solution: Install required language tools
## Hook Fixing Strategies
### Auto-fixable Issues
+
These hooks automatically fix issues:
+
- `trailing-whitespace`
- `end-of-file-fixer`
- `black`
@@ -274,7 +288,9 @@ These hooks automatically fix issues:
**Workflow**: Run pre-commit, review changes, commit fixed files
### Manual Fixes Required
+
These hooks only report issues:
+
- `check-yaml`
- `check-json`
- `flake8`
diff --git a/.github/skills/security-scan-codeql.SKILL.md b/.github/skills/security-scan-codeql.SKILL.md
index ab38c9422..f7b1fdee4 100644
--- a/.github/skills/security-scan-codeql.SKILL.md
+++ b/.github/skills/security-scan-codeql.SKILL.md
@@ -119,14 +119,14 @@ Get text-formatted detailed findings:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| language | string | No | all | Language to scan (go, javascript, all) |
-| format | string | No | summary | Output format (sarif, text, summary) |
+| --- | --- | --- | --- | --- |
+| language | string | No | all | Language to scan (go, javascript, all) |
+| format | string | No | summary | Output format (sarif, text, summary) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| CODEQL_THREADS | No | 0 | Analysis threads (0 = auto-detect) |
| CODEQL_FAIL_ON_ERROR | No | true | Fail on HIGH/CRITICAL findings |
@@ -135,7 +135,7 @@ Get text-formatted detailed findings:
This skill uses the **security-and-quality** suite to match CI:
| Language | Suite | Queries | Coverage |
-|----------|-------|---------|----------|
+| --- | --- | --- | --- |
| Go | go-security-and-quality.qls | version-dependent | Security + quality issues |
| JavaScript | javascript-security-and-quality.qls | version-dependent | Security + quality issues |
@@ -159,7 +159,7 @@ This skill uses the **security-and-quality** suite to match CI:
### CWE Coverage
| Category | Description | Languages |
-|----------|-------------|-----------|
+| ---------- | ------------- | ----------- |
| CWE-079 | Cross-Site Scripting (XSS) | JS |
| CWE-089 | SQL Injection | Go, JS |
| CWE-117 | Log Injection | Go |
@@ -182,7 +182,8 @@ This skill uses the **security-and-quality** suite to match CI:
```
Output:
-```
+
+```text
[STEP] CODEQL: Scanning Go code...
[INFO] Creating database for backend/
[INFO] Analyzing with security-and-quality suite (61 queries)
@@ -224,18 +225,21 @@ pre-commit run codeql-check-findings --all-files
### Common Issues
**CodeQL version too old**:
+
```bash
Error: Extensible predicate API mismatch
Solution: Upgrade CodeQL CLI: gh codeql set-version latest
```
**Query pack not found**:
+
```bash
Error: Could not resolve pack codeql/go-queries
Solution: codeql pack download codeql/go-queries codeql/javascript-queries
```
**Database creation failed**:
+
```bash
Error: No source files found
Solution: Verify source-root points to correct directory
@@ -258,7 +262,7 @@ Solution: Verify source-root points to correct directory
This skill is specifically designed to match GitHub Actions CodeQL workflow:
| Parameter | Local | CI | Aligned |
-|-----------|-------|-----|---------|
+| ----------- | ------- | ----- | --------- |
| Query Suite | security-and-quality | security-and-quality | β |
| Query Expansion | version-dependent | version-dependent | β (when versions match) |
| Threading | auto | auto | β |
@@ -289,7 +293,7 @@ SARIF files are automatically uploaded to GitHub Security tab in CI.
## Performance
| Language | Database Creation | Analysis | Total |
-|----------|------------------|----------|-------|
+| ---------- | ------------------ | ---------- | ------- |
| Go | ~30s | ~30s | ~60s |
| JavaScript | ~45s | ~45s | ~90s |
| All | ~75s | ~75s | ~150s |
diff --git a/.github/skills/security-scan-docker-image.SKILL.md b/.github/skills/security-scan-docker-image.SKILL.md
index a6cfe1e5e..b5a3577c1 100644
--- a/.github/skills/security-scan-docker-image.SKILL.md
+++ b/.github/skills/security-scan-docker-image.SKILL.md
@@ -102,7 +102,7 @@ This skill replicates the **exact CI supply chain verification process** used in
## Key Differences from Trivy Filesystem Scan
| Aspect | Trivy (Filesystem) | This Skill (Image Scan) |
-|--------|-------------------|------------------------|
+| -------- | ------------------- | ------------------------ |
| **Scan Target** | Source code + dependencies | Built Docker image |
| **Alpine Packages** | β Not detected | β Detected |
| **Compiled Binaries** | β Not detected | β Detected |
@@ -202,14 +202,14 @@ FAIL_ON_SEVERITY="Critical" \
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
+| --- | --- | --- | --- | --- |
| image_tag | string | No | charon:local | Docker image tag to build and scan |
| no_cache | boolean | No | false | Build without Docker cache (pass "no-cache" as second arg) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| SYFT_VERSION | No | v1.17.0 | Syft version (matches CI) |
| GRYPE_VERSION | No | v0.107.0 | Grype version (matches CI) |
| IMAGE_TAG | No | charon:local | Default image tag if not provided |
@@ -231,7 +231,7 @@ FAIL_ON_SEVERITY="Critical" \
### Output Format
-```
+```text
[INFO] Building Docker image: charon:local...
[BUILD] Using Dockerfile with multi-stage build
[BUILD] Image built successfully: charon:local
@@ -388,7 +388,7 @@ fi
**Project Standards (Matches CI)**:
| Severity | CVSS Range | Action | Exit Code |
-|----------|-----------|--------|-----------|
+| ---------- | ----------- | -------- | ----------- |
| π΄ **CRITICAL** | 9.0-10.0 | **MUST FIX** - Blocks commit/push | 1 |
| π **HIGH** | 7.0-8.9 | **SHOULD FIX** - Blocks commit/push | 1 |
| π‘ **MEDIUM** | 4.0-6.9 | Fix in next release (logged) | 0 |
@@ -399,36 +399,42 @@ fi
### Common Issues
**Docker not running**:
+
```bash
[ERROR] Docker daemon is not running
Solution: Start Docker Desktop or Docker service
```
**Syft not installed**:
+
```bash
[ERROR] Syft not found - install from: https://github.com/anchore/syft
Solution: Install Syft v1.17.0 using installation instructions above
```
**Grype not installed**:
+
```bash
[ERROR] Grype not found - install from: https://github.com/anchore/grype
Solution: Install Grype v0.107.0 using installation instructions above
```
**Build failure**:
+
```bash
[ERROR] Docker build failed with exit code 1
Solution: Check Dockerfile syntax and dependency availability
```
**Network timeout (vulnerability scan)**:
+
```bash
[WARNING] Failed to update Grype vulnerability database
Solution: Check internet connection or retry later
```
**Disk space insufficient**:
+
```bash
[ERROR] No space left on device
Solution: Clean up Docker images and containers: docker system prune -a
@@ -472,7 +478,7 @@ The QA_Security agent **MUST**:
This skill **exactly replicates** the supply-chain-pr.yml workflow:
| Step | CI Workflow | This Skill | Match |
-|------|------------|------------|-------|
+| ------ | ------------ | ------------ | ------- |
| Build Image | β Docker build | β Docker build | β |
| Load Image | β Load from artifact | β Use built image | β |
| Syft Version | v1.17.0 | v1.17.0 | β |
diff --git a/.github/skills/security-scan-go-vuln.SKILL.md b/.github/skills/security-scan-go-vuln.SKILL.md
index 1b09aefeb..d12b4a168 100644
--- a/.github/skills/security-scan-go-vuln.SKILL.md
+++ b/.github/skills/security-scan-go-vuln.SKILL.md
@@ -105,14 +105,14 @@ GOVULNCHECK_FORMAT=json .github/skills/scripts/skill-runner.sh security-scan-go-
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| format | string | No | text | Output format (text, json, sarif) |
-| mode | string | No | source | Scan mode (source or binary) |
+| --- | --- | --- | --- | --- |
+| format | string | No | text | Output format (text, json, sarif) |
+| mode | string | No | source | Scan mode (source or binary) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| GOVULNCHECK_FORMAT | No | text | Output format override |
## Outputs
@@ -127,14 +127,14 @@ GOVULNCHECK_FORMAT=json .github/skills/scripts/skill-runner.sh security-scan-go-
### Text Output (Default)
-```
+```text
Scanning for dependencies with known vulnerabilities...
No vulnerabilities found.
```
Or if vulnerabilities are found:
-```
+```text
Found 2 vulnerabilities in dependencies
Vulnerability #1: GO-2023-1234
@@ -163,7 +163,8 @@ cd backend
```
Output:
-```
+
+```text
Scanning your code and 125 packages across 23 dependent modules for known vulnerabilities...
No vulnerabilities found.
```
@@ -202,24 +203,28 @@ No vulnerabilities found.
### Common Issues
**Go not installed**:
+
```bash
Error: Go 1.23+ is required
Solution: Install Go 1.23 or higher
```
**Network unavailable**:
+
```bash
Error: Failed to fetch vulnerability database
Solution: Check internet connection or proxy settings
```
**Vulnerabilities found**:
+
```bash
Exit code: 3
Solution: Review vulnerabilities and update affected packages
```
**Module not found**:
+
```bash
Error: go.mod file not found
Solution: Run from a valid Go module directory
@@ -238,7 +243,7 @@ Solution: Run from a valid Go module directory
## Notes
-- `govulncheck` uses the official Go vulnerability database at https://vuln.go.dev
+- `govulncheck` uses the official Go vulnerability database at
- Database is automatically updated during each scan
- Only checks vulnerabilities that are reachable from your code
- Does not require building the code (analyzes source)
diff --git a/.github/skills/security-scan-gorm.SKILL.md b/.github/skills/security-scan-gorm.SKILL.md
index b329ad8f8..fe319ec19 100644
--- a/.github/skills/security-scan-gorm.SKILL.md
+++ b/.github/skills/security-scan-gorm.SKILL.md
@@ -63,6 +63,7 @@ This skill is essential for maintaining secure database models and preventing in
## When to Use This Skill
Use this skill when:
+
- β Creating or modifying GORM database models
- β Reviewing code for security issues before commit
- β Validating API response DTOs for ID exposure
@@ -87,6 +88,7 @@ Use this skill when:
**Risk:** Information disclosure, IDOR vulnerability, database enumeration
**Example:**
+
```go
// β BAD: Internal database ID exposed
type User struct {
@@ -110,6 +112,7 @@ type User struct {
**Risk:** Credential exposure, unauthorized access
**Example:**
+
```go
// β BAD: API key visible in responses
type User struct {
@@ -129,6 +132,7 @@ type User struct {
**Risk:** Unintentional ID exposure through embedding
**Example:**
+
```go
// β BAD: Inherits exposed ID from models.ProxyHost
type ProxyHostResponse struct {
@@ -213,15 +217,15 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| mode | string | No | --report | Operating mode (--report, --check, --enforce) |
+| --- | --- | --- | --- | --- |
+| mode | string | No | --report | Operating mode (--report, --check, --enforce) |
| output_file | string | No | (stdout) | Path to save report file (e.g., docs/reports/gorm-scan.txt) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
-| VERBOSE | No | 0 | Enable verbose debug output (set to 1) |
+| --- | --- | --- | --- |
+| VERBOSE | No | 0 | Enable verbose debug output (set to 1) |
## Outputs
@@ -234,7 +238,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
### Output Format
-```
+```text
π GORM Security Scanner v1.0.0
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
@@ -276,6 +280,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
**Target:** GORM models with numeric IDs exposed in JSON
**Detection Logic:**
+
1. Find `type XXX struct` declarations
2. Apply GORM model detection heuristics:
- File in `internal/models/` directory, OR
@@ -292,6 +297,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
**Target:** Response/DTO structs that embed GORM models
**Detection Logic:**
+
1. Find structs with "Response" or "DTO" in name
2. Look for embedded model types (from `models` package)
3. Check if embedded model has exposed ID field
@@ -302,6 +308,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
**Target:** API keys, tokens, passwords, secrets with visible JSON tags
**Detection Logic:**
+
1. Find fields matching: `APIKey`, `Secret`, `Token`, `Password`, `Hash`
2. Check if JSON tag is NOT `json:"-"`
3. Flag as **CRITICAL**
@@ -311,6 +318,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
**Target:** ID fields without `gorm:"primaryKey"`
**Detection Logic:**
+
1. Find ID fields with GORM tags
2. Check if `primaryKey` directive is missing
3. Flag as **MEDIUM**
@@ -320,6 +328,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
**Target:** Foreign key fields without index tags
**Detection Logic:**
+
1. Find fields ending with `ID` or `Id`
2. Check if GORM tag lacks `index` directive
3. Flag as **INFO**
@@ -329,6 +338,7 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
**Target:** Models with exposed IDs but no external identifier
**Detection Logic:**
+
1. Find models with exposed `json:"id"`
2. Check if `UUID` field exists
3. Flag as **HIGH** if missing
@@ -346,6 +356,7 @@ Use inline comments to suppress false positives:
### Examples
**External API Response:**
+
```go
// gorm-scanner:ignore External API response, not a GORM model
type GitHubUser struct {
@@ -354,6 +365,7 @@ type GitHubUser struct {
```
**Legacy Code During Migration:**
+
```go
// gorm-scanner:ignore Legacy model, scheduled for refactor in #1234
type OldModel struct {
@@ -362,6 +374,7 @@ type OldModel struct {
```
**Internal Service (Never Serialized):**
+
```go
// gorm-scanner:ignore Internal service struct, never serialized to HTTP
type InternalProcessorState struct {
@@ -378,6 +391,7 @@ The scanner uses three heuristics to identify GORM models (prevents false positi
3. **Embedding-based:** Struct embeds `gorm.Model`
**Non-GORM structs are ignored:**
+
- Docker container info structs
- External API response structs
- WebSocket connection tracking
@@ -386,6 +400,7 @@ The scanner uses three heuristics to identify GORM models (prevents false positi
## Performance Metrics
**Measured Performance:**
+
- **Execution Time:** 2.1 seconds (average)
- **Target:** <5 seconds per full scan
- **Performance Rating:** β **Excellent** (58% faster than requirement)
@@ -453,24 +468,28 @@ VERBOSE=1 ./scripts/scan-gorm-security.sh --report
### Common Issues
**Scanner not found:**
+
```bash
Error: ./scripts/scan-gorm-security.sh not found
Solution: Ensure script has execute permissions: chmod +x scripts/scan-gorm-security.sh
```
**Permission denied:**
+
```bash
Error: Permission denied: backend/internal/models/user.go
Solution: Check file permissions and current user access
```
**No Go files found:**
+
```bash
Warning: No Go files found in backend/
Solution: Verify you're running from project root
```
**False positive on valid code:**
+
```bash
Solution: Add suppression comment: // gorm-scanner:ignore [reason]
```
@@ -482,6 +501,7 @@ Solution: Add suppression comment: // gorm-scanner:ignore [reason]
**Cause:** Non-GORM struct incorrectly flagged
**Solution:**
+
1. Add suppression comment with reason
2. Verify struct doesn't match GORM heuristics
3. Report as enhancement if pattern needs refinement
@@ -491,6 +511,7 @@ Solution: Add suppression comment: // gorm-scanner:ignore [reason]
**Cause:** Custom MarshalJSON implementation or XML/YAML tags
**Solution:**
+
1. Manual code review for custom marshaling
2. Check for `xml:` or `yaml:` tags (not yet supported)
3. See "Known Limitations" section
@@ -500,6 +521,7 @@ Solution: Add suppression comment: // gorm-scanner:ignore [reason]
**Cause:** Large codebase or slow filesystem
**Solution:**
+
1. Run on specific directory: `cd backend && ../scripts/scan-gorm-security.sh`
2. Use incremental scanning in pre-commit (only changed files)
3. Check filesystem performance
@@ -529,6 +551,7 @@ Solution: Add suppression comment: // gorm-scanner:ignore [reason]
## Security Thresholds
**Project Standards:**
+
- **π΄ CRITICAL**: Must fix immediately (blocking)
- **π‘ HIGH**: Should fix before PR merge (warning)
- **π΅ MEDIUM**: Fix in current sprint (informational)
@@ -614,11 +637,13 @@ type ProxyHostResponse struct {
## Report Files
**Recommended Locations:**
+
- **Development:** `docs/reports/gorm-scan-YYYYMMDD.txt` (dated reports)
- **CI/CD:** `docs/reports/gorm-scan-ci.txt` (uploaded as artifact)
- **Pre-Release:** `docs/reports/gorm-scan-release.txt` (audit trail)
**Report Format:**
+
- Plain text with ANSI color codes (terminal-friendly)
- Includes severity breakdown and summary metrics
- Contains file:line references for all issues
@@ -626,6 +651,7 @@ type ProxyHostResponse struct {
**Agent Usage:**
AI agents can read saved reports instead of parsing terminal output:
+
```bash
# Generate report
.github/skills/scripts/skill-runner.sh security-scan-gorm --report docs/reports/gorm-scan.txt
diff --git a/.github/skills/security-scan-trivy.SKILL.md b/.github/skills/security-scan-trivy.SKILL.md
index 65fbec30e..96fc41c5a 100644
--- a/.github/skills/security-scan-trivy.SKILL.md
+++ b/.github/skills/security-scan-trivy.SKILL.md
@@ -116,14 +116,14 @@ Get results in JSON format for parsing:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| scanners | string | No | vuln,secret,misconfig | Comma-separated list of scanners to run |
-| format | string | No | table | Output format (table, json, sarif) |
+| --- | --- | --- | --- | --- |
+| scanners | string | No | vuln,secret,misconfig | Comma-separated list of scanners to run |
+| format | string | No | table | Output format (table, json, sarif) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| TRIVY_SEVERITY | No | CRITICAL,HIGH,MEDIUM | Severities to report |
| TRIVY_TIMEOUT | No | 10m | Maximum scan duration |
@@ -138,20 +138,26 @@ Get results in JSON format for parsing:
## Scanner Types
### Vulnerability Scanner (vuln)
+
Scans for known CVEs in:
+
- Go dependencies (go.mod)
- npm packages (package.json)
- Docker base images (Dockerfile)
### Secret Scanner (secret)
+
Detects exposed secrets:
+
- API keys
- Passwords
- Tokens
- Private keys
### Misconfiguration Scanner (misconfig)
+
Checks configuration files:
+
- Dockerfile best practices
- Kubernetes manifests
- Terraform files
@@ -167,7 +173,8 @@ Checks configuration files:
```
Output:
-```
+
+```text
2025-12-20T10:00:00Z INFO Trivy version: 0.48.0
2025-12-20T10:00:01Z INFO Scanning filesystem...
Total: 0 (CRITICAL: 0, HIGH: 0, MEDIUM: 0)
@@ -201,18 +208,21 @@ TRIVY_SEVERITY=CRITICAL .github/skills/scripts/skill-runner.sh security-scan-tri
### Common Issues
**Docker not running**:
+
```bash
Error: Cannot connect to Docker daemon
Solution: Start Docker service
```
**Network timeout**:
+
```bash
Error: Failed to download vulnerability database
Solution: Increase TRIVY_TIMEOUT or check internet connection
```
**Vulnerabilities found**:
+
```bash
Exit code: 1
Solution: Review and remediate reported vulnerabilities
@@ -241,6 +251,7 @@ Solution: Review and remediate reported vulnerabilities
## Security Thresholds
**Project Standards**:
+
- **CRITICAL**: Must fix before release (blocking)
- **HIGH**: Should fix before release (warning)
- **MEDIUM**: Fix in next release cycle (informational)
diff --git a/.github/skills/test-backend-coverage.SKILL.md b/.github/skills/test-backend-coverage.SKILL.md
index 106711894..b8413c711 100644
--- a/.github/skills/test-backend-coverage.SKILL.md
+++ b/.github/skills/test-backend-coverage.SKILL.md
@@ -126,13 +126,13 @@ For use in GitHub Actions or other CI/CD pipelines:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose test output (-v flag) |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose test output (-v flag) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| ---------- | ---------- | --------- | ------------- |
| CHARON_ENCRYPTION_KEY | No | auto-generated for test run | Backend test encryption key. If missing/invalid, an ephemeral 32-byte base64 key is generated for the run. |
| CHARON_MIN_COVERAGE | No | 85 | Minimum coverage percentage required for success |
| CPM_MIN_COVERAGE | No | 85 | Legacy name for minimum coverage (fallback) |
@@ -143,18 +143,23 @@ For use in GitHub Actions or other CI/CD pipelines:
## Outputs
### Success Exit Code
+
- **0**: All tests passed and coverage meets threshold
### Error Exit Codes
+
- **1**: Coverage below threshold or coverage file generation failed
- **Non-zero**: Tests failed or other error occurred
### Output Files
+
- **backend/coverage.txt**: Go coverage profile (text format)
### Console Output
+
Example output:
-```
+
+```text
Filtering excluded packages from coverage report...
Coverage filtering complete
total: (statements) 87.4%
@@ -180,6 +185,7 @@ export CHARON_MIN_COVERAGE=90
## Excluded Packages
The following packages are excluded from coverage analysis:
+
- `github.com/Wikid82/charon/backend/cmd/api` - API server entrypoint
- `github.com/Wikid82/charon/backend/cmd/seed` - Database seeding tool
- `github.com/Wikid82/charon/backend/internal/logger` - Logging infrastructure
@@ -192,12 +198,15 @@ The following packages are excluded from coverage analysis:
### Common Errors
#### Error: coverage file not generated by go test
+
**Solution**: Review test output for failures; fix failing tests
#### Error: go tool cover failed or timed out
+
**Solution**: Clear Go cache and re-run tests
#### Error: Coverage X% is below required Y%
+
**Solution**: Add tests for uncovered code paths or adjust threshold
## Related Skills
diff --git a/.github/skills/test-backend-unit.SKILL.md b/.github/skills/test-backend-unit.SKILL.md
index 1a2bc4f33..52964dfd1 100644
--- a/.github/skills/test-backend-unit.SKILL.md
+++ b/.github/skills/test-backend-unit.SKILL.md
@@ -104,27 +104,31 @@ For use in GitHub Actions or other CI/CD pipelines:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose test output (-v flag) |
-| package | string | No | ./... | Package pattern to test |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose test output (-v flag) |
+| package | string | No | ./... | Package pattern to test |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| CHARON_ENCRYPTION_KEY | No | auto-generated for test run | Backend test encryption key. If missing/invalid, an ephemeral 32-byte base64 key is generated for the run. |
## Outputs
### Success Exit Code
+
- **0**: All tests passed
### Error Exit Codes
+
- **Non-zero**: One or more tests failed
### Console Output
+
Example output:
-```
+
+```text
ok github.com/Wikid82/charon/backend/internal/handlers 0.523s
ok github.com/Wikid82/charon/backend/internal/models 0.189s
ok github.com/Wikid82/charon/backend/internal/services 0.742s
@@ -167,12 +171,15 @@ ok github.com/Wikid82/charon/backend/internal/services 0.742s
### Common Errors
#### Error: package not found
+
**Solution**: Verify package path is correct; run `go list ./...` to see available packages
#### Error: build failed
+
**Solution**: Fix compilation errors; run `go build ./...` to identify issues
#### Error: test timeout
+
**Solution**: Increase timeout with `-timeout` flag or fix hanging tests
## Related Skills
diff --git a/.github/skills/test-e2e-playwright-coverage.SKILL.md b/.github/skills/test-e2e-playwright-coverage.SKILL.md
index ccd3ed6b0..d8bef7757 100644
--- a/.github/skills/test-e2e-playwright-coverage.SKILL.md
+++ b/.github/skills/test-e2e-playwright-coverage.SKILL.md
@@ -75,9 +75,9 @@ Runs Playwright end-to-end tests with code coverage collection using `@bgotink/p
**IMPORTANT**: This skill starts the **Vite dev server** (not Docker) because V8 coverage requires access to source files. Running coverage against the Docker container will result in `0%` coverage.
| Mode | Base URL | Coverage Support |
-|------|----------|-----------------|
-| Docker (`localhost:8080`) | β No - Shows "Unknown% (0/0)" |
-| Vite Dev (`localhost:5173`) | β Yes - Real coverage data |
+| ------ | ---------- | ----------------- |
+| Docker (`localhost:8080`) | β No - Shows "Unknown% (0/0)" | |
+| Vite Dev (`localhost:5173`) | β Yes - Real coverage data | |
## Prerequisites
@@ -130,27 +130,30 @@ For use in GitHub Actions or other CI/CD pipelines:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| project | string | No | firefox | Browser project: chromium, firefox, webkit |
+| --- | --- | --- | --- | --- |
+| project | string | No | firefox | Browser project: chromium, firefox, webkit |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
-| PLAYWRIGHT_BASE_URL | No | http://localhost:8080 | Application URL to test against |
+| ---------- | ---------- | --------- | ------------- |
+| PLAYWRIGHT_BASE_URL | No | | Application URL to test against |
| PLAYWRIGHT_HTML_OPEN | No | never | HTML report auto-open behavior |
| CI | No | "" | Set to "true" for CI environment behavior |
## Outputs
### Success Exit Code
+
- **0**: All tests passed and coverage generated
### Error Exit Codes
+
- **1**: One or more tests failed
- **Non-zero**: Configuration or execution error
### Output Directories
+
- **coverage/e2e/**: Coverage reports (LCOV, HTML, JSON)
- `lcov.info` - LCOV format for Codecov upload
- `coverage.json` - JSON format for programmatic access
@@ -178,7 +181,7 @@ npx playwright show-report --port 9323
The skill generates coverage in multiple formats:
| Format | File | Purpose |
-|--------|------|---------|
+| -------- | ------ | --------- |
| LCOV | `coverage/e2e/lcov.info` | Codecov upload |
| HTML | `coverage/e2e/index.html` | Visual inspection |
| JSON | `coverage/e2e/coverage.json` | Programmatic access |
diff --git a/.github/skills/test-e2e-playwright-debug.SKILL.md b/.github/skills/test-e2e-playwright-debug.SKILL.md
index 03c7eb3a3..907196323 100644
--- a/.github/skills/test-e2e-playwright-debug.SKILL.md
+++ b/.github/skills/test-e2e-playwright-debug.SKILL.md
@@ -95,6 +95,7 @@ Runs Playwright E2E tests in headed/debug mode for troubleshooting. This skill p
- **Single Test Focus**: Run individual tests or test files
**Use this skill when:**
+
- Debugging failing E2E tests
- Understanding test flow and interactions
- Developing new E2E tests
@@ -169,18 +170,18 @@ Adjust the delay between actions (in milliseconds):
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| file | string | No | "" | Specific test file to run |
-| grep | string | No | "" | Filter tests by title pattern |
-| slowmo | number | No | 500 | Delay between actions (ms) |
+| ----------- | ------ | ---------- | --------- | ------------- |
+| file | string | No | "" | Specific test file to run |
+| grep | string | No | "" | Filter tests by title pattern |
+| slowmo | number | No | 500 | Delay between actions (ms) |
| inspector | boolean | No | false | Open Playwright Inspector |
-| project | string | No | chromium | Browser to use |
+| project | string | No | chromium | Browser to use |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
-| PLAYWRIGHT_BASE_URL | No | http://localhost:8080 | Application URL |
+| ---------- | ---------- | --------- | ------------- |
+| PLAYWRIGHT_BASE_URL | No | | Application URL |
| PWDEBUG | No | "" | Set to "1" for Inspector mode |
| DEBUG | No | "" | Verbose logging (e.g., "pw:api") |
@@ -189,6 +190,7 @@ Adjust the delay between actions (in milliseconds):
### Using Playwright Inspector
The Inspector provides:
+
- **Step-through Execution**: Execute one action at a time
- **Locator Playground**: Test and refine selectors
- **Call Log**: View all Playwright API calls
@@ -200,6 +202,7 @@ The Inspector provides:
```
In the Inspector:
+
1. Use **Resume** to continue to next action
2. Use **Step** to execute one action
3. Use the **Locator** tab to test selectors
@@ -228,7 +231,8 @@ DEBUG=pw:api .github/skills/scripts/skill-runner.sh test-e2e-playwright-debug
### Screenshot on Failure
Tests automatically capture screenshots on failure. Find them in:
-```
+
+```text
test-results//
βββ test-failed-1.png
βββ trace.zip
@@ -248,6 +252,7 @@ npx playwright show-trace --port 9322
```
Traces include:
+
- DOM snapshots at each step
- Network requests/responses
- Console logs
@@ -301,7 +306,7 @@ npx playwright show-trace test-results/*/trace.zip
## Test File Locations
| Path | Description |
-|------|-------------|
+| ------ | ------------- |
| `tests/` | All E2E test files |
| `tests/auth.setup.ts` | Authentication setup |
| `tests/login.spec.ts` | Login flow tests |
@@ -314,11 +319,13 @@ npx playwright show-trace test-results/*/trace.zip
### No Browser Window Opens
**Linux**: Ensure X11/Wayland display is available
+
```bash
echo $DISPLAY # Should show :0 or similar
```
**Remote/SSH**: Use X11 forwarding or VNC
+
```bash
ssh -X user@host
```
@@ -328,6 +335,7 @@ ssh -X user@host
### Test Times Out
Increase timeout for debugging:
+
```bash
# In your test file
test.setTimeout(120000); // 2 minutes
@@ -336,6 +344,7 @@ test.setTimeout(120000); // 2 minutes
### Inspector Doesn't Open
Ensure PWDEBUG is set:
+
```bash
PWDEBUG=1 npx playwright test --headed
```
@@ -343,11 +352,13 @@ PWDEBUG=1 npx playwright test --headed
### Cannot Find Test File
Check the file exists:
+
```bash
ls -la tests/*.spec.ts
```
Use relative path from tests/ directory:
+
```bash
--file=login.spec.ts # Not tests/login.spec.ts
```
@@ -355,7 +366,7 @@ Use relative path from tests/ directory:
## Common Issues and Solutions
| Issue | Solution |
-|-------|----------|
+| ------- | ---------- |
| "Target closed" | Application crashed - check container logs |
| "Element not found" | Use Inspector to verify selector |
| "Timeout exceeded" | Increase timeout or check if element is hidden |
diff --git a/.github/skills/test-e2e-playwright.SKILL.md b/.github/skills/test-e2e-playwright.SKILL.md
index d7ba43754..47cbfa3b6 100644
--- a/.github/skills/test-e2e-playwright.SKILL.md
+++ b/.github/skills/test-e2e-playwright.SKILL.md
@@ -168,29 +168,32 @@ For use in GitHub Actions or other CI/CD pipelines:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| project | string | No | firefox | Browser project: chromium, firefox, webkit, all |
-| headed | boolean | No | false | Run with visible browser window |
-| grep | string | No | "" | Filter tests by title pattern (regex) |
+| --- | --- | --- | --- | --- |
+| project | string | No | firefox | Browser project: chromium, firefox, webkit, all |
+| headed | boolean | No | false | Run with visible browser window |
+| grep | string | No | "" | Filter tests by title pattern (regex) |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
-| PLAYWRIGHT_BASE_URL | No | http://localhost:8080 | Application URL to test against |
+| ---------- | ---------- | --------- | ------------- |
+| PLAYWRIGHT_BASE_URL | No | | Application URL to test against |
| PLAYWRIGHT_HTML_OPEN | No | never | HTML report auto-open behavior |
| CI | No | "" | Set to "true" for CI environment behavior |
## Outputs
### Success Exit Code
+
- **0**: All tests passed
### Error Exit Codes
+
- **1**: One or more tests failed
- **Non-zero**: Configuration or execution error
### Output Directories
+
- **playwright-report/**: HTML report with test results and traces
- **test-results/**: Test artifacts, screenshots, and trace files
@@ -238,7 +241,7 @@ Open `playwright-report/index.html` directly in a browser.
Tests are located in the `tests/` directory and follow Playwright conventions:
-```
+```text
tests/
βββ auth.setup.ts # Authentication setup (runs first)
βββ dashboard.spec.ts # Dashboard tests
@@ -252,22 +255,29 @@ tests/
### Common Errors
#### Error: Target page, context or browser has been closed
+
**Solution**: Ensure the application is running at the configured base URL. Rebuild if needed:
+
```bash
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e
```
#### Error: page.goto: net::ERR_CONNECTION_REFUSED
+
**Solution**: Start the Charon application before running tests:
+
```bash
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e
```
#### Error: browserType.launch: Executable doesn't exist
+
**Solution**: Run `npx playwright install` to install browser binaries
#### Error: Timeout waiting for selector
+
**Solution**: The application may be slow or in an unexpected state. Try:
+
```bash
# Rebuild with clean state
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e --clean
@@ -277,7 +287,9 @@ tests/
```
#### Error: Authentication state is stale
+
**Solution**: Remove stored auth and let setup recreate it:
+
```bash
rm -rf playwright/.auth/user.json
.github/skills/scripts/skill-runner.sh test-e2e-playwright
@@ -288,27 +300,32 @@ rm -rf playwright/.auth/user.json
When E2E tests fail, follow this workflow:
1. **Check container health**:
+
```bash
docker ps --filter "name=charon-playwright"
docker logs charon-playwright --tail 50
```
2. **Verify the application is accessible**:
+
```bash
curl -sf http://localhost:8080/api/v1/health
```
3. **Rebuild with clean state if needed**:
+
```bash
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e --clean
```
4. **Debug specific failing test**:
+
```bash
.github/skills/scripts/skill-runner.sh test-e2e-playwright-debug --grep="test name"
```
5. **View the HTML report for details**:
+
```bash
npx playwright show-report --port 9323
```
@@ -316,7 +333,7 @@ When E2E tests fail, follow this workflow:
## Key File Locations
| Path | Purpose |
-|------|---------|
+| ------ | --------- |
| `tests/` | All E2E test files |
| `tests/auth.setup.ts` | Authentication setup fixture |
| `playwright.config.js` | Playwright configuration |
diff --git a/.github/skills/test-frontend-coverage.SKILL.md b/.github/skills/test-frontend-coverage.SKILL.md
index 8d3f90a02..34d0596e6 100644
--- a/.github/skills/test-frontend-coverage.SKILL.md
+++ b/.github/skills/test-frontend-coverage.SKILL.md
@@ -110,32 +110,37 @@ For use in GitHub Actions or other CI/CD pipelines:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| verbose | boolean | No | false | Enable verbose test output |
+| --- | --- | --- | --- | --- |
+| verbose | boolean | No | false | Enable verbose test output |
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| CHARON_MIN_COVERAGE | No | 85 | Minimum coverage percentage required for success |
| CPM_MIN_COVERAGE | No | 85 | Legacy name for minimum coverage (fallback) |
## Outputs
### Success Exit Code
+
- **0**: All tests passed and coverage meets threshold
### Error Exit Codes
+
- **1**: Coverage below threshold or coverage file generation failed
- **Non-zero**: Tests failed or other error occurred
### Output Files
+
- **frontend/coverage/coverage-summary.json**: Vitest coverage summary (JSON format)
- **frontend/coverage/index.html**: HTML coverage report (viewable in browser)
### Console Output
+
Example output:
-```
+
+```text
Computed frontend coverage: 87.5% (minimum required 85%)
Frontend coverage requirement met
```
@@ -168,12 +173,15 @@ xdg-open frontend/coverage/index.html # Linux
### Common Errors
#### Error: Coverage summary file not found
+
**Solution**: Check that Vitest is configured with `--coverage` and `--reporter=json-summary`
#### Error: Frontend coverage X% is below required Y%
+
**Solution**: Add tests for uncovered components or adjust threshold
#### Error: npm ci failed
+
**Solution**: Clear node_modules and package-lock.json, then reinstall dependencies
## Related Skills
diff --git a/.github/skills/test-frontend-unit.SKILL.md b/.github/skills/test-frontend-unit.SKILL.md
index 54e82e25c..1b30e45c2 100644
--- a/.github/skills/test-frontend-unit.SKILL.md
+++ b/.github/skills/test-frontend-unit.SKILL.md
@@ -103,9 +103,9 @@ For use in GitHub Actions or other CI/CD pipelines:
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| watch | boolean | No | false | Run tests in watch mode |
-| filter | string | No | "" | Filter tests by name pattern |
+| --- | --- | --- | --- | --- |
+| watch | boolean | No | false | Run tests in watch mode |
+| filter | string | No | "" | Filter tests by name pattern |
## Environment Variables
@@ -114,14 +114,18 @@ No environment variables are required for this skill.
## Outputs
### Success Exit Code
+
- **0**: All tests passed
### Error Exit Codes
+
- **Non-zero**: One or more tests failed
### Console Output
+
Example output:
-```
+
+```text
β src/components/Button.test.tsx (3)
β src/utils/helpers.test.ts (5)
β src/hooks/useAuth.test.ts (4)
@@ -167,12 +171,15 @@ Test Files 3 passed (3)
### Common Errors
#### Error: Cannot find module
+
**Solution**: Run `npm install` to ensure all dependencies are installed
#### Error: Test timeout
+
**Solution**: Increase timeout in vitest.config.ts or fix hanging async tests
#### Error: Unexpected token
+
**Solution**: Check for syntax errors in test files
## Related Skills
diff --git a/.github/skills/utility-bump-beta.SKILL.md b/.github/skills/utility-bump-beta.SKILL.md
index 365e155ed..f62921b00 100644
--- a/.github/skills/utility-bump-beta.SKILL.md
+++ b/.github/skills/utility-bump-beta.SKILL.md
@@ -90,7 +90,7 @@ This skill requires no environment variables.
### Output Example
-```
+```text
Starting Beta Version Bump...
Current Version: 0.3.0-beta.2
New Version: 0.3.0-beta.3
@@ -108,18 +108,21 @@ Remember to push: git push origin feature/beta-release --tags
### Current Version is Beta (x.y.z-beta.N)
Increments the beta number:
+
- `0.3.0-beta.2` β `0.3.0-beta.3`
- `1.0.0-beta.5` β `1.0.0-beta.6`
### Current Version is Plain Semver (x.y.z)
Bumps minor version and starts beta.1:
+
- `0.3.0` β `0.4.0-beta.1`
- `1.2.0` β `1.3.0-beta.1`
### Current Version is Alpha or Unrecognized
Defaults to safe fallback:
+
- `0.3.0-alpha` β `0.3.0-beta.1`
- `invalid-version` β `0.3.0-beta.1`
@@ -157,7 +160,7 @@ git push origin feature/beta-release --tags
After updating files, the script prompts:
-```
+```text
Do you want to commit and tag this version? (y/n)
```
diff --git a/.github/skills/utility-clear-go-cache.SKILL.md b/.github/skills/utility-clear-go-cache.SKILL.md
index 69cd76820..f20752041 100644
--- a/.github/skills/utility-clear-go-cache.SKILL.md
+++ b/.github/skills/utility-clear-go-cache.SKILL.md
@@ -76,7 +76,7 @@ This skill accepts no parameters.
## Environment Variables
| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
+| --- | --- | --- | --- |
| XDG_CACHE_HOME | No | $HOME/.cache | XDG cache directory location |
## Outputs
@@ -87,7 +87,7 @@ This skill accepts no parameters.
### Output Example
-```
+```text
Clearing Go build and module caches...
Clearing gopls cache...
Re-downloading modules...
@@ -148,6 +148,7 @@ This skill clears the following:
## When to Use This Skill
Use this skill when experiencing:
+
- Build failures after dependency updates
- gopls crashes or incorrect diagnostics
- Module checksum mismatches
diff --git a/.github/skills/utility-db-recovery.SKILL.md b/.github/skills/utility-db-recovery.SKILL.md
index 8a3857e06..9cf5efb59 100644
--- a/.github/skills/utility-db-recovery.SKILL.md
+++ b/.github/skills/utility-db-recovery.SKILL.md
@@ -85,9 +85,9 @@ Use the task: **Utility: Database Recovery**
## Parameters
| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| --force | flag | No | false | Skip confirmation prompts |
-| -f | flag | No | false | Alias for --force |
+| --- | --- | --- | --- | --- |
+| --force | flag | No | false | Skip confirmation prompts |
+| -f | flag | No | false | Alias for --force |
## Environment Variables
@@ -103,7 +103,7 @@ This skill requires no environment variables. It auto-detects Docker vs local en
### Success Output Example (Healthy Database)
-```
+```text
==============================================
Charon Database Recovery Tool
==============================================
@@ -133,7 +133,7 @@ ok
### Recovery Output Example (Corrupted Database)
-```
+```text
==============================================
Integrity Check Results
==============================================
@@ -195,6 +195,7 @@ When corruption is detected, the recovery process:
## When to Use This Skill
Use this skill when:
+
- Application fails to start with database errors
- SQLite reports "database disk image is malformed"
- Random crashes or data inconsistencies
@@ -212,6 +213,7 @@ Use this skill when:
## WAL Mode
The skill ensures Write-Ahead Logging (WAL) is enabled:
+
- **Benefits**: Better concurrency, atomic commits, crash resistance
- **Trade-offs**: Multiple files (db, wal, shm) instead of single file
- **Recommended**: For all production deployments
diff --git a/.markdownlint.jsonc b/.markdownlint.jsonc
index acd50bc43..760aae3d2 100644
--- a/.markdownlint.jsonc
+++ b/.markdownlint.jsonc
@@ -2,20 +2,22 @@
// `make lint-markdown`, and `npm run lint:md`.
{
"default": true,
- // TODO: the rules set to false below fire on pre-existing prose and are off so a commit touching an
- // older doc does not fail on lines it did not write. Re-enable each one (delete its entry, which
- // restores the default) once the repo-wide markdown cleanup lands. MD013 and MD040 were the loudest.
- "MD001": false, // heading increment (pre-existing hit in CONTRIBUTING.md)
- "MD013": false, // line length (was: line_length 120, heading_line_length 120, code_block_line_length 150, tables false)
- "MD025": false, // multiple H1s (front-matter title + "# Title" in docs-site pages)
- "MD036": false, // emphasis used as a heading
- "MD040": false, // fenced code language
- "MD060": false, // table column style
+ // Rules turned off below stay off on purpose; each entry says why.
+ "MD013": false, // line length: rewrapping prose is high churn for little value, and long table rows and URLs are unavoidable
+ "MD036": false, // bold lead-in labels ("Who", "When", "Step 3: ...") are deliberate captions, not missing headings
+ "MD025": {
+ "front_matter_title": "" // docs-site pages carry a front-matter title and a matching "# Title"
+ },
+ "MD010": {
+ "code_blocks": false // tabs are idiomatic in Go/Makefile snippets
+ },
+ "MD029": false, // numbered steps intentionally continue across interleaved code blocks (the rendered numbers are correct)
+ "MD028": false, // adjacent callout blockquotes are separated by a blank line on purpose
"MD024": {
"siblings_only": true
},
"MD033": {
- "allowed_elements": ["details", "summary", "br", "sup", "sub", "kbd", "img"]
+ "allowed_elements": ["details", "summary", "br", "sup", "sub", "kbd", "img", "p", "a", "h1", "strong", "em"]
},
"MD041": false,
"MD046": {
diff --git a/.markdownlintignore b/.markdownlintignore
index e39f9a4c5..cfdd7c04c 100644
--- a/.markdownlintignore
+++ b/.markdownlintignore
@@ -2,9 +2,15 @@
docs/reports/
docs/implementation/
docs/issues/
+docs/patches/
docs/plans/archive/
backend/
CODEQL_*.md
COVERAGE_*.md
SECURITY_REMEDIATION_COMPLETE.md
ISSUE_*.md
+
+# Claude Code instruction files: agent/command definitions use XML-style tags and are
+# edited deliberately, so they are not linted.
+CLAUDE.md
+.claude/
diff --git a/.vibe-wise/profile.md b/.vibe-wise/profile.md
index dc5ad4584..d2179f348 100644
--- a/.vibe-wise/profile.md
+++ b/.vibe-wise/profile.md
@@ -4,30 +4,36 @@ Learning mode: active
Onboarding: complete
## Project
+
Situation: Existing
Building: Charon, self-hosted reverse proxy manager (Go + React/TS)
Codebase familiarity: Know it well
Learning scope: entire system
## Experience
+
Overall programming: Beginner
Stack familiarity: Intermediate (Go/Gin/GORM, React/TS, Docker/Caddy, self-reported)
## Goals
+
Primary: Understand the whole system end to end (default for beginner; changeable)
Capability goal: Not specified
## Preferences
+
Checkpoint frequency: Normal
Question style: Open-ended
Implementation style: AI writes code
## Strong Concepts
+
No demonstrated understanding recorded yet.
## Developing Concepts
+
None recorded yet.
## Revisit
-None recorded yet.
+None recorded yet.
diff --git a/.vibe-wise/project-map.md b/.vibe-wise/project-map.md
index 703fd230f..d577b1b05 100644
--- a/.vibe-wise/project-map.md
+++ b/.vibe-wise/project-map.md
@@ -1,12 +1,15 @@
# Project Map
## Purpose
+
Charon: self-hosted web app for managing reverse proxy host configs (novice-friendly), shipped as one binary + static assets (per CLAUDE.md).
## Requirements
+
Simplicity, usability, reliability, security; no external dependencies (per CLAUDE.md).
## Components
+
- backend/ β Go (module github.com/Wikid82/charon/backend); cmd/api entry, internal/ (config, server, models, api/routes)
- frontend/ β React 19 + Vite + TanStack Query, TypeScript
- agent/, plugins/, configs/ β not yet inspected
@@ -14,13 +17,17 @@ Simplicity, usability, reliability, security; no external dependencies (per CLAU
- tests/, playwright* β E2E tests
## Main Flow
+
Browser (React) --API--> Gin handlers (internal/api) --> services? --> GORM/SQLite (internal/models). Details unverified beyond CLAUDE.md.
## Data and Trust Boundaries
+
SQLite via GORM per CLAUDE.md; auth details unknown (unverified).
## Build and Deployment
+
Backend: `cd backend && go run ./cmd/api`, `go test ./...`. Frontend: `cd frontend && npm run build`. Dockerfile, Makefile, lefthook.yml at root.
## Unknowns
+
Service layer structure, Caddy integration, agent/ role, auth model.
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 2929cdaff..6323b9b61 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -96,7 +96,7 @@ graph TB
### Component Communication
| Source | Target | Protocol | Purpose |
-|--------|--------|----------|---------|
+| -------- | -------- | ---------- | --------- |
| Frontend | Backend | HTTP/1.1 | REST API calls for CRUD operations |
| Frontend | Backend | WebSocket | Real-time log streaming |
| Backend | Caddy | HTTP/JSON | Dynamic configuration updates |
@@ -122,7 +122,7 @@ graph TB
### Backend
| Component | Technology | Version | Purpose |
-|-----------|-----------|---------|---------|
+| ----------- | ----------- | --------- | --------- |
| **Language** | Go | 1.26.0 | Primary backend language |
| **HTTP Framework** | Gin | Latest | Routing, middleware, HTTP handling |
| **Database** | SQLite | 3.x | Embedded database |
@@ -143,7 +143,7 @@ graph TB
### Frontend
| Component | Technology | Version | Purpose |
-|-----------|-----------|---------|---------|
+| ----------- | ----------- | --------- | --------- |
| **Framework** | React | 19.2.3 | UI framework |
| **Language** | TypeScript | 6.x | Type-safe JavaScript |
| **Build Tool** | Vite | 8.0.0-beta.18 | Fast bundler and dev server |
@@ -159,7 +159,7 @@ graph TB
### Infrastructure
| Component | Technology | Version | Purpose |
-|-----------|-----------|---------|---------|
+| ----------- | ----------- | --------- | --------- |
| **Containerization** | Docker | 24+ | Application packaging |
| **Base Image** | Debian Trixie Slim | Latest | Security-hardened base |
| **CI/CD** | GitHub Actions | N/A | Automated testing and deployment |
@@ -173,7 +173,7 @@ graph TB
## Directory Structure
-```
+```text
/projects/Charon/
βββ backend/ # Go backend source code
β βββ cmd/ # Application entrypoints
@@ -398,7 +398,7 @@ The stats subsystem collects, aggregates, and broadcasts request metrics for the
**API Endpoints** (all require JWT authentication, mounted under `/api/stats/`):
| Method | Path | Description |
-|--------|------|-------------|
+| -------- | ------ | ------------- |
| `GET` | `/summary` | 24 h / 7 d / 30 d request counts |
| `GET` | `/top-hosts` | Top hosts by request count (`period`, `limit`) |
| `GET` | `/status-distribution` | HTTP status code breakdown (`period`) |
@@ -504,7 +504,7 @@ ticker.
`POST /api/v1/settings` and the SystemSettings "Uptime Monitoring" card):
| Key | Default | Bounds | Hot-reload |
-|-----|---------|--------|-----------|
+| ----- | --------- | -------- | ----------- |
| `uptime.default_interval_seconds` | 60 | 30 β 86400 | Yes (~60 s TTL) |
| `uptime.worker_pool_size` | 30 | 1 β 200 | No β restart to apply |
| `uptime.heartbeat_retention_days` | 30 | 1 β 3650 | Yes (read each pruner pass) |
@@ -512,7 +512,7 @@ ticker.
**API Endpoints** (JWT auth, mounted in the `management` group):
| Method | Path | Description |
-|--------|------|-------------|
+| -------- | ------ | ------------- |
| `GET` | `/api/v1/uptime/monitors/summary` | Per-monitor status, latency, last check, 24 h uptime %, and a `recent_beats` sparkline series β one response, 30 s cached |
| `GET` | `/api/v1/uptime/monitors/:id/history` | Detailed heartbeat history for one monitor β `limit` (β€ 500), `before` cursor |
| `GET` | `/api/v1/uptime/health` | `heartbeats_dropped`, `checks_enqueue_dropped`, `queue_depth`, `worker_pool_size` |
@@ -536,7 +536,7 @@ Format-v2 backup archives, a validated safe-restore pipeline, optional archive e
**API Endpoints** (mounted under the existing `management` group in `internal/api/routes/routes.go`; mutating routes additionally require admin):
| Method | Path | Description |
-|--------|------|-------------|
+| -------- | ------ | ------------- |
| `GET` | `/api/v1/backups` | List backups (DB-backed, reconciled with filesystem) |
| `POST` | `/api/v1/backups` | Start a backup job β `202 Accepted` + `job_id` (admin) |
| `DELETE` | `/api/v1/backups/:filename` | Delete backup (admin) |
@@ -1007,7 +1007,7 @@ pin (the stage already carries ~40 such pins).
**Route-group boundary** (`backend/internal/api/routes/routes.go`):
| Group | Middleware chain | Who reaches it |
-|-------|------------------|----------------|
+| ------- | ------------------ | ---------------- |
| `protected` | `AuthMiddleware` | any authenticated caller |
| `management` | `+ RequireManagementAccess()` | `admin` and `user` (rejects `passthrough`) |
| `managementAdmin` | `+ RequireRole(models.RoleAdmin)` | `admin` only |
@@ -1095,7 +1095,7 @@ Charon operates with **two distinct traffic flows** on separate ports, each with
**Traffic Separation Example:**
-```
+```text
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Charon Container β
β β
@@ -1357,7 +1357,7 @@ Operator runbook: `docs/ci/toolchain-image.md`.
### Port Mapping
| Port | Protocol | Purpose | Bind |
-|------|----------|---------|------|
+| ------ | ---------- | --------- | ------ |
| 8080 | HTTP | Web UI + REST API | 0.0.0.0 |
| 80 | HTTP | Caddy reverse proxy | 0.0.0.0 |
| 443 | HTTPS | Caddy reverse proxy (TLS) | 0.0.0.0 |
@@ -1367,14 +1367,14 @@ Operator runbook: `docs/ci/toolchain-image.md`.
### Volume Mounts
| Container Path | Purpose | Required |
-|----------------|---------|----------|
+| --- | --- | --- |
| `/app/data` | Database, certificates, backups | **Yes** |
| `/var/run/docker.sock` | Docker container discovery | Optional |
### Environment Variables
| Variable | Purpose | Default | Required |
-|----------|---------|---------|----------|
+| ---------- | --------- | --------- | ---------- |
| `CHARON_ENV` | Environment (production/development) | `production` | No |
| `CHARON_ENCRYPTION_KEY` | 32-byte base64 key for credential encryption | Auto-generated | No |
| `CHARON_EMERGENCY_TOKEN` | 64-char hex for break-glass access | None | Optional |
@@ -1520,7 +1520,7 @@ See "Branch Promotion Chain" below for how changes flow `main` β `development`
**Example:**
-```
+```text
feat: add DNS-01 challenge support for Cloudflare
Implement Cloudflare DNS provider for automatic wildcard certificate
@@ -1605,7 +1605,7 @@ deliberate, not an inconsistency to "fix" into one uniform pipeline:
### Test Pyramid
-```
+```text
/\ E2E (Playwright) - 10%
/ \ Critical user flows
/____\
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 334f7b3ac..9ee9a4b62 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -113,7 +113,7 @@ When the project's Go version is updated (usually by Renovate):
Development tools like golangci-lint and gopls are compiled programs. When you upgrade Go, these tools still run on the old version and will break with errors like:
-```
+```text
error: some/file.go:123:4: undefined: runtime.NewlyAddedFunction
```
@@ -123,7 +123,7 @@ Rebuilding tools with `./scripts/rebuild-go-tools.sh` fixes this by compiling th
Don't worry! The lefthook pre-commit hook will detect the version mismatch and automatically rebuild tools for you. You'll see:
-```
+```text
β οΈ golangci-lint Go version mismatch:
golangci-lint: 2.13.0
system Go: 1.26.0
@@ -183,7 +183,7 @@ npm run dev # Start frontend dev server
The project uses a three-tier branching model:
-```
+```text
development β nightly β main
(unstable) (testing) (stable)
```
@@ -217,7 +217,7 @@ git checkout -b feature/your-feature-name
Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification:
-```
+```text
():
@@ -237,7 +237,7 @@ Follow the [Conventional Commits](https://www.conventionalcommits.org/) specific
**Examples:**
-```
+```text
feat(proxy-hosts): add SSL certificate upload
- Implement certificate upload endpoint
@@ -247,7 +247,7 @@ feat(proxy-hosts): add SSL certificate upload
Closes #123
```
-```
+```text
fix(import): resolve conflict detection bug
When importing Caddyfiles with multiple domains, conflicts
@@ -872,7 +872,7 @@ mkdir -p .github/skills/{skill-name}-scripts
Use the template structure:
-```markdown
+````markdown
---
# agentskills.io specification v1.0
name: "skill-name"
@@ -942,7 +942,7 @@ command example
**Maintained by**: Charon Project
**Source**: Original implementation or script path
-```
+````
#### 4. Create the Execution Script
diff --git a/README.md b/README.md
index 1be489139..08c41caf7 100644
--- a/README.md
+++ b/README.md
@@ -206,16 +206,16 @@ No premium tiers. No feature paywalls. No usage limits. Everything you see is yo
π Get Help
+ src="https://img.shields.io/github/issues/Wikid82/Charon">
---
β Support This Project
@@ -343,6 +383,7 @@ result = expensive_function(x)
```
### Example 5: Asynchronous I/O in Node.js
+
```javascript
// BAD: Blocking file read
const data = fs.readFileSync('file.txt');
@@ -355,6 +396,7 @@ fs.readFile('file.txt', (err, data) => {
```
### Example 6: Profiling a Python Function
+
```python
import cProfile
import pstats
@@ -368,6 +410,7 @@ p.sort_stats('cumulative').print_stats(10)
```
### Example 7: Using Redis for Caching in Node.js
+
```javascript
const redis = require('redis');
const client = redis.createClient();
@@ -388,6 +431,7 @@ function getCachedData(key, fetchFunction) {
---
## References and Further Reading
+
- [Google Web Fundamentals: Performance](https://web.dev/performance/)
- [MDN Web Docs: Performance](https://developer.mozilla.org/en-US/docs/Web/Performance)
- [OWASP: Performance Testing](https://owasp.org/www-project-performance-testing/)
diff --git a/.github/instructions/playwright-typescript.instructions.md b/.github/instructions/playwright-typescript.instructions.md
index e9b1b8718..6c921ad42 100644
--- a/.github/instructions/playwright-typescript.instructions.md
+++ b/.github/instructions/playwright-typescript.instructions.md
@@ -6,31 +6,32 @@ applyTo: '**'
## Test Writing Guidelines
### Code Quality Standards
+
- **Locators**: Prioritize user-facing, role-based locators (`getByRole`, `getByLabel`, `getByText`, etc.) for resilience and accessibility. Use `test.step()` to group interactions and improve test readability and reporting.
- **Assertions**: Use auto-retrying web-first assertions. These assertions start with the `await` keyword (e.g., `await expect(locator).toHaveText()`). Avoid `expect(locator).toBeVisible()` unless specifically testing for visibility changes.
- **Timeouts**: Rely on Playwright's built-in auto-waiting mechanisms. Avoid hard-coded waits or increased default timeouts.
- **Clarity**: Use descriptive test and step titles that clearly state the intent. Add comments only to explain complex logic or non-obvious interactions.
-
### Test Structure
+
- **Imports**: Start with `import { test, expect } from '@playwright/test';`.
- **Organization**: Group related tests for a feature under a `test.describe()` block.
- **Hooks**: Use `beforeEach` for setup actions common to all tests in a `describe` block (e.g., navigating to a page).
- **Titles**: Follow a clear naming convention, such as `Feature - Specific action or scenario`.
-
### File Organization
+
- **Location**: Store all test files in the `tests/` directory.
- **Naming**: Use the convention `
+ 
+