ApyGuard Validation Benchmarks is an API-focused benchmark suite for testing authenticated attack paths against intentionally vulnerable services.
Warning
This repository contains intentionally vulnerable systems and must only be used in controlled environments. Do not use, deploy, or adapt these benchmark services for production environments.
- Goals
- Prerequisites
- What This Is
- Who This Is For
- Quick Start
- Benchmark Catalog
- OWASP API Top 10 Coverage
- Project Structure
- Auth-First Design
- Using The Root Makefile
- Authoring Rules
- Documentation
- Safety
- License
- Evaluate API security testing capability instead of browser-only behavior
- Cover common OWASP-style API and application risks
- Keep each benchmark focused on one primary exploit objective
- Make benchmarks easy to build, run, and extend locally
- Docker Engine with
docker composeplugin - GNU Make
- OpenSSL (used by benchmark build logic for deterministic flag generation)
- Python 3.10+ (for
tools/benchmark_ui.py)
ApyGuard Validation Benchmarks is a structured dataset of intentionally vulnerable APIs designed to evaluate:
- API security scanners
- behavior-based anomaly detection systems
- automated vulnerability discovery tools
Unlike traditional web benchmarks, this suite focuses on authenticated API attack paths and real-world exploit chains.
- API security engineers
- red teamers and pentesters
- security tool vendors
- researchers evaluating detection systems
# 1. List all available benchmarks
make list
# 2. Build a benchmark
make build BENCHMARK=APY-001-26
# 3. Start it
make run BENCHMARK=APY-001-26
# 4. Stop it when done
make stop BENCHMARK=APY-001-26To run all benchmarks via a browser dashboard:
python tools/benchmark_ui.py
# then open http://127.0.0.1:809036 benchmarks across authorization, injection, authentication, and other vulnerability classes.
| ID | Title | Vulnerability Class |
|---|---|---|
APY-001-26 |
Order Management API | BOLA |
APY-002-26 |
Support Admin API | BFLA |
APY-006-26 |
Workspace API | Broken Object Property Level Authorization |
APY-014-26 |
BOLA to Account Takeover | Chained Exploit |
APY-015-26 |
Invoice API | Multi-Tenant BOLA |
APY-016-26 |
Orders API | Bulk Authorization Gap |
APY-017-26 |
Profile API | Method-Level Authorization Mismatch |
APY-018-26 |
Billing API | Token Scope Confusion |
APY-019-26 |
Admin API | Policy Shadowing Route Conflict |
APY-020-26 |
Audit API | Wildcard Role Matching |
APY-021-26 |
API Key Service | Delegation Token Scope Escalation |
APY-022-26 |
Projects API | Ownership Transfer Bypass |
APY-023-26 |
Records API | Soft-Deleted Object Recovery BOLA |
APY-024-26 |
Billing API | Tenant Filter Isolation Bypass |
APY-025-26 |
Reporting API | Async Job Authorization |
APY-026-26 |
Files API | Upload Metadata Privilege Abuse |
APY-027-26 |
Discovery API | Admin Route Guard Parity Gap |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-005-26 |
Reports API | SQL Injection |
APY-012-26 |
Auth API | NoSQL Injection |
APY-028-26 |
Search API | NoSQL Operator Injection |
APY-031-26 |
Rules API | Expression Injection |
APY-032-26 |
Ledger API | SQL Injection — Column Selection |
APY-033-26 |
Tools API | Command Injection in Diagnostics |
APY-034-26 |
Messaging API | Server-Side Template Injection |
APY-035-26 |
Catalog API | XXE Import |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-004-26 |
Partner API | JWT Forgery |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-003-26 |
Workspace API | Mass Assignment |
APY-007-26 |
— | Excessive Data Exposure |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-008-26 |
Coupon API | Business Flow Abuse |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-009-26 |
Integration Fetch API | SSRF |
APY-030-26 |
Integrations API | Decimal-IP SSRF Bypass |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-010-26 |
Debug API | Security Misconfiguration |
APY-011-26 |
Legacy Admin API | Improper Inventory Management |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-013-26 |
— | File Upload Abuse — Path Traversal Download |
| ID | Title | Vulnerability Class |
|---|---|---|
APY-029-26 |
Analytics API | Numeric Notation Limit Bypass |
APY-036-26 |
Projects API | Duplicate Parameter Pollution |
Coverage mapped to the OWASP API Security Top 10 (2023).
| OWASP Category | Benchmarks |
|---|---|
| API1 — Broken Object Level Authorization | APY-001, APY-014, APY-015, APY-016, APY-022, APY-023, APY-024, APY-025 |
| API2 — Broken Authentication | APY-004 |
| API3 — Broken Object Property Level Authorization | APY-006, APY-007, APY-003, APY-026 |
| API4 — Unrestricted Resource Consumption | APY-029 |
| API5 — Broken Function Level Authorization | APY-002, APY-017, APY-018, APY-019, APY-020, APY-021, APY-027 |
| API6 — Unrestricted Access to Sensitive Business Flows | APY-008 |
| API7 — Server Side Request Forgery | APY-009, APY-030 |
| API8 — Security Misconfiguration | APY-010, APY-036 |
| API9 — Improper Inventory Management | APY-011 |
| API10 — Unsafe Consumption of APIs | — |
Additional coverage beyond the OWASP Top 10:
| Class | Benchmarks |
|---|---|
| SQL Injection | APY-005, APY-032 |
| NoSQL Injection | APY-012, APY-028 |
| Expression / Template / Command Injection | APY-031, APY-033, APY-034 |
| XXE | APY-035 |
| File Upload / Path Traversal | APY-013 |
| Chained Exploits | APY-014 |
.
├── benchmarks/<BENCHMARK_ID>/ # One folder per benchmark
├── common.mk # Shared build and run logic
├── docs/ # Design notes and conventions
├── tools/ # benchmark_ui.py dashboard
└── assets/ # Logo and static assets
Each benchmark folder contains:
benchmarks/APY-001-26/
├── benchmark.yaml
├── benchmark.json
├── docker-compose.yml
├── Makefile
└── app/ # Service source code
All benchmarks follow the same base authentication workflow:
POST /api/auth/login— exchange credentials for a token- Extract
access_tokenfrom the response - Send
Authorization: Bearer <token>on all subsequent requests
This makes the evaluation flow consistent across the entire suite and keeps scenarios realistic.
See docs/auth-first-benchmark-pattern.md for full details.
| Command | Description |
|---|---|
make list |
List all available benchmarks |
make build BENCHMARK=<ID> |
Build a single benchmark |
make run BENCHMARK=<ID> |
Start a benchmark |
make stop BENCHMARK=<ID> |
Stop a running benchmark |
make build BENCHMARK=<ID> FLAG=FLAG{custom} |
Build with a custom flag |
Browser dashboard — start all benchmarks from a single UI:
python tools/benchmark_ui.pyThen open http://127.0.0.1:8090 (or http://<your-lan-ip>:8090).
The dashboard supports per-benchmark start/stop, search, filtering, bulk actions, and URL copy. Place your logo at assets/Logo.png and it will appear automatically.
- Keep each benchmark centered on one exploit objective
- Put the flag behind the vulnerability, not behind unrelated guessing
- Require authentication up front unless the benchmark is explicitly about auth bypass
- Keep the API surface small and intentional
- Use health checks so
docker compose up --waitis stable
| Document | Description |
|---|---|
| docs/apyguard-api-benchmark-cases.md | Full benchmark case descriptions |
| docs/apyguard-api-benchmark-cases.yaml | Machine-readable case definitions |
| docs/auth-first-benchmark-pattern.md | Auth pattern design reference |
| CHANGELOG.md | Version history |
| CODE_OF_CONDUCT.md | Community standards |
| SUPPORT.md | How to get help |
These benchmarks intentionally contain vulnerabilities.
- Do not deploy them in production
- Run them only in isolated lab or sandbox environments
- Treat any included secrets, credentials, and flags as test data only
This project is licensed under the Apache License 2.0. See LICENSE.