End-to-end quality assurance suite for the ShortBeyond URL shortener API (beyondtest/shortb-api:hotfix backed by PostgreSQL 15). Covers functional API contract testing with Playwright, BDD feature specifications in Gherkin, API documentation in OpenAPI/Swagger and Postman/Bruno, transactional PostgreSQL CTE data management, Artillery load/spike testing, and automated execution in GitHub Actions.
- QA Strategy & Artifacts
- Repository Structure
- Architecture & Design Decisions
- API Contracts (Swagger, Postman & Bruno)
- Running Tests Locally
- Observed Results & Benchmarks
- CI/CD Pipeline (GitHub Actions)
- Author
Rather than treating test automation in isolation, this repository documents the full testing lifecycle from planning to defect analysis:
| Artifact | Purpose | Location |
|---|---|---|
| Test Plan | Scope, risk matrix, traceability, and exit criteria | docs/qa/TEST_PLAN.md |
| Test Case Inventory | 24 mapped scenarios (20 functional + 4 performance) | docs/qa/TEST_CASES.md |
| BDD Specifications | Gherkin acceptance criteria (health, auth, links) |
specs/features/ |
| Bug Report Example | Root-cause investigation of 100 req/s spike failures | docs/qa/bug-reports/BUG-001 |
| Architecture Guide | Service objects, CTE SQL queries, and SLA definitions | docs/ARCHITECTURE_GUIDE.md |
| Performance Report | Latency percentile analysis and Artillery runbooks | performance/README.md |
.
├── .github/workflows/
│ └── ci.yml # CI pipeline (PostgreSQL 15 + API container + Playwright)
├── docs/
│ ├── auth/ # Auth endpoint specs (Bruno / OpenCollection)
│ ├── health/ # Health endpoint spec (Bruno / OpenCollection)
│ ├── links/ # Links CRUD specs (Bruno / OpenCollection)
│ ├── postman/
│ │ ├── ShortBeyond.postman_collection.json
│ │ └── ShortBeyond.local.postman_environment.json
│ ├── qa/
│ │ ├── bug-reports/
│ │ │ └── BUG-001-spike-500-and-auth-degradation.md
│ │ ├── TEST_CASES.md # Test case matrix with IDs and contract expectations
│ │ └── TEST_PLAN.md # Test strategy, risk matrix, and traceability
│ ├── swagger/
│ │ └── openapi.yaml # OpenAPI 3.0.3 API specification
│ ├── ARCHITECTURE_GUIDE.md # Technical patterns and database guide
│ └── opencollection.yml # Bruno collection entrypoint
├── performance/
│ ├── data/usuarios.csv # Seeded user dataset for load tests
│ ├── reports/ # Raw Artillery JSON reports (login, register, spike)
│ ├── tests/ # Artillery workload definitions (YAML)
│ ├── insert-users.js # Bulk DB seeding script entrypoint
│ └── README.md # Load and spike test documentation
├── playwright/
│ ├── e2e/ # API contract specs (auth, health, links)
│ └── support/
│ ├── factories/ # Faker + ULID data generators
│ ├── fixtures/ # Playwright custom fixtures (auth, links)
│ └── services/ # HTTP service clients & PostgreSQL CTE controller
├── specs/features/ # BDD scenarios written in Gherkin (.feature)
├── global-setup.js # Pre-suite DB sanitization via SQL CTE
├── playwright.config.js # Runner config (API project, retries, HTML reporter)
├── shortbeyond.yaml # Pod specification (postgres:15, adminer, shortb-api)
└── package.json
- API-First Playwright Runner (
APIRequestContext):playwright.config.jsconfigures only theapi-testsproject without browser binaries, reducing execution time and eliminating UI flakiness while keeping Playwright's fixtures, assertions, and HTML reporting. - Service Objects & Dependency Injection: HTTP interactions live in
playwright/support/services/(auth.js,links.js) and are injected into tests through custom fixtures (playwright/support/fixtures/index.js). - Collision-Free Factories:
user.jsandutils.jscombine@faker-js/fakerwith 26-character Crockford ULIDs and tag generated test emails with@lfernandobra.devfor safe bulk cleanup. - Transactional PostgreSQL Cleanup & Seeding (
database.js):global-setup.jsinvokescleanupTestData()before every test run using a SQL Common Table Expression (CTE) to remove testlinks(including orphaned rows) andusersatomically.insertTestUsers()computes a single Bcrypt hash (cost 10) upfront and inserts 500-row batches directly into PostgreSQL, seeding 2,000–4,000 test accounts in under 1 second while generatingperformance/data/usuarios.csv.
The API exposes /health, /api/auth/register, /api/auth/login, and /api/links (GET/POST/DELETE):
- OpenAPI / Swagger:
docs/swagger/openapi.yamldefines request/response schemas, JWT bearer authentication, ULID path parameters, and exact validation messages. - Postman: Import
ShortBeyond.postman_collection.jsonandShortBeyond.local.postman_environment.json. Requests includepm.testassertions and auto-captureauthTokenandlinkIdfor sequential Collection Runner execution. - Bruno (Git-native): Endpoint YAML files remain versioned under
docs/auth/,docs/health/, anddocs/links/.
- Node.js
>= 18.x - Running PostgreSQL 15 (
ShortDB) andbeyondtest/shortb-api:hotfixcontainers (seeshortbeyond.yaml)
# 1. Install dependencies
npm ci
# 2. Configure local environment variables
cp .env.example .env
# 3. Run Playwright API test suite
npm test
# 4. Open Playwright HTML report
npm run test:report# Reset test records and seed 2,000 users into PostgreSQL + CSV
node playwright/support/services/database.js
# Run authentication workload test (4 req/s for 60s)
npm run test:perf
# Run 5-phase traffic spike test (up to 100 req/s)
npm run test:spike- 20 automated API test cases across
health.spec.js,auth/register.spec.js,auth/login.spec.js,links/post.spec.js,links/get.spec.js, andlinks/delete.spec.js. - Validates HTTP status codes, JSON payload schemas, password non-exposure, 5-char
short_coderegex (^([a-zA-Z0-9_-]){5}$), and Portuguese API validation messages.
Metrics captured directly in performance/reports/:
| Test Suite | Workload Profile | Error Rate | p95 Latency | Outcome |
|---|---|---|---|---|
health.yaml |
5 req/s for 30s | 0% | 13.9 ms |
Passed (target < 50 ms) |
register.yaml |
3 req/s for 45s (80% 201 / 20% 400) | 0% unexpected | 100.5 ms |
Passed (target < 500 ms) |
login.yaml |
4 req/s for 60s (70% 200 / 30% 401) | 0% unexpected | 94.6 ms |
Passed (target < 400 ms) |
spike.yaml |
5 phases up to 100 req/s (4,000 VUs) | 162 HTTP 500s, 3 timeouts |
6,312.2 ms (peak) |
Degraded under peak; recovered to 89.1 ms |
Detailed bucket-by-bucket spike analysis and root-cause hypotheses are documented in BUG-001.
Workflow ci.yml runs on every push and pull_request to main:
- Spins up
postgres:15andbeyondtest/shortb-api:hotfixcontainers onubuntu-latest. - Waits for
pg_isreadyandGET /healthreadiness checks. - Validates Postman JSON integrity and executes
npm test(withretries: 2,workers: 1,forbidOnly: true). - Dumps container logs on failure and archives
playwright-report/for 14 days.
Luis Fernando Brandão
QA Automation Engineer