feat: GitHub Actions + Cursor Automation 자동 improvement loop#11
feat: GitHub Actions + Cursor Automation 자동 improvement loop#11dtype2100 wants to merge 3 commits into
Conversation
- Split liveness/readiness health endpoints for Kubernetes and Docker - Enable optional API key auth on data endpoints via API_KEY env var - Add Prometheus metrics, request ID tracing, rate limiting, security headers - Register global exception handlers with request_id in error responses - Add GitHub Actions CI workflow and fix Makefile to use python -m - Add Docker healthcheck and curl in Dockerfile - Add integration tests for ops endpoints (73 tests passing) Co-authored-by: jwlee <dtype2100@users.noreply.github.com>
…→feedback loop - Define canonical improvement loop in app/core/improvement_loop.py - Restructure LangGraph: wire run_judge, add test_retrieval phase - Add unified route_after_feedback router with retry_retrieval/generation paths - Reorder run_evals.py to match loop phases; add run_feedback_eval.py - Add unit tests for feedback routing (78 tests passing) Co-authored-by: jwlee <dtype2100@users.noreply.github.com>
…ion) - Add improvement-loop.yml workflow (daily schedule, push to master, manual dispatch) - Add make evals-ci and --ci flag to skip LLM-dependent judge phase in CI - Add iter_eval_phases() and requires_llm flag in improvement_loop.py - Add Cursor Automation prompt template in .cursor/automation/improvement-loop.md - Document automation setup in README Co-authored-by: jwlee <dtype2100@users.noreply.github.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces robust production operations, security features, and the canonical "improvement loop" structure to the Advanced-RAG pipeline. Key changes include API key authentication, CORS configuration, rate limiting, security headers, Prometheus metrics, and Kubernetes liveness/readiness probes. The feedback identifies several critical security and stability improvements: resolving a CORS vulnerability when using wildcard origins with credentials, preventing timing attacks on API key validation via constant-time comparison, correctly parsing client IPs behind reverse proxies for rate limiting, addressing a potential memory leak in the in-memory rate limiter, and safeguarding against a TypeError in the CRAG state finalization.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| app.add_middleware( | ||
| CORSMiddleware, | ||
| allow_origins=get_cors_origins(), | ||
| allow_credentials=True, | ||
| allow_methods=["*"], | ||
| allow_headers=["*"], | ||
| ) |
There was a problem hiding this comment.
When allow_credentials is set to True, using a wildcard * in allow_origins is a severe security risk. Starlette's CORSMiddleware will dynamically reflect the request's Origin header in the response, allowing any website to make authenticated requests (with cookies/credentials) to your API. To prevent this vulnerability, we should set allow_credentials to False if * is present in the allowed origins, or explicitly specify the allowed origins.
| app.add_middleware( | |
| CORSMiddleware, | |
| allow_origins=get_cors_origins(), | |
| allow_credentials=True, | |
| allow_methods=["*"], | |
| allow_headers=["*"], | |
| ) | |
| origins = get_cors_origins() | |
| app.add_middleware( | |
| CORSMiddleware, | |
| allow_origins=origins, | |
| allow_credentials="*" not in origins, | |
| allow_methods=["*"], | |
| allow_headers=["*"], | |
| ) |
| if self._limit <= 0 or request.url.path in _EXEMPT_PATHS: | ||
| return await call_next(request) | ||
|
|
||
| client_ip = request.client.host if request.client else "unknown" |
There was a problem hiding this comment.
If the application is deployed behind a reverse proxy (such as Nginx, Traefik, or Cloudflare), request.client.host will always return the IP of the proxy rather than the actual client. This will cause all users to share the same rate-limiting bucket, leading to premature rate-limiting of legitimate traffic. To resolve this, check the X-Forwarded-For or X-Real-IP headers first.
| client_ip = request.client.host if request.client else "unknown" | |
| client_ip = request.headers.get("X-Forwarded-For") or (request.client.host if request.client else "unknown") | |
| if client_ip and "," in client_ip: | |
| client_ip = client_ip.split(",")[0].strip() |
| def __init__(self, app, limit_per_minute: int) -> None: | ||
| super().__init__(app) | ||
| self._limit = max(limit_per_minute, 0) | ||
| self._windows: dict[str, deque[float]] = defaultdict(deque) |
There was a problem hiding this comment.
The self._windows dictionary grows indefinitely as new client IPs connect, because there is no mechanism to clean up inactive IPs. In a production environment with many unique clients, this will lead to unbounded memory growth (memory leak). Consider periodically purging old keys, using a size-limited cache (e.g., cachetools.TTLCache), or leveraging Redis (which is already configured in the project) for a robust distributed rate limiter.
| if api_key != settings.api_key: | ||
| raise HTTPException(status_code=401, detail="Invalid or missing API key") |
There was a problem hiding this comment.
Using standard string comparison (!=) for API key validation is vulnerable to timing attacks, allowing an attacker to potentially brute-force the key character-by-character by measuring response times. Use secrets.compare_digest to perform a constant-time comparison.
| if api_key != settings.api_key: | |
| raise HTTPException(status_code=401, detail="Invalid or missing API key") | |
| import secrets | |
| if not secrets.compare_digest(api_key or "", settings.api_key): | |
| raise HTTPException(status_code=401, detail="Invalid or missing API key") |
|
|
||
| def finalize_rejected(state: CRAGState) -> dict: | ||
| """Mark the pipeline as rejected after exhausting feedback retries.""" | ||
| answer = state.get("answer", "") |
There was a problem hiding this comment.
If the "answer" key exists in the state but its value is None, state.get("answer", "") will return None. This will cause a TypeError on the subsequent line when checking suffix not in answer. Use state.get("answer") or "" to safely default to an empty string in all cases.
| answer = state.get("answer", "") | |
| answer = state.get("answer") or "" |
2 participants
Summary
자동 improvement loop를 GitHub Actions + Cursor Automation 템플릿으로 설정했습니다.
GitHub Actions (
.github/workflows/improvement-loop.yml)make evals-cifull_loop=true시make evals(OPENAI_API_KEY secret 필요)CI-safe loop
LLM 없이 돌아가는 subset: analysis → search → test → evaluation → feedback
Cursor Cloud Automation
Dashboard → Automations에 붙일 프롬프트:
.cursor/automation/improvement-loop.md