Repository navigation
Conversation
Add the Studio 10 assignment: students attack a course-provided AgentDojo Slack agent with their own injection, exfiltration and unauthorized-action attacks, crack it, climb a model ladder (luna, 5.4-mini, 5.5, sol; terra is a leaderboard challenge), then add one defense and measure what it costs. Files: README, instruction, grading rubric, starter kit (redteam package, attack and defense templates, one benign sample), submission template and .gitignore; one entry in studio/README.md. Checkpoints: 1 Crack the target (3 lines), 2 Climb the ladder (3 lines), 3 Defend the target (2 lines), plus the explanation (2). 10 points. Generated with [Claude Code](https://claude.ai/code) via [HappyHerd](https://github.com/NickGuAI/HappyHerd) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Defense report records the per-attack rows in both the baseline and defended blocks, same ids in the same order, so a grader can confirm the two conditions ran the identical attack set over the full 21 tasks (run_defense). - --model-id override must start with the alias's canonical id, so an easy model can never be recorded under a harder alias (models.resolve_model). - Rubric: the luna and ladder cracks earn points; cracking terra and extra defense blocking stay ungraded. 3.1 names the per-attack rows and the 21 tasks. - README documents both behaviors. Generated with [Claude Code](https://claude.ai/code) via [HappyHerd](https://github.com/NickGuAI/HappyHerd) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- A --model-id override must equal the alias's canonical id or a dated variant (canonical + "-" + a date digit), so gpt-5.5-mini or gpt-5.5-pro can no longer stand in for gpt-5.5 (models.valid_model_id, resolve_model). - The leaderboard aggregator validates every model entry's id against its alias, so a report that labels an easy model under a harder alias is rejected. - Make the defense-report example internally consistent (trials match the shown row). - README documents the dated-variant rule. Generated with [Claude Code](https://claude.ai/code) via [HappyHerd](https://github.com/NickGuAI/HappyHerd) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the Studio 10 (Red Team Day) assignment and its starter kit.
Students attack a course-provided AgentDojo Slack agent with their own injection, exfiltration and unauthorized-action attacks, crack it, climb a model ladder, then add one defense and measure what it costs in ordinary work. The attacks and the defense are the team's; the AgentDojo suite, the target models and the success checks are fixed. The model ladder is
luna(gpt-6-luna),54mini(gpt-5.4-mini),55(gpt-5.5) andsol(gpt-6-sol);warmup(gpt-4o-mini) is setup only andterra(gpt-6.1-terra) is a leaderboard challenge. GPT-6 models run with reasoning off.Files:
studio/studio-10/README.md,instruction/Studio_Instruction.md,instruction/GRADING_RUBRIC.mdinstruction/code/: theredteampackage (harness,models,run_attacks,run_defense,leaderboard,defenses),attack_template.json,defense_template.py, one benignsamples/example_attack.json,setup.sh,requirements.txt,README.mdsubmission/README.md,submission/_template/EXPLANATION.md,.gitignorestudio/README.mdGrading, 10 points:
The grading agent reads the committed evidence JSON and does not run student code. The due date is in the CourseWorks assignment.
🤖 Generated with Claude Code