Skip to content

fix: keep dataset file names inside their base directory - #732

Draft
omkargaikwad23 wants to merge 1 commit into
mainfrom
fix/dataset-file-names-contained
Draft

omkargaikwad23 wants to merge 1 commit into
mainfrom
fix/dataset-file-names-contained

Conversation

@omkargaikwad23

@omkargaikwad23 omkargaikwad23 commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

What this fixes

Two places joined a file name from a dataset to a base directory without a check. A name with ../, or an absolute path, pointed outside the directory.

  • env_files of a containerized case. _collect_env_files read the file on the host and put its content into the case container. _materialize_env_files wrote the file in the container.
  • SQLite database name. SQLiteDB made directories and connected to the path. get_sqlite_ground_truth read the path.

How

  • New helper contained_path(base, name) in util/safepath.py. It raises ValueError if name is empty, is absolute, or is outside base after symlinks are resolved. A name in a subdirectory of base is valid.
  • _collect_env_files and _materialize_env_files skip such a name and write a log line.
  • SQLiteDB._get_connection_path raises ValueError. ensure_database_exists now uses this function.
  • get_sqlite_ground_truth returns an empty list, as for a database file that does not exist.

Behavior change

A SQLite database name and an env_files name must be relative to their base directory. An absolute name, or a name that leaves the directory with ../, is now rejected.

Testing

  • New tests in safepath_test.py.
  • A before/after comparison is in a comment on this PR.

@omkargaikwad23

omkargaikwad23 commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Verification

I ran the same script on main (6ca77dc) and on this branch. All files are in a temporary directory. outside is a directory next to the base directory.

# Input main This PR
1 env_files: ["../../outside/host.txt"] in _collect_env_files Returns the content of the host file Returns {}
2 env_files: ["env/ok.py"] Collected Collected
3 _materialize_env_files with the name ../../outside/written.txt Writes outside/written.txt Writes no file outside
4 _materialize_env_files with the name env/new.py Written Written
5 SQLiteDB with the database name ../outside/evil Connects and makes outside/evil.sqlite ValueError, no file
6 SQLiteDB with the database name good Connects Connects
7 get_sqlite_ground_truth with the database ../outside/other Returns the rows of outside/other.sqlite Returns []
8 get_sqlite_ground_truth with the database good Returns the rows Returns the rows

Local tests:

  • safepath_test.py and the existing container, SQLite, and agent evaluator tests: 113 passed.
  • pytest evalbench/test with SKIP_CLOUD_TESTS=true: 1303 passed, 11 skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant