Security: default lf.query Python protocol executes model output via exec() (CWE-94/95)

[thegr1ffyn]

Summary

lf.query(prompt, schema, lm=...) parses the model's response into a Python object. On the default protocol (protocol='python'), the parse step executes the model's response as Python via pyglove.coding.evaluate, with sandbox=False. The only guard is an AST node-type filter (CodePermission.ASSIGN|CALL) that restricts node types but not which names or builtins may be called. As a result, attacker-influenced model output can reach exec() on default settings — i.e. anything the model can be made to say, langfun will run as Python in the host process.

This was reported to Google's OSS VRP (issue tracker 518137699) and classified Won't Fix (Intended Behavior); the triager acknowledged it as a product vulnerability but noted langfun's tier makes it ineligible for a reward and suggested raising it here directly. Filing this as a public, constructive record with a proposed fix. Quoting triager's comments below for reference:

Hey there! Thanks for your report on the langfun project regarding the eval injection leading to RCE via lf.query's default 'python' protocol. You've clearly identified a product vulnerability here.
While langfun is indeed a Google open-source project and falls under the OSS VRP, our program rules state that product vulnerabilities in OT2 (Standard) and OT3 (Low-Priority) tiered repositories are not eligible for monetary reward. The langfun repository currently falls into one of these tiers (as 'TIER_UNSPECIFIED' implies).
This means we won't be able to offer a reward for this specific finding, but we definitely appreciate you bringing it to our attention. You're more than welcome to open an issue or submit a pull request directly to the langfun GitHub repository to help get this fixed.

Impact: why this is a vulnerability, not just code execution by design

This is remote code execution. In any deployment where the model's input can be influenced by an attacker — a RAG app over user-supplied documents, an agent processing tool output or web pages, anything summarizing untrusted text — a successful prompt injection turns into arbitrary Python execution inside the application's process. Concretely, an attacker who lands the payload can:

• read process secrets (API keys, DB credentials, tokens in env/memory),
• read and write the filesystem with the app's privileges,
• make outbound network requests (exfiltration, SSRF, pivoting to internal services),
• and otherwise run anything the host process can run.

The execution is silent: the malicious response also returns a valid schema object, so the calling application receives exactly what it expected and the side effect is invisible. There is no error, no log, nothing for the developer to notice.

The reason this is a vulnerability and not merely "the Python protocol runs code" is that it is the default, it is undocumented at the call site, and the one control that looks like a sandbox does not work. A developer who follows the README — lf.query("...", MySchema, lm=...) — has not opted into executing model output; they asked the library to parse a structured object. Nothing in the call signature or docstring indicates exec() is involved, and the AST permission filter that appears to be a security boundary fails open (see below). "The model is trusted" is not a safe assumption — prompt injection from untrusted content in the prompt is the baseline threat for LLM applications, and treating model output as trusted Python source is the vulnerability.

Affected

• PyPI langfun 0.1.1 (latest release; full history is 0.0.1 / 0.1.0 / 0.1.1) — confirmed
• main @ e80093b — confirmed
• No hardening of this path is present on main (the only structured-parser commit since 0.1.1 is refactor b1d6d95, which moved the file but preserved sandbox=False and the ASSIGN|CALL default).

Resolved defaults on a no-args lf.query call

protocol='python', sandbox=False, permission=ASSIGN|CALL (value 9)

• querying.py:466-467 — protocol default resolves to 'python'
• schema/python.py:167 — sandbox=False
• schema/python.py:142-144 — permission default ASSIGN|CALL
• schema/python.py:163 comment, verbatim: # We are creating objects here, so we execute the code without a sandbox.

Call chain (public API → exec())

lf.query -> Mapping.parse_result (mapping.py:490)
  -> PythonPromptingProtocol.parse_value (schema/python.py:110-135)
  -> structure_from_python(sandbox=False) (schema/python.py:138-171)
  -> correction.run_with_correction (correction.py:42)
  -> execution.run (execution.py:69)
  -> pg.coding.evaluate -> exec(compile(...)) (pyglove execution.py:128)

lf.parse and lf.call reach the same sink on the same default.

Minimal reproduction

Using a mock LM so the "model output" is supplied directly — no network, no API keys. The entire model response is a single expression that looks like an ordinary schema literal:

import os, pathlib
import pyglove as pg
import langfun as lf

CANARY = pathlib.Path(f"/tmp/lf_canary_{os.getpid()}")
if CANARY.exists(): CANARY.unlink()

class Person(pg.Object):
    name: str
    age: int

# Single expression; no __builtins__ reference, no __import__, no lambda.
body = f"Person(name=open('{CANARY}','w').write('hit') and 'A', age=1)"
r = lf.query("prompt", Person, lm=lf.llms.StaticResponse(f"```python\n{body}\n```"))

print("result:", r)                       # valid Person(name='A', age=1)
print("canary written:", CANARY.exists())  # True -> code executed as a side effect
if CANARY.exists(): CANARY.unlink()

lf.query returns a valid Person and writes the canary file as a side effect. write() returns an int (truthy), so <int> and 'A' short-circuits to 'A' and the schema validates normally.

Reproduction note: verify via the filesystem canary, not via print. pg.coding.evaluate wraps the exec in contextlib.redirect_stdout (pyglove execution.py:104), so a print-based check appears to do nothing even when the code executed.

Why the AST filter does not prevent this

The permission=ASSIGN|CALL filter allows ast.Call and all expression nodes, but it does not constrain which callables are invoked. CPython auto-populates __builtins__ into the exec globals, so open and __import__ are reachable by bare name (both confirmed). Removing __builtins__ is not a complete fix — it auto-repopulates, and the schema's own classes plus pg itself remain reachable with arbitrary Call nodes permitted. The filter restricts node types, not node contents; as a code-execution boundary it does not hold.

Threat model

An application built on langfun (RAG, agent, document processing) feeds content into the prompt that an attacker can influence — an ingested document, a user message, a scraped page, a tool output, a retrieved email. Prompt injection steers the model to emit the expression above, and on default settings langfun executes it in the application's process — yielding arbitrary code execution with the application's privileges (secret theft, filesystem access, outbound network). No host access, no knowledge of the schema, and no non-default configuration is required.

Proposed remediation (in priority order)

1. Replace exec-based parsing with a literal-only / schema-aware evaluator — an ast.literal_eval-style walker that permits construction only of the schema's declared classes (plus stdlib containers like list, dict, tuple, set). Legitimate structured-output responses are always of the form Schema(field=...) with literal or nested-schema values; they never require Call on free names like open or __import__. This removes the sink entirely.
2. Interim hardening: default sandbox=True in structure_from_python and fail loudly if a sandbox cannot be created.
3. Document the behavior: note in the lf.query / lf.parse / lf.call docstrings and a README "Security model" section that the Python protocol executes model output, and consider a runtime warning until (1) lands.

I'm happy to open a PR, if that's a welcome direction.

Note on the JSON protocol: the non-default protocol='json' path is not a safe drop-in alternative — it routes to pg.from_json_str(auto_import=True) in pyglove, which imports attacker-named modules and, for _type: "function", runs marshal.loads(...). That is a separate concern that belongs upstream in google/pyglove. Flagging only so "switch the default to JSON" isn't read as a fix on its own.

[thegr1ffyn]

@daiyip @yifenglou