Switch to PyPA simple api for pypi by abstractedfox · Pull Request #5 · abstractedfox/agents

abstractedfox · 2026-07-24T03:14:20Z

Local PR of agents cloudflare#1950 so it can be reviewed with sensible diffs while waiting for an earlier PR to be merged. A fair amount of this changes in followup branches, notably using a metadata fetch gets replaced with reading the dependencies out of the wheel.

Reconstitute work on this repo Add index.py as default entry point Start of basic package support This is the beginning of work on this feature, it's a little comment-heavy to that end and has some patterns that may not be permanent (these have generally been noted as such). As of this commit, there are tests that start to poke at this, namely the one that brings in fastapi, which currently fails due to an extension it doesn't like. Python package proof of concept Proof of concept for package support in dynamic Python workers. As of this commit, it can retrieve an sdist with a flat layout from pypi and install it into the virtual filesystem. It doesn't do dependency resolution, so the current tests fail when attempting to import FastAPI's deps. Clarifying comment Use wheels properly Used sdists before, now it uses wheels. Pre cleanup commit Wheel changeover cleanup Use simpler packages for minimal package support test This test only needs to confirm that retrieval of a hard list of pure python packages, without dependency resolution, works. Move python logic out of 'bundle' path This seems to be specific to JS packages Slight cleanup, remove test More cleanup Re-add parameter that was mistakenly deleted Revert "remove dependency that was causing issues" This dependendency caused issues when working in a container, but its removal was purely for my sake This reverts commit 4f8990d. Order deps alphabetically Remove unused variable Make createWorker Python return more uniform with JS Make createWorker registry pattern more consistent with JS Also makes PyprojectToml interface less stringent Small fixes These got caught by the checks done by `pnpm run check` Style fixes Remove parsing for version string Comment dep version parameter Compensate for distribution packages' names not matching their imports Remove default values from test These are already set implicitly by createWorker Move Python package metadata collection into its own function Move Py deps installation into its own function Account for type checker Nested dependency installation without version resolution Undo unintended compat date change Bump up compatibility date (test) Prevent adding python and JS deps to the same worker Implicitly add workers-runtime-sdk package Remove check for when there are no dependencies, since there will always be at least one Add python_workers flag if user-supplied flags exclude it Typo Clarifying comment Include installWarnings with returned object Type checker fix 1st pass moving to 'simple API' on PyPI A fair amount of this is AI code, a beautification pass would be beneficial Merge from python-package-resolution Change tests to assert existence of compat flags Remove redundant compat flag default This is implicit on Python dynamic workers now Mark comment as TODO Finish propagating `result` object and explicitly disable pyproject.toml and package.json in one worker Clear warning There were warnings in the console about a binary/octet-stream response being parsed as text. Since the response actually is text, this avoids that warning. Change return type

Add index.py as default entry point Start of basic package support This is the beginning of work on this feature, it's a little comment-heavy to that end and has some patterns that may not be permanent (these have generally been noted as such). As of this commit, there are tests that start to poke at this, namely the one that brings in fastapi, which currently fails due to an extension it doesn't like. Python package proof of concept Proof of concept for package support in dynamic Python workers. As of this commit, it can retrieve an sdist with a flat layout from pypi and install it into the virtual filesystem. It doesn't do dependency resolution, so the current tests fail when attempting to import FastAPI's deps. Clarifying comment Use wheels properly Used sdists before, now it uses wheels. Pre cleanup commit Wheel changeover cleanup Use simpler packages for minimal package support test This test only needs to confirm that retrieval of a hard list of pure python packages, without dependency resolution, works. Move python logic out of 'bundle' path This seems to be specific to JS packages Slight cleanup, remove test More cleanup Re-add parameter that was mistakenly deleted Order deps alphabetically Remove unused variable Make createWorker Python return more uniform with JS Make createWorker registry pattern more consistent with JS Also makes PyprojectToml interface less stringent Small fixes These got caught by the checks done by `pnpm run check` Style fixes Remove parsing for version string Comment dep version parameter Compensate for distribution packages' names not matching their imports Remove default values from test These are already set implicitly by createWorker Move Python package metadata collection into its own function Move Py deps installation into its own function Account for type checker Undo unintended compat date change Bump up compatibility date (test) Prevent adding python and JS deps to the same worker Implicitly add workers-runtime-sdk package Remove check for when there are no dependencies, since there will always be at least one Add python_workers flag if user-supplied flags exclude it Typo Clarifying comment Include installWarnings with returned object Change tests to assert existence of compat flags Remove redundant compat flag default This is implicit on Python dynamic workers now Mark comment as TODO Change installDependenciesPython to replicate the JS impl's usage of the result object Finish propagating `result` object and explicitly disable pyproject.toml and package.json in one worker Add more TODOs

ryanking13 · 2026-07-24T07:35:05Z

+    if (
+      !latest ||
+      !latestVersion ||
+      comparePythonVersions(version, latestVersion) > 0


(no action needed for this PR)

regex parsing is often an expensive operation. So Ideally we should find a way to optimize the number of regex parsing.

In this case, the version string should be very short, and there would not be too many versions for a single package, so probably it is okay for now. We can optimize it later when there is an issue.

(Funny story, Cloudflare had a global incident in 2019 because of regex 😉)

So long as you don't use certain regex operations like lookaheads it should be fine and not much slower than manual parsing. Regex itself isn't fundamentally expensive.

That said, you likely do want to switch over to parsing these manually the more complex the parsing gets. Just to make it easier for you.

For now though, regex is probably fine.

Ahh I see, thanks for the info @ryanking13 @dom96 . Will add it to my list to rethink this

dom96

Seems like there is some rebasing issues with some of the code. Also the version parser needs unit tests.

Otherwise LGTM.

dom96 · 2026-07-24T15:16:46Z

+ */
+function selectWheel(files: PypiSimpleFile[]): PypiSimpleFile | undefined {
+  const wheels = files.filter((f) => f.filename.endsWith(".whl"));
+  if (wheels.length === 0) return undefined;


nit: return null instead of undefined

dom96 · 2026-07-24T15:19:52Z

+    if (
+      !latest ||
+      !latestVersion ||
+      comparePythonVersions(version, latestVersion) > 0


So long as you don't use certain regex operations like lookaheads it should be fine and not much slower than manual parsing. Regex itself isn't fundamentally expensive.

That said, you likely do want to switch over to parsing these manually the more complex the parsing gets. Just to make it easier for you.

For now though, regex is probably fine.

dom96 · 2026-07-24T15:33:10Z

 * Returns a tuple of `[package_name, null, null]`. The second and third slots
 * are placeholders reserved for future use (e.g. extras, version specifier).
 */
 function parsePythonVersionString(spec: string): { name: string } {


This is already in 0b2bd02, so why am I seeing it here?

Actually not sure, looks like something I missed while rebasing. Will follow up on this

abstractedfox added 3 commits July 23, 2026 23:01

Remove leftover merge conflict line

6a8c4e3

abstractedfox force-pushed the python-package/pypi-simple-api branch from 17e3c03 to 6a8c4e3 Compare July 24, 2026 04:20

Merge branch 'main' into python-package/pypi-simple-api

fec9c92

ryanking13 reviewed Jul 24, 2026

View reviewed changes

dom96 reviewed Jul 24, 2026

View reviewed changes

abstractedfox added 3 commits July 24, 2026 17:37

Add tests for comparePythonVersions

36ba890

Feedback implementation

9f84b35

Merge branch 'main-wtf' into python-package/pypi-simple-api

9e21086

ryanking13 approved these changes Jul 27, 2026

View reviewed changes

Alter selectWheel to return null instead of undefined

3d597f9

abstractedfox merged commit 45e7a03 into main Jul 27, 2026
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to PyPA simple api for pypi - #5

Switch to PyPA simple api for pypi#5
abstractedfox merged 8 commits into
mainfrom
python-package/pypi-simple-api

abstractedfox commented Jul 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

ryanking13 Jul 24, 2026

Uh oh!

dom96 Jul 24, 2026

Uh oh!

abstractedfox Jul 24, 2026 •

edited

Loading

Uh oh!

dom96 left a comment

Uh oh!

dom96 Jul 24, 2026

Uh oh!

hoodmane Jul 24, 2026

Uh oh!

dom96 Jul 24, 2026

Uh oh!

Uh oh!

Uh oh!

dom96 Jul 24, 2026

Uh oh!

abstractedfox Jul 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

abstractedfox commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanking13 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

dom96 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

abstractedfox Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dom96 left a comment

Choose a reason for hiding this comment

Uh oh!

dom96 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

hoodmane Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

dom96 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dom96 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

abstractedfox Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

abstractedfox commented Jul 24, 2026 •

edited

Loading

abstractedfox Jul 24, 2026 •

edited

Loading