Feature(open-in-overleaf): add the Overleaf API (/docs) and /devs page - #82
Feature(open-in-overleaf): add the Overleaf API (/docs) and /devs page#82Musicminion wants to merge 3 commits into
Conversation
- backend: POST|GET /docs creates a project from snip / encoded_snip / snip_uri[] (+snip_name[]) / zip_uri / data: URLs, with engine and main_document; external URLs are fetched through linked-url-proxy (SSRF-guarded), capped at MAX_UPLOAD_SIZE with a 60s timeout - frontend: /devs documentation page mirroring the official one, with live examples that open real projects on this server - enable the module in settings.defaults.js
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 26c168b04b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| webRouter.post( | ||
| '/docs', | ||
| AuthenticationController.requireLogin(), | ||
| OpenInOverleafController.openInOverleaf |
There was a problem hiding this comment.
When a signed-out user submits any documented POST form, requireLogin() redirects to login while AuthenticationController.setRedirectInSession() preserves only the path and query string, so the POST body is discarded; after login the redirected GET /docs has no input and returns the “no snippet” error. This also affects typical cross-site forms for otherwise logged-in users because the default SameSite=Lax session cookie is not sent with a cross-site POST. Preserve the submitted parameters through the login round trip rather than redirecting with only /docs.
Useful? React with 👍 / 👎.
| const files = [] | ||
| for (const [index, { url, name }] of snipUris.entries()) { | ||
| const buffer = await fetchUrlToBuffer(url) | ||
| files.push({ | ||
| buffer, | ||
| name: safeFileName(name, url, index), | ||
| isZip: looksLikeZip(buffer), |
There was a problem hiding this comment.
For snip_uri[] submissions, every URL is independently allowed up to settings.maxUploadSize, and every downloaded buffer remains retained in files until the archive is built. A single request containing many entries can therefore consume many multiples of the configured 50 MB default—potentially exhausting the web process memory—despite the advertised upload cap. Track cumulative bytes and reject the request once the aggregate exceeds the project upload limit.
Useful? React with 👍 / 👎.
| webRouter.post( | ||
| '/docs', | ||
| AuthenticationController.requireLogin(), | ||
| OpenInOverleafController.openInOverleaf |
There was a problem hiding this comment.
Both /docs methods invoke project creation directly without a rate limiter, bypassing the 20-per-minute limits used by /project/new and /project/new/upload. An authenticated client can consequently issue unlimited project imports, each potentially performing a remote download and archive extraction, which can consume storage and worker resources. Add an equivalent per-user project/import rate limiter to these routes.
Useful? React with 👍 / 👎.
| // A single .tex file → straight snippet project (keeps the nice main.tex). | ||
| return await ProjectCreationHandler.promises.createProjectFromSnippet( | ||
| ownerId, | ||
| projectName, | ||
| prepareSnippet(file.buffer.toString('utf8')).split('\n') |
There was a problem hiding this comment.
When exactly one non-zip snip_uri is supplied, the computed file.name is discarded and createProjectFromSnippet() always creates main.tex. Thus snip_name=chapter.tex does not perform the documented filename override, and a matching main_document=chapter.tex silently fails to select the requested document. Import the file under file.name, or use the zip-import path when a custom name is present.
Useful? React with 👍 / 👎.
|
@codex, code review |
Musicminion
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a4ed6dd050
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| async function stashForLogin(req, res, next) { | ||
| if (SessionManager.isUserLoggedIn(req.session)) return next() | ||
| const token = crypto.randomUUID() | ||
| await rclient.setex( |
There was a problem hiding this comment.
For every signed-out POST /docs, this setex stores the parsed submission for ten minutes and stashForLogin then redirects without reaching the newly added rate limiter later in OpenInOverleafRouter.mjs. Consequently, an unauthenticated client can repeatedly park request bodies—up to the global 2 MB form-body limit each—in the shared Redis instance and exhaust its memory. The fresh issue in this revision is specifically that the new limiter is ordered after this early-returning middleware; apply an IP-based limit before writing the resume key.
Useful? React with 👍 / 👎.
| async function importFilesAsZip(files, ownerId, projectName) { | ||
| const zipPath = await buildZipFromFiles(files) | ||
| try { | ||
| return await ProjectUploadManager.promises.createProjectFromZipArchive( |
There was a problem hiding this comment.
When a zip_uri or multi-file submission includes a name/project_name, this call treats that value only as a fallback: createProjectFromZipArchive derives the actual project name from the archive's TeX \title whenever one exists. Thus an archive containing \title{Paper} ignores a submitted name=Requested Name, unlike snippet imports. Use the name-preserving archive-import path (and adapt its return value) when the API supplies the project name.
Useful? React with 👍 / 👎.
- preserve POST parameters across the login redirect using a short-lived single-use redis token, so signed-out and cross-site (SameSite=Lax) submissions are not lost - rate-limit /docs at 20/min, matching project creation - share one upload budget across multi-file snip_uri[] imports - honor snip_name for single-file imports
- rate-limit /docs before stashForLogin - keep the requested name on zip imports (not the TeX \title)
a4ed6dd to
1d6bbf4
Compare
|
@codex, code review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1d6bbf4c59
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // snip_name given: import under that name so main_document can select it. | ||
| return await importFilesAsZip(files, ownerId, projectName, preserveName) |
There was a problem hiding this comment.
When a single non-zip URI includes snip_name, this archive-import branch bypasses prepareSnippet(), unlike the otherwise identical unnamed-file path above. Consequently, a fragment such as snip_uri=data:text/plain,Hello&snip_name=chapter.tex is imported without the documented \documentclass wrapper and will not compile. Apply the snippet preparation while preserving the requested filename.
Useful? React with 👍 / 👎.
| files.push({ | ||
| buffer, | ||
| ...safeFileName(name, url, index), | ||
| isZip: looksLikeZip(buffer), |
There was a problem hiding this comment.
When multiple URIs have the same path basename, such as /a/main.tex and /b/main.tex, safeFileName() assigns both entries the same name. buildZipFromFiles() then emits duplicate archive paths, and the sequential extraction writes both to the same destination, causing the later file to overwrite the earlier one before project import. Reject duplicate names or generate unique defaults so multi-file submissions do not silently lose content.
Useful? React with 👍 / 👎.
1 participant
Description
Add devs page for ayakaleaf pro. Reference: http://www.overleaf.com/devs
Full pages:
Related issues / Pull Requests
Contributor Agreement