feat(core): add FILE instrument kind and storage schemas#1379
Merged
Conversation
Introduce the FILE instrument kind in runtime-core (file-type constants, FileInstrument types with per-group basename/type/count) and the matching zod schemas in @opendatacapture/schemas, including file location/metadata, presigned URL info, and the upload-complete payload. Add the `pending` flag and file metadata to instrument records, and extend instrument-utils guards/translation helpers for the new kind. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a StorageService backed by an S3-compatible client (presigned upload and download URLs, bucket bootstrap on init) and a FilesService/controller exposing list, presigned upload-url, and upload-complete routes scoped to an instrument record. Records for FILE instruments are created in a `pending` state and excluded from results until uploads complete. Wire up the new InstrumentRecordFile auth subject and abilities, add the storage env vars, register the storage module, and seed file instruments in demo data. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the FileInstrumentContent components (dropzone, upload progress, error box, upload store) and a NavigationBlockerDialog to guard against leaving mid-upload, and render them through the instrument renderer. On the web app, wire presigned upload/download via useInstrumentRecordFilesQuery and the upload mutation, add a per-record file view under the datahub table route, and allow opting out of the default request timeout for long uploads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add ARBITRARY_SINGLE_FILE and MRI_SCAN_SESSION example instruments to the library (and register them in the availability script), a file instrument stub, and update serve-instrument to render and serve the FILE kind. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a RustFS (S3-compatible) service to docker-compose with a persistent data volume, proxy /storage/* to it via Caddy, and generate storage access/secret keys in generate-env.sh. Document the new STORAGE_* and RUSTFS_VERSION variables in .env.template, ignore the rustfs data dir, and update the app Dockerfiles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the new storage/upload dependencies, refresh the pnpm lockfile and workspace catalog, update turbo pipeline config, and bump the pnpm packageManager version. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the new S3-compatible storage requirement, STORAGE_* env vars, Caddy/reverse-proxy and docker-compose changes, and backup implications for upgrading deployments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (105)
WalkthroughAdds file-instrument support end to end: shared types and schemas, auth and persistence, storage-backed upload/download endpoints, web upload and file views, and serve/runtime/package wiring. ChangesFile instruments and storage flow
Sequence Diagram(s)✨ Finishing Touches🧪 Generate unit tests (beta)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A summary of the file-instrument feature: a new instrument kind that lets
subjects/users upload arbitrary files, with the files stored in an S3-compatible
object store (RustFS) rather than in the database. Covers the schema/type
foundation, the API storage service and upload/download endpoints, the web +
react-coreupload/download UI, example instruments and tooling, the deploymentstack, and docs.
NOT TECHNICALLY BREAKING CHANGE BUT RELEASED UNDER V2 BECAUSE PEOPLE SHOULD UDPATE DOCKER COMPOSE STACK.
Core types & schemas
feat(core): add FILE instrument kind and storage schemasFILEinstrument kind alongside the existing kinds, with its ownruntime type (
runtime-core/src/types/instrument.file.ts) and validationschema (
schemas/src/instrument/instrument.file.ts).(
instrument.base,instrument.core,instrument.interactive,instrument.ts) and the runtimedefine/i18n/constantsentry points.schemas/src/storage/storage.ts) describing stored-filemetadata, and extended instrument-records schemas to carry file references.
instrument-utils(guards.ts,translate.ts).API — object storage & file endpoints
feat(api): add object storage service and file upload endpointsStorageModule/StorageServiceproviding an S3-compatible object-storeclient (
apps/api/src/storage/).(
instrument-records/files/): controller, service, and types for uploadingand retrieving record files.
permission model (
ability.factory.ts,ability.utils.ts,auth.types.ts,plus added ability tests).
(
core/schemas/env.schema.ts); demo service seeds accordingly.Web + react-core — upload/download UI
feat(web): add file instrument upload and download UIFileInstrumentContentcomponent family inreact-core(
Dropzone,UploadProgressBar,ErrorBox, a statestore, and types) forselecting, validating, and uploading files with progress.
InstrumentRenderer/ScalarInstrumentRenderer/ series renderers extendedto render the file-instrument content and report results.
NavigationBlocker+NavigationBlockerDialogto warn againstnavigating away mid-upload.
datahub/$subjectId/table/$recordIdwith file download support; newuseInstrumentRecordFilesQueryand an updateduseUploadInstrumentRecordsMutation(multipart/progress).Instruments — examples, stubs & serving
feat(instruments): add file instrument examples and stub/serve supportinstrument-library:ARBITRARY_SINGLE_FILEandMRI_SCAN_SESSION.instrument-stubs/src/file.js) and interactive stubupdates.
serve-instrument(CLI, root, server) extended to bundle and serve fileinstruments.
Deployment & configuration
chore: add RustFS object storage to the deployment stackchore(deps): update lockfile and workspace config for file instrumentsmake storage optionaldocker-compose.yaml, with Caddyrouting, Dockerfile updates,
.env.templatekeys, andgenerate-env.shsupport.
pnpm-workspace.yaml,turbo.json, root/gatewaypackage.json, andthe lockfile for the new dependencies.
configured (env schema,
StorageModule/StorageService, andinstrument-records service degrade gracefully when storage is absent).
Docs
docs: add v2.0.0 migration guide for object storagedocs/en/6-updating/v2.0.0.mddescribing how to provision/configureobject storage when upgrading to v2.0.0.
Summary by CodeRabbit
Release Notes v2.0.0
New Features
Infrastructure