Repository navigation
feat(transcription): per-user language setting for the Vexa bot - #236
Conversation
- UserSettings gains `transcription_language` (ISO 639-1, '' = auto-detect; validated by pattern). Defaults to 'en' for new and existing users. openapi.json regenerated. - Settings → Transcription shows a Language select under Enable Feature, saved through the modal's normal draft / save-on-close / Reset flow. - TranscriptionMenu reads the saved value at Start and passes it to dispatchBot, which sends `language` on POST /bots (omitted for auto-detect so Vexa falls back to per-window detection). Signed-off-by: Cameron Target <camerontarget@gmail.com>
|
|
||
| export const DEFAULT_TRANSCRIPTION_LANGUAGE = 'en'; | ||
|
|
||
| export const TRANSCRIPTION_LANGUAGES: ReadonlyArray<{ |
There was a problem hiding this comment.
I am thinking with should check with @DmitriyG228 on this. Depending on the transcription model being used, different languages may be supported. I am not sure what endpoints exists in vexa to get the avalible languages based on the loaded model.
We could have a gateway endpoint in DNA to get the avalible languages from the transcription provider.
const getAvalibleLanguages ()
FE -> DNA -> Transcription provider -> Vexa
There was a problem hiding this comment.
Sounds good -- I looked at what Vexa exposes. Per the current Meetings API and Settings API docs:
POST /bots takes language as a bare “ISO code (e.g. en)”; omitting it means per-window auto-detect, which is what I believe we send for the Auto-detect option?
Looks like there’s no endpoint that lists supported language. Claude says the old allowed_languages was dropped in 0.12, and the mid-call PUT .../config language change isn’t wired yet (returns 404).
So a FE -> DNA -> Transcription provider -> Vexa call can’t actually query Vexa at the moment -- the provider would be answering from its own knowledge of the loaded model. Makes sense to still do the gateway idea you sketched: get_supported_languages() on TranscriptionProviderBase, GET /transcription/languages, and the Settings dropdown populated from it: May make sense as a follow-up PR so this one stays small?
@DmitriyG228 -- two questions so the Vexa provider’s list is accurate: which Whisper (or other STT) model does the hosted Vexa run, and is there any plan to expose a supported-languages endpoint? If not, we could go with a static list per provider and widen it as needed? @jspada200
There was a problem hiding this comment.
Thanks @camerontarget14 , if Vexa API does not support this, I think it is still importent to support this later. On the transcription provider base, I would add a method that just returns the const that you currently have defined in the frontend. The idea here is that we setup the system in such a way that we can query this info if the provider supplies it.
FE (app) -> FE (core:getTranscriptionLanguages) -> DNA -> Transcription provider (base:get_transcription_languages) -> The returned consts.
With this setup, we can update transcription providers to be able to supply the language list.
There was a problem hiding this comment.
Ok took a pass at this!
Per review on #236: `AUTO_DETECT_LANGUAGE`, `DEFAULT_TRANSCRIPTION_LANGUAGE` and `TRANSCRIPTION_LANGUAGES` now live in `@dna/core` alongside the `UserSettings` type they describe, rather than in `app/src/components`. SettingsModal and TranscriptionMenu import them from `@dna/core`. No behaviour change; file contents are unchanged (git rename). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Cameron Target <camerontarget@gmail.com>
…-language-setting Signed-off-by: Cameron Target <camerontarget@gmail.com>
provider Per review on #236: the Settings → Transcription dropdown is now populated via GET /transcription/languages rather than a frontend constant. The endpoint delegates to TranscriptionProviderBase.get_transcription_languages(), whose default implementation returns a static Whisper subset; providers that can query their backend for the loaded model's languages override it. FE (app) -> core:getTranscriptionLanguages -> DNA -> provider -> constants The frontend keeps only AUTO_DETECT_LANGUAGE and DEFAULT_TRANSCRIPTION_LANGUAGE as pre-fetch fallbacks; user-settings defaults now reference the backend constant instead of a bare "en". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Cameron Target <camerontarget@gmail.com>
|
Thanks @camerontarget14. Two of these change in Vexa 0.13.4 (releasing soon): mid-call change works via |
- transcription_language in the user-settings request and response examples; User Settings tag description mentions it - DispatchBotRequest.language described as an ISO 639-1 code from GET /transcription/languages, with "en" in the Google Meet example - TranscriptionLanguagesResponse gets an example and the languages route documents its 400 - Concepts step 5 notes the web app fills language from the reviewer's settings, so other tools should pass it explicitly - Regenerate both openapi.json copies Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Cameron Target <camerontarget@gmail.com>
# Summary
Makes the backend API usable by someone outside the project (e.g. a
ShotGrid or AYON tooling developer) without reading the source. The
OpenAPI document now documents itself, and the docs site gets
hand-written guide pages around the generated reference.
**Backend (spec is self-documenting; `/docs`, `/redoc` and the site all
benefit)**
- New `dna/openapi_schema.py` layers onto the generated spec:
- `x-tagGroups` splitting endpoints into **Production Tracking** (proxy
the provider; DNA stores nothing) and **DNA** (DNA's own storage).
- A description on every opaque-ID parameter (`project_id`,
`version_id`, `user_email`, ...), injected centrally.
- A uniform 401 on every authenticated operation, and an
`error_responses()` helper so routes declare the 4xx/5xx they raise. All
error bodies reference a new `ErrorResponse` model (`{"detail": ...}`).
- `main.py`: API description and every tag description rewritten to say
what proxies the tracking provider vs. what DNA stores. New
**Publishing** and **Project Glossary** tags; glossary/publish routes
retagged. Longer descriptions on `publish-notes`, draft PUT, `find`,
`generate-note`, `run-qc-checks`.
- `json_schema_extra` examples on ~50 schemas, all using one fictional
production (project `124`, version `mk020_0010_comp_v003`,
`sam.reviewer@studio.example`) so examples agree across endpoints.
- `test_openapi_spec.py` now asserts: every tag is in exactly one group,
every secured op has a 401, every parameter is described, every
documented error body is `ErrorResponse`. These fail if a route is added
without documenting it, which is the point.
**Docs site**
- Spec copy was three weeks stale (integer IDs, missing endpoint). `npm
run build:full` now runs `fetch-openapi` first so it cannot drift again.
- Hand-written **Authentication**, **Concepts**, **Errors** and
**Real-time** pages (the last covers the `/ws` and extension-ingest
WebSockets, which are not in OpenAPI). Landing page rewritten as a
first-request walkthrough.
- Reference sidebar grouped by tag group.
- API explorer code samples trimmed to cURL, Python, JavaScript, C,
Rust, Go; bash Prism grammar loaded so shell blocks highlight.
**Caught up with `main`** (merged in twice; one README conflict resolved
in favour of the `fetch-openapi` build step)
- #224 Zoom: second `DispatchBotRequest` example; bot route notes Zoom
needs `meeting_url` and the passcode only works as the link's `pwd`.
- #233 transcript links: `publish-notes` and the Concepts walkthrough
say the Note is linked to the version's newest transcript row (best
effort), and the walkthrough publishes the transcript before the notes.
- #236 transcription language: `transcription_language` in the
user-settings examples; `DispatchBotRequest.language` documented as an
ISO 639-1 code from `GET /transcription/languages`; that route gets an
example and its 400.
- #240 GitHub Pages: no changes needed; the workflow already calls
`build:full`.
## Testing
- [x] I have tested these changes locally
- [x] I have run all relevant automated tests
- [x] I have verified this does not break existing workflows
- [ ] For changes that can be tested in UI, I have included screenshots
or gif animations of the changes.
## How I Tested
1. `cd backend && .venv/bin/python scripts/export_openapi.py` —
regenerated `docs/openapi.json` from the merged code (41 paths, 58
schemas).
2. `.venv/bin/python -m pytest tests/test_openapi_spec.py
tests/test_user_settings.py
tests/test_transcription_languages_endpoint.py` — 39 passed, including
the new spec guards and the drift test.
3. `cd dna-docs && npm run build:full` — fetch, regenerate and build
succeed with `onBrokenLinks: 'throw'`.
4. The API explorer renders examples client-side, so I decoded the
zlib+base64 `api:` front matter of the generated
`api/reference/*.api.mdx` for `publish-notes`, `transcription/bot`,
`transcription/languages` and user-settings PUT, and confirmed the new
descriptions, examples (`language: "en"`, `transcription_language:
"en"`, Zoom second example) and error statuses are present.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Cameron Target <camerontarget@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Summary
transcription_language(ISO 639-1, '' = auto-detect; validated by pattern). Defaults to 'en' for new and existing users. openapi.json regenerated.languageon POST /bots (omitted for auto-detect so Vexa falls back to per-window detection).Testing
How I Tested
Stood up DNA with vexa hosted and tried speaking chinese during a meeting. Transcript came back english only. IMPORTANT: if you directly speak another language, the model will give you that. It's when you include chinese in an english sentence or something, then it stays english. Sort of a when in doubt system that I think works well to address the issue.
Translation is a feature we could utilize if we wanted to make 100% sure everything was confirmed to a language.