Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions docs/integrity-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,41 @@ the camera required, because the recording notice describes a video recording.
- `declined`: the candidate skipped a camera with no error, or with an error
that names none of the causes above.

## Microphone switches

A candidate can move the interview to another microphone from the interview
page, and the page does so by itself when the input in use ends. Either way,
once the new input is in use, the browser records a signed `MICROPHONE_SWITCHED`
event, because the heartbeat that follows names a live microphone and would
otherwise read as the one the interview started with. Its detail is
`reason=chosen` or `reason=lost`, then `microphone=` and the new device's label,
last because a label is the browser's text and may contain the separators. A
switch is not a microphone stopping, so it records no `MICROPHONE_STOPPED`; an
input that ended before the page switched away from it still records one, since
it did stop.

A chosen switch that fails leaves the previous input in use and records nothing,
unless the previous input cannot be handed back to the sender either; the page
then drops it, says the interviewer cannot hear the candidate, and asks for a
microphone again. That drop records `MICROPHONE_STOPPED` with
`reason=switch_failed`, because stopping a working input fires no `ended` and
the state poll skips a missing track, so nothing else would. A move after a lost
input that fails leaves the interview with no working microphone; the page says
so, records no switch, and tries the move again whenever a device is plugged in,
while the `MICROPHONE_STOPPED` for the loss stays the last word on the input
that ended.

While a switch is under way, LiveKit's own mute disables the input still in
use, so that the replacement is not sent before the Mic button says. For that
stretch the heartbeat and the state poll report the Mic button's state instead
of the input's, since a `MICROPHONE_STATE_CHANGED` to `off` there would record
the switch muting a candidate who never pressed Mic.

The server drops an integrity event whose type it does not list before it
checks the chain, so that event's sequence number is never accepted and every
later event is refused as out of sequence. A new event type is therefore added
to `sanitize_integrity_event` in the same change that starts sending it.

## Response windows

The replay page lists a *response window* for each turn the interviewer took:
Expand Down
15 changes: 15 additions & 0 deletions scripts/gen-wire-fixtures.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ const { reportRecoveryLimits } = await import(
join(ROOT, "web", "report-recovery.js")
);
const lib = await import(join(ROOT, "web", "lib.js"));
const { switchDetail } = await import(join(ROOT, "web", "mic-switch.js"));
const { ALL_LANGUAGES } = await import(
join(ROOT, "web", "compiler-explorer.js")
);
Expand Down Expand Up @@ -361,6 +362,20 @@ const INTEGRITY_INPUTS = [
"5555555555555",
],
},
// Last, so the rows above keep their hashes. A type the agent's allowlist
// lacks is refused and stops the chain, which is how a switch the browser
// records would otherwise vanish from every report. The detail comes from the
// producer the page uses, with a label carrying the separators it is placed
// last to survive.
{
type: "MICROPHONE_SWITCHED",
source: "microphone",
severity: "info",
at: "2026-08-18T06:40:30.512Z",
detail: switchDetail("chosen", {
label: "USB Audio; Rear=Mic (0d8c:0014)",
}),
},
];

async function integrityChain() {
Expand Down
6 changes: 6 additions & 0 deletions src/agent/integrity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,12 @@ pub fn sanitize_integrity_event(payload: &serde_json::Value) -> Option<serde_jso
"CAMERA_RELEASED_TO_PRESENTER" => "CAMERA_RELEASED_TO_PRESENTER",
"CAMERA_NOT_USED" => "CAMERA_NOT_USED",
"MICROPHONE_STATE_CHANGED" => "MICROPHONE_STATE_CHANGED",

// The candidate moved the interview to another input, or the page did
// after the one in use ended. Without it the heartbeat goes on naming a
// live microphone and nothing says it is no longer the one the
// interview started with.
"MICROPHONE_SWITCHED" => "MICROPHONE_SWITCHED",
_ => return None,
};
let at = text("at")?;
Expand Down
61 changes: 60 additions & 1 deletion tests/browser/devices.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { test } from "node:test";
import assert from "node:assert/strict";
import { FakeStream, FakeTrack, settle } from "./fake-media.js";

import { createDevicePool } from "../../web/devices.js";
import { createDevicePool, openMicrophone } from "../../web/devices.js";

function poolWith(getUserMedia, options = {}) {
const changes = { count: 0 };
Expand Down Expand Up @@ -571,3 +571,62 @@ test("the pool's own drop of an ended track also removes before notifying", asyn
finished = true;
pool.cancelRetry();
});

// The switch the candidate asked for names its device exactly: opening some
// other input and calling it theirs would be worse than saying it failed.
test("a mid-interview microphone is asked for by exact device", async () => {
const asked = [];
const track = new FakeTrack("audio");
const opened = await openMicrophone(
{
getUserMedia: async (constraints) => {
asked.push(constraints);
return new FakeStream([track]);
},
},
"headset",
);

assert.equal(opened, track);
assert.deepEqual(asked, [
{
audio: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
deviceId: { exact: "headset" },
},
},
]);
});

test("the default microphone is asked for with no device id", async () => {
const asked = [];
await openMicrophone(
{
getUserMedia: async (constraints) => {
asked.push(constraints);
return new FakeStream([new FakeTrack("audio")]);
},
},
"",
);

assert.equal("deviceId" in asked[0].audio, false);
});

test("a grant with no live input is refused and leaves nothing open", async () => {
const spare = new FakeTrack("video");
const dead = new FakeTrack("audio");
dead.readyState = "ended";

await assert.rejects(
openMicrophone(
{ getUserMedia: async () => new FakeStream([dead, spare]) },
"headset",
),
/no active audio track/,
);
assert.ok(spare.stopped, "a track nothing will read was left running");
assert.ok(dead.stopped);
});
Loading