Skip to content

Add run config, Prolific identifiers, and completion-code exits (C&WG) #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
htsukamoto5 wants to merge 2 commits into
base: chore/vendor-multiplayer-bundles
Choose a base branch
from
Open

Add run config, Prolific identifiers, and completion-code exits (C&WG) #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e02a99f
feat(cwg): add run config, Prolific identifiers, and completion-code …
htsukamoto5 Jul 30, 2026
c2c69c2
fix(cwg): give dyad_id a fallback, and correct the round_timeout claim
htsukamoto5 Jul 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,12 @@ unrestricted two-way chat — faithful to both papers and load-bearing for the r
effect both measure. Both were verified parameter-by-parameter against the papers **and** the
original hawkrobe/tangrams experiment code, and play-tested two-tab at full schedule length.

> **`round_timeout` is not from either paper.** Both builds set one (60s Hawkins, 180s C&WG) purely so
> a disconnected or absent partner cannot hang the trial forever — neither original was timed. Rounds
> it ends are logged as `ended_by: "timeout"` with a null assignment, so they're easy to exclude.
> **`round_timeout` is not from either paper.** Both builds set one (60s Hawkins, 180s C&WG) so a
> disconnected or absent partner cannot hang the trial forever — neither original was timed. Note what
> it is, though: an **unconditional wall-clock bound armed when the round starts**, which ends every
> round at that value whether or not anyone has dropped. It is not a dropout detector and cannot
> distinguish a dead partner from a slow one. Rounds it ends are logged as `ended_by: "timeout"` with a
> null assignment, so they're easy to exclude.

## Running it

Expand Down
203 changes: 194 additions & 9 deletions reference-game-cwg.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@
<script src="https://unpkg.com/@jspsych/plugin-survey-text"></script>
<script src="https://unpkg.com/@jspsych/plugin-html-keyboard-response"></script>
<script src="https://unpkg.com/@jspsych/plugin-preload"></script>
<!-- STUDY PROTOTYPE: on the LOCAL adapter for two-tab piloting on one machine. For the paid Prolific
run, swap adapter-multiplayer-local → adapter-multiplayer-firebase (and pair via a real waiting
room); nothing else in this file changes. -->
<!-- STUDY PROTOTYPE: on the LOCAL adapter for two-tab piloting on one machine. NOT ready for a paid
run. Swapping adapter-multiplayer-local → adapter-multiplayer-firebase is NOT sufficient on its
own, and is not a drop-in: `SEED` and `DYAD_ID` below both read ?mp_session=, which the
LocalAdapter constructor generates and Firebase will not, so every dyad would silently share one
trial order; `persistParticipant` is LocalAdapter-only; and pairing needs a rolling waiting room
that sync/role do not provide. -->
<!-- These four load from vendor/ in this repo, NOT from npm: reference-game is unpublished and the
published 0.1.0 builds predate the jsPsych.multiplayer namespace migration. Repoint to
cdn.jsdelivr.net/npm/... and delete vendor/ once jspsych-multiplayer PR #35 publishes.
Expand Down Expand Up @@ -46,10 +49,13 @@
// • After each trial, feedback reveals the correct order to both (original: mismatches reported).
//
// The other departure is not a design choice: `round_timeout` (180s) has no counterpart in the
// original, which ran untimed. It exists so a disconnected or absent partner cannot hang the trial
// forever. Rounds it ends are recorded as `ended_by: "timeout"` with a null assignment — filter on
// that when analysing. Watch it in piloting: trial 1 of a 12-figure board is the longest in the
// study, and a cap that truncates it would bite exactly where the effect is largest.
// original, which ran untimed. It was added so a disconnected or absent partner cannot hang the
// trial forever — but note what it actually is: an UNCONDITIONAL wall-clock bound armed when the
// round starts, which ends EVERY round at 180s whether or not anyone has dropped. It cannot tell
// a dead partner from a slow one. Rounds it ends are recorded as `ended_by: "timeout"` with a null
// assignment — filter on that when analysing. Watch it in piloting: trial 1 of a 12-figure board is
// the longest in the study, and a cap that truncates it would bite exactly where the effect is
// largest.
//
// Key replication DVs (logged per round): message_count / messages_sent (the words-per-figure
// effect: 41→8 across trials 1→6 in the original), accuracy / n_correct (original error rate 2%).
Expand All @@ -60,6 +66,47 @@
// Faithful is 6 trials. Lower for a quick two-tab smoke test, then restore to 6.
const TRIALS = 6;

// ===============================================================================================
// RUN CONFIG — every value that has to agree with the Prolific study configuration.
//
// Deliberately one block, because these are decisions rather than code, and several of them
// CANNOT BE CHANGED once a Prolific study is published (the reward, the screen-out reward, the
// advertised duration). Keeping them in one place is what makes them reviewable and what keeps
// participant-facing copy from drifting away from what is actually configured.
//
// See DECISIONS.md (group B, "locked at publish") and the launch checklist there.
// ===============================================================================================
const CONFIG = {
// --- Prolific completion codes. Created with the study; paste them here. -------------------
// FOUR codes, not one: every exit has to be separately reconcilable, and three of the four are
// paths where the participant did NOT finish but must still be paid.
COMPLETION_CODES: {
complete: "", // TODO(#7): finished the game
partner_dropped: "", // TODO(#7): survivor of a dropout (#5)
no_match: "", // TODO(#7): lobby timeout or spectator overflow (#6)
screened_out: "", // TODO(#7): failed the exposure item or declined the commitment gate (#13)
},

// --- Timing -------------------------------------------------------------------------------
// NOT in the original C&WG design. An UNCONDITIONAL wall-clock bound on every round, armed when
// the round starts — not a dropout detector. It ends a round at this value whether the partner
// is gone, slow, or simply thinking, and it is why #5 has to infer dropout from consecutive
// timeouts rather than read it off this. Watch it in piloting: trial 1 of a 12-figure board is
// the longest in the study, and a cap that truncates it would bite exactly where the effect is
// largest.
ROUND_TIMEOUT_MS: 180000,

// --- Data ---------------------------------------------------------------------------------
// TODO(#3): DataPipe experiment ID. Until this is set there is NO data egress except the
// voluntary download button on the final screen, i.e. every participant who does not finish
// contributes nothing.
DATAPIPE_EXPERIMENT_ID: "",

// Where the study is hosted, for the Prolific redirect. Kept here so it is obvious that the
// production value has to be a real https URL, not a file:// path.
PROLIFIC_SUBMIT_URL: "https://app.prolific.com/submissions/complete",
};

const jsPsych = initJsPsych();
const localAdapter = new jsPsychAdapterMultiplayerLocal({ persistParticipant: true });

Expand All @@ -78,6 +125,136 @@
// without exchanging anything.
const SCHEDULE = Tangrams.fullBoardSchedule(TRIALS, SEED);

// ===============================================================================================
// PROLIFIC IDENTIFIERS AND PROVENANCE (#4)
//
// Prolific requires that identifiers be recorded so paired responses can be linked, and without
// them collected data cannot be reconciled against submissions in order to approve payment.
// ===============================================================================================

// Prolific appends these to the study URL. Read from the ORIGINAL query string, which is safe:
// the LocalAdapter's replaceState preserves existing params when it adds ?mp_session=.
const prolificParams = new URLSearchParams(location.search);
const PROLIFIC_PID = prolificParams.get("PROLIFIC_PID");
const STUDY_ID = prolificParams.get("STUDY_ID");
const SESSION_ID = prolificParams.get("SESSION_ID");

// The dyad identifier. Read from the same source as SEED and for the same reason: it is the one
// value both partners share. Note the coupling — whatever fixes SEED for Firebase (#9) has to fix
// this in the same change, or rows will carry a dyad id that does not identify a dyad.
//
// Falls back to the SAME literal SEED uses, deliberately. A null here would put `dyad_id: null` on
// every row while the save filenames (#3) fall back to SEED, producing files named after a dyad
// whose rows claim not to belong to one — self-inconsistent data that still LOOKS reconcilable,
// which is worse than either failure alone. The fallback makes it consistent; the warning below
// makes it loud, because a constant dyad id shared across all dyads is still wrong.
const DYAD_ID = prolificParams.get("mp_session") ?? "cwg-fullboard";

// Attach to EVERY row. Analysis here is entirely dyad-level, so a row that cannot be traced to a
// pair is not analysable; and a row that cannot be traced to a PROLIFIC_PID cannot be paid for.
jsPsych.data.addProperties({
prolific_pid: PROLIFIC_PID,
study_id: STUDY_ID,
session_id: SESSION_ID,
dyad_id: DYAD_ID,
// Own participant id, so the two sides of a dyad are distinguishable within it.
participant_id: localAdapter.participantId ?? null,
// Run provenance — needed to interpret data collected while piloting at different values.
experiment_file: "reference-game-cwg.html",
config_trials: TRIALS,
config_n_stimuli: N,
config_round_timeout_ms: CONFIG.ROUND_TIMEOUT_MS,
// A 6×2 tangram grid on a small screen is a different task. Captured so that can be checked
// rather than assumed, and so a mobile block (#13) can be verified to have worked.
viewport_w: window.innerWidth,
viewport_h: window.innerHeight,
screen_w: window.screen?.width ?? null,
screen_h: window.screen?.height ?? null,
user_agent: navigator.userAgent,
});

// ===============================================================================================
// PROLIFIC SUBMISSION (#7)
//
// Without this NO participant can submit at all, including one who plays the whole game. Built as
// standalone helpers rather than inline in the final screen, because #5 and #6 both need to exit
// from paths that never reach `doneTrial`.
// ===============================================================================================

// The code is always shown as selectable text alongside the button. If the redirect is blocked —
// popup blocker, embedded webview, flaky network — a visible code still lets the participant
// submit manually. A stranded participant with no code is the failure this whole issue exists to
// remove, so it should not be reintroduced by trusting `location.href` to work.
function submissionBlockHTML(codeKey) {
const code = CONFIG.COMPLETION_CODES[codeKey];
if (!code) {
// Give an actionable route, not just an apology. Prolific participants can message the
// researcher from the study page, and telling them so is the difference between a resolved
// submission and an unpaid one. "Do not close this tab" alone strands exactly the person this
// whole issue exists to protect.
return `<p style="color:#b00"><strong>Something has gone wrong on our end.</strong> Your
responses have been recorded, but the completion code for this exit
(<code>${codeKey}</code>) is missing, so the automatic submission cannot run.</p>
<p style="color:#b00">Please <strong>message the researcher from this study's page on
Prolific</strong> and quote <code style="user-select:all">${codeKey}</code>. You will be
paid — do not return the study. Please keep this tab open until you have sent it.</p>`;
}
return `<p><button id="prolific-submit" type="button">Submit and return to Prolific</button></p>
<p style="font-size:0.9em">If that button does not work, your completion code is
<code style="user-select:all">${code}</code> — paste it into Prolific to submit.</p>`;
}

function redirectToProlific(codeKey) {
const code = CONFIG.COMPLETION_CODES[codeKey];
if (!code) {
console.error(`No Prolific completion code configured for "${codeKey}".`);
return;
}
window.location.href = `${CONFIG.PROLIFIC_SUBMIT_URL}?cc=${encodeURIComponent(code)}`;
}

function wireSubmissionButton(codeKey) {
document
.getElementById("prolific-submit")
?.addEventListener("click", () => redirectToProlific(codeKey));
}

// Fail loudly at startup rather than at the moment a participant tries to submit. Unset codes are
// the kind of thing that looks fine in a two-tab smoke test and strands a paid participant later.
(function warnOnUnsetConfig() {
const unset = Object.entries(CONFIG.COMPLETION_CODES)
.filter(([, v]) => !v)
.map(([k]) => k);
if (unset.length) {
console.warn(
`[config] Completion codes not set: ${unset.join(", ")}. ` +
`Participants reaching those exits will not be able to submit.`
);
}
if (!CONFIG.DATAPIPE_EXPERIMENT_ID) {
console.warn(
"[config] No DATAPIPE_EXPERIMENT_ID (#3). There is no data egress except the voluntary " +
"download button — every participant who does not finish will contribute nothing."
);
}
if (!PROLIFIC_PID) {
console.warn("[config] No PROLIFIC_PID in the URL. Expected when testing locally.");
}
// The highest-severity config fault, and the quietest. Every DV here is dyad-level, so rows that
// all share one fallback dyad id are not merely mislabelled — they collapse every pair into a
// single apparent dyad, which analysis cannot detect and cannot undo. Under Firebase (#9) this
// is the DEFAULT state until SEED and DYAD_ID are re-derived from the room id, so it must be
// impossible to miss.
if (!prolificParams.get("mp_session")) {
console.error(
`[config] No ?mp_session= in the URL — dyad_id fell back to "${DYAD_ID}". Every row will ` +
`carry the SAME dyad id, so partners cannot be paired and no dyad-level analysis is ` +
`possible. Expected on a first tab before the adapter writes the param; NOT expected once ` +
`a partner has joined, and never acceptable in collected data (#9).`
);
}
})();

// The 12 tangrams are real PNGs, and `rt` is a reported DV — without this the first exposure to
// each image loads mid-trial. `stimuli` is an OBJECT parameter, so jsPsych's automatic media
// preloading cannot discover the `src` values; they have to be listed explicitly.
Expand Down Expand Up @@ -161,7 +338,7 @@
// order, revision behavior. The core word-count DV doesn't need it, but it's ~free and matches
// the Hawkins file.
save_interaction_history: true,
round_timeout: 180000, // NOT in the original — bounds the round if the partner leaves (see header)
round_timeout: CONFIG.ROUND_TIMEOUT_MS, // NOT in the original — see CONFIG and the header
prompt: (role) =>
role === "director"
? "<p><strong>You are the Director.</strong> Describe your tangrams in badge order (1, 2, 3, …) so your partner can arrange theirs the same way. You can talk back and forth as much as you need.</p>"
Expand All @@ -186,10 +363,18 @@
<p>Together you scored <strong>${total} / ${played * N}</strong> across ${played} trial${
played === 1 ? "" : "s"
}.</p>
<p>You may close this tab. <button id="dl">Download data (JSON)</button></p>`;
<p>Thank you — please submit to complete the study.</p>
${submissionBlockHTML("complete")}
<p style="font-size:0.85em;color:#666">
Researcher tools: <button id="dl" type="button">Download data (JSON)</button>
</p>`;
},
choices: "NO_KEYS",
on_load: () => {
wireSubmissionButton("complete");
// Retained as a local fallback while there is no save target (#3). Once DataPipe is wired,
// this is a debugging convenience rather than the only egress — it should NOT be the thing a
// participant is relied upon to click.
document
.getElementById("dl")
?.addEventListener("click", () =>
Expand Down
6 changes: 4 additions & 2 deletions reference-game-hawkins.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,9 @@
//
// Every parameter above was verified against the paper + hawkrobe/tangrams code. The one departure
// is not a design choice: `round_timeout` (60s) has no counterpart in the original, which ran
// untimed. It exists so a disconnected or absent partner cannot hang the trial forever. Rounds it
// untimed. It was added so a disconnected or absent partner cannot hang the trial forever — but it
// is an UNCONDITIONAL wall-clock bound armed when the round starts, ending every round at that
// value whether or not anyone dropped. It is not a dropout detector. Rounds it
// ends are recorded as `ended_by: "timeout"` with a null assignment — filter on that when
// analysing, and raise it if piloting shows real trials running long.
//
Expand Down Expand Up @@ -163,7 +165,7 @@
// Guarantees a referring expression per trial and blocks blind-guessing.
require_message_before_response: true,
show_running_score: true,
round_timeout: 60000, // NOT in the original — bounds the round if the partner leaves (see header)
round_timeout: 60000, // NOT in the original — unconditional bound on EVERY round (see header)
save_interaction_history: true,
prompt: (role) =>
role === "director"
Expand Down