Skip to content

Disentangle LoRA rank from data mixture in the final ablation step #4

Description

@Awshesh12

The ablation ladder in Table 5 goes 43.6 → 61.8 → 80.8 ScriptAcc. The first step cleanly isolates Conformer encoder LoRA (+18.2, data held fixed). The last step changes two things at once: LoRA rank 8 → 32 and the addition of the FLEURS-balanced data mix.

The paper states the confound explicitly rather than hiding it, but that costs us the ability to attribute a +19.0 point gain — the single largest jump in the table.

What's needed

Two runs to break the tie:

  1. r=32 encoder LoRA, without the FLEURS mix (original data)
  2. r=8 encoder LoRA, with the FLEURS mix

Either one splits the effect. Both give a clean 2x2.

Done when

Table 5 has a row that isolates rank from data, and §7.2 can attribute the gain instead of flagging a confound.

Metadata

Metadata

Assignees

No one assigned

    Labels

    paper-blockerMust be resolved before the paper can be submittedtrainingFine-tuning recipes, ablations, hyperparameters

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions