Skip to content

Analysis: statistical conventions for the charts (lab review) #132

Open

Description

@Luen

What. PR #122 (for #119) adds a statistics line under every chart that shows a trend, a relationship or a difference between groups. Before this becomes the house style, the lab may want to change some of the choices:

  • Threshold. A pattern is called unlikely to be chance at p < 0.05. The Bonferroni correction is applied only inside the correlation grids on /drivers and /pressure; every other chart is tested on its own.
  • Catch rates. These are compared with a negative-binomial model, with log(net-hours) as the offset. "Adjusted" tests add nursery and season as fixed effects (each against the average) and a natural spline of day of the season.
  • Scatters report R² and p for a fitted line, or Spearman ρ.
  • Group differences use Kruskal–Wallis, Welch's t-test or χ².

Questions for @JodieRummer

  1. Is p < 0.05 the right threshold? Should the site correct for multiple tests across a whole page (e.g. false discovery rate), rather than only within the correlation grids?
  2. Should season and nursery be random effects in the catch-rate tests, as they are in the sex-ratio model (Analysis: sex ratio by nursery and season (mixed model) #117)?
  3. For a public site, are p-values and R² the right things to show? Or would you prefer effect sizes with 95% intervals, described in plain words ("no clear difference")?

Acceptance

  • AGENTS.md records the lab's choices (chart statistics).
  • src/lib/server/analytics/tests.ts applies them.

Part of #73.

Activity

  1. added
    questionFurther information is requested
    analyticsCharts, statistics and analyses
    on Nov 3, 2026
  2. JodieRummer commented on Nov 5, 2026

    @JodieRummer
    Collaborator
    1. yes p < 0.05 is the right threshold. I don't know about the whole page/correlation grids - needs more info.
    2. season and nursery are fixed effects because they don't change.
    3. both are fine. Use your best judgement. this data site is primarily for Dr Jodie Rummer (me) and my lab team.
  3. Luen commented on Nov 8, 2026

    @Luen
    CollaboratorAuthor

    Thanks. #167 records your choices in the project notes: p < 0.05, nursery and season as fixed effects, and p-values and R² stay as they are.

    More on Q1 (page-wide correction). Each test at p < 0.05 has a 1 in 20 chance of a false alarm when nothing real is going on. A page with 20 tests will therefore show about one "significant" result by chance alone. A false discovery rate (FDR) correction raises the bar on busy pages, so that across the page no more than about 5% of the results called significant are expected to be chance.

    What it would change on the site today, counting every p-value under the charts (real data, 24 pages):

    • Scale: 126 p-values outside the two correlation grids, which already use Bonferroni. 73 are below 0.05, and 54 of those are below 0.001.
    • With a 5% FDR per page: 71 of the 73 stay significant. Two would not:
      • the overview's blacktip season difference (p = 0.03);
      • /recaptures' recapture rate by nursery (p = 0.04).
    • With one correction across the whole site: 69 of the 73 stay.

    So most results are far from the threshold, and a page-wide correction would mainly flag a couple of borderline ones.

    @JodieRummer:

    1. Page-wide correction: which do you prefer?

      • (a) Keep testing each chart on its own at p < 0.05, as now. Changes nothing.
      • (b) Also control the FDR at 5% within each page. The two results above would be worded as "weak, may be chance".

      I'd suggest (b): it's a light touch, and it stops busy pages over-reading borderline results.

    2. The sex-ratio model: you said nursery and season are fixed effects, but the sex-ratio model on /morphometrics (Analysis: sex ratio by nursery and season (mixed model) #117) treats them as random. Random effects pull nurseries and seasons with few sharks towards the average. Should it use fixed effects too, like the catch-rate tests? The nursery comparison would become an ordinary logistic regression, and thin nurseries would show wider intervals instead of being pulled in. It also decides how Biology: does a nursery's sex ratio change between seasons? (nursery × season term) #165 (nursery × season) is built.

  4. added a commit that references this issue on Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    analyticsCharts, statistics and analysesquestionFurther information is requested

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions