Skip to content

feat: support multiple source datasets in one meta_adam (#174) - #183

Draft
yihui wants to merge 5 commits into
mainfrom
174-multiple-source-datasets
Draft

yihui wants to merge 5 commits into
mainfrom
174-multiple-source-datasets

Conversation

@yihui

@yihui yihui commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #174 (scenario 3: one metadata object driving analyses that read from different source datasets — e.g. an AE summary from adae plus a lab box plot from adlb).

Previously meta_adam() held a single observation dataset, so multiple named observations were only ever subsets of one source. Multiple distinct source datasets were impossible.

API

meta_adam()'s observation/population arguments now accept datasets in the package's usual ";"-separated style (matching the semicolon-list convention used throughout). Each argument takes one of:

  • a data frame — unchanged, backward compatible;
  • a ";"-separated string of dataset names (e.g. "adae;adlb"), each retrieved with get();
  • a named list of data frames.

When more than one dataset is supplied, the first is the primary dataset (kept in its own slot as the single source of truth) and the rest are registered in data_source.

define_population(from = ...) / define_observation(from = ...) select which registered dataset a term reads from. Defaults "population" / "observation" (resolve to the primary slots). The primary dataset's own name also resolves to the live slot, so it stays in sync with in-place edits (meta_add_total(), meta_split()).

Usage

adsl <- forestly_adsl
adae <- forestly_adae
adlb <- boxly_adlb

meta <- meta_adam(population = "adsl", observation = "adae;adlb") |>
  define_plan(plan) |>
  define_observation(name = "apat", group = "TRTA", subset = SAFFL == "Y", var = "AEDECOD") |>
  define_observation(name = "lb", group = "TRTA", subset = SAFFL == "Y", var = "AVAL", from = "adlb") |>
  ...
  meta_build()

meta |> collect_observation_record("apat", "apat", "rel")  # reads adae
meta |> collect_observation_record("apat", "lb", "alt")    # reads adlb

Backward compatibility

Data-frame observation/population and no from behave exactly as before. Full existing suite passes; the print() snapshot was updated for the new from column (with pinned console width) and _snaps was un-ignored so the reference ships in the package build.

Tests

test-independent-testing-multi-source.R covers the ";"-string and named-list forms, from routing to distinct datasets, primary-name resolution, unregistered-from / non-data-frame / unnamed-list errors, and the single-source backward-compat path.

🤖 Generated with Claude Code

yihui and others added 5 commits September 1, 2026 13:49
Previously a `meta_adam` object could hold only a single observation
dataset, so a single metadata object could not drive analyses that read
from different source datasets (e.g. an AE table from `adae` plus a lab
box plot from `adlb`).

This adds a named source-dataset registry:

- `meta_adam(..., adlb = adlb)` registers extra datasets by name. The
  primary `population`/`observation` datasets keep their own slots and
  remain the single source of truth; only extra datasets live in the new
  `data_source` list.
- `define_population(from = ...)` and `define_observation(from = ...)`
  select which registered source a term reads from. Defaults are
  `"population"` and `"observation"` respectively, so existing metadata
  is unaffected.
- `collect_data_source()` resolves a term's `from` to its data frame;
  `collect_*()` and `meta_validate()` route through it. Parameters have
  no `from`; they are validated against the source of each observation
  they are paired with in the plan.

Backward compatible: without `from`/`...`, behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The print.meta_adam snapshot wraps the added `from` column based on the
console width, which differs between local and CI (R CMD check) runs.
Pin width = 80 so the snapshot is deterministic across environments.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_snaps` was listed in .Rbuildignore, so R CMD check built the package
tarball without the reference snapshot. Under current testthat this makes
the snapshot test report 'Adding new snapshot' and fail during check
(while passing under a plain test run). Remove the ignore so the reference
snapshot ships and is compared during R CMD check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow reviewer feedback: rather than passing extra datasets through a
`...` argument, let the `observation`/`population` arguments of
`meta_adam()` accept the datasets themselves in the package's usual
";"-separated style. This matches the semicolon-list convention used
throughout the package.

`observation`/`population` now accept:
- a data frame (unchanged, backward compatible);
- a ";"-separated string of dataset names (e.g. "adae;adlb"), each
  retrieved with get() from the calling environment;
- a named list of data frames.

When more than one dataset is given, the first is the primary dataset
(kept in its own slot as the single source of truth) and the rest are
registered in `data_source`. `define_observation(from = ...)` /
`define_population(from = ...)` select any registered dataset by name;
the primary dataset name and the generic keywords "observation"/
"population" both resolve to the live primary slot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@elong0527

Copy link
Copy Markdown
Collaborator

I am wondering about the motivation for introducing more than one dataset into a single metalite object, as this starts to expand the scope of metalite and may introduce additional questions.

Originally, a metalite object was designed to group similar analyses together, assuming they shared the same subject-level and observation-level data.

If analyses required separate observation data, the recommendation was to create multiple metalite objects to separate them (e.g., one metalite object for safety analysis and another for LB analysis).

The full CSR could then be compiled using a collection of metalite objects.

@LittleBeannie

Copy link
Copy Markdown
Collaborator

I am wondering about the motivation for introducing more than one dataset into a single metalite object, as this starts to expand the scope of metalite and may introduce additional questions.

Originally, a metalite object was designed to group similar analyses together, assuming they shared the same subject-level and observation-level data.

If analyses required separate observation data, the recommendation was to create multiple metalite objects to separate them (e.g., one metalite object for safety analysis and another for LB analysis).

The full CSR could then be compiled using a collection of metalite objects.

Thanks for sharing your insights! I agree with you that a list of multiple metalite objects serves our needs. However, using multiple metalite objects may require duplicate definitions. For example, the AE analysis and LAB analysis may share the same population_where. If we have one metadata object for AE and another for LAB, we would need to define the same population twice, which violates the "single-entry" principle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build 1 metadata for multiple interactive outputs

3 participants