Feature/summary - #98
Conversation
b911880 to
3d21aca
Compare
|
This comes from a feature request from Mario D' Antuono from WA, especially the connectedness part. |
There was a problem hiding this comment.
Comments are a bit too verbose to me, but that's a preference I think.
There was a problem hiding this comment.
I think we should stick to an explicit return (in the function under lapply, vapply, .. as well).
Also, comments could be more concise.
Buffer edge case
- If a user adds buffer after the optimization, but before the
summarycall,.neighbour_balancewill throw a warning when making a matrix from mismatched dimension and result in incorrect score. - Design efficiency cannot be calculated post-buffer -- not sure if this is expected.
| print.summary.design <- function(x, ...) { | ||
| pad <- 14 | ||
| lab <- function(s) formatC(s, width = -pad) | ||
| indent <- strrep(" ", pad) | ||
| fmt_int <- function(n) format(n, big.mark = ",", scientific = FALSE, trim = TRUE) | ||
| fmt_num <- function(n) format(round(n, 4), big.mark = ",", trim = TRUE) | ||
| section <- function(title) cat("\n", title, "\n", strrep("-", nchar(title)), "\n", sep = "") | ||
|
|
||
| cat("Design Summary\n") | ||
| cat("==============\n") | ||
|
|
||
| # --- Flags (only when something is worth flagging) --- | ||
| flag_lines <- character() | ||
| if (length(x$flags$hit_iteration_cap)) { | ||
| where <- if (x$hierarchical) { | ||
| paste0(" (", paste(x$flags$hit_iteration_cap, collapse = ", "), ")") | ||
| } else { | ||
| "" | ||
| } | ||
| flag_lines <- c(flag_lines, | ||
| paste0("! Ran to iteration cap - may not have converged", where)) | ||
| } | ||
| if (length(x$flags$disconnected)) { | ||
| where <- if (x$hierarchical) { | ||
| paste0(" (", paste(x$flags$disconnected, collapse = ", "), ")") | ||
| } else { | ||
| "" | ||
| } | ||
| flag_lines <- c(flag_lines, paste0("! DISCONNECTED design", where)) | ||
| } | ||
| if (isTRUE(x$flags$unequal_replication)) { | ||
| flag_lines <- c(flag_lines, "! Unequal replication (expected for p-rep designs)") | ||
| } | ||
| if (length(flag_lines)) { | ||
| section("Flags") | ||
| for (fl in flag_lines) cat(fl, "\n", sep = "") | ||
| } | ||
|
|
||
| # --- Structure --- | ||
| section("Structure") | ||
| lo <- x$layout | ||
| if (isTRUE(lo$has_grid)) { | ||
| cat(lab("Layout:"), | ||
| sprintf("%d rows x %d cols (%s plots)", lo$nrow, lo$ncol, fmt_int(lo$n_plots)), | ||
| "\n", sep = "") | ||
| } else { | ||
| cat(lab("Layout:"), sprintf("%s plots", fmt_int(lo$n_plots)), "\n", sep = "") | ||
| } | ||
| for (lv in x$levels) .print_level_structure(x, lv, lab, indent, fmt_num) | ||
|
|
||
| # --- Optimisation --- | ||
| section("Optimisation") | ||
| cat(lab("Seed:"), x$seed, "\n", sep = "") | ||
| # The overall score is the sum of per-level scores; only meaningful to show | ||
| # separately when there is more than one level. | ||
| if (x$hierarchical) cat(lab("Total score:"), fmt_num(x$score), "\n", sep = "") | ||
| for (lv in x$levels) .print_level_optim(x, lv, lab, indent, fmt_int, fmt_num) | ||
|
|
||
| # --- Evaluation --- | ||
| section("Evaluation") | ||
| for (lv in x$levels) .print_level_evaluation(x, lv, lab, fmt_num) | ||
|
|
||
| invisible(x) | ||
| } |
There was a problem hiding this comment.
Should we add flag when running efficiency on design with spatial factors other than row and column?
I can see that we will eventually move to dae (can be optional if it's heavy) with <stuff1> + ... + row * col, so if that's happening soonTM, I guess we can ignore this.
The "ran to iteration cap" could be adjusted a bit when iterations is actually less than stop early iterations.
We could use text color/weight here
- Make flag stand out a bit more:
crayon::bold(crayon::magenta('message\n')) - Section readability/compact:
cat(crayon::bold('Section\n')) - Stop reason: bold magenta 'ran to cap' and blue/green 'stopped early' -- I think this is an important info.
Example
| # --------------------------------------------------------------------------- | ||
| # Phase 4 evaluation helpers (base R, no new dependencies). Each returns a list | ||
| # with an `available` flag; when FALSE it carries a `reason` string so the | ||
| # printer can show a one-line note instead of a value. | ||
| # --------------------------------------------------------------------------- |
| run <- if (hierarchical) length(object$scores[[lv]]) else object$iterations_run | ||
| stopped <- if (hierarchical) isTRUE(object$stopped_early[[lv]]) else isTRUE(object$stopped_early) | ||
|
|
||
| # --- Evaluation metrics (Phase 4) --- |
There was a problem hiding this comment.
Should we drop "(Phase 4)"?
| swap_cols <- swap_cols[swap_cols %in% names(df)] | ||
| if (!length(swap_cols)) return(df) |
There was a problem hiding this comment.
These lines are redundant with verification in speed call.
| #' @param connectedness `NULL` (default) checks whether treatments are estimable | ||
| #' (connected), but skips the check for very large designs where the model fit | ||
| #' would be expensive; `TRUE` forces it regardless of size; `FALSE` skips it. | ||
| #' The check fits `lm(~ <spatial factors + block> + treatment)` and looks for | ||
| #' aliased treatment contrasts. | ||
| #' @param concurrence `NULL` (default) computes within-block treatment | ||
| #' concurrences only when an *incomplete* block factor is present (they are | ||
| #' uninformative for complete blocks such as RCBD/split-plot); `TRUE` forces | ||
| #' them even for complete blocks, `FALSE` skips them. |
There was a problem hiding this comment.
We might need clear explanation on connectedness and/or concurrence here. E.g., users should be able to interpret what 'lambda 2-3' means for connectedness.
For connectedness, maybe I'm not familiar with 'estimable', so a bit lost.
| #' Produces a richer, statistically meaningful evaluation of a design than | ||
| #' [print()][print.design()]. Where `print()` is a compact output, | ||
| #' `summary()` decomposes the optimised score and reports structural and | ||
| #' evaluation metrics that let you interrogate and defend a design. | ||
| #' | ||
| #' The returned object is a list of class `"summary.design"`; it can be assigned | ||
| #' and queried programmatically (e.g. `s <- summary(d); s$per_level[[1]]$score`). | ||
| #' Printing it is handled by [print.summary.design()]. |
There was a problem hiding this comment.
I think explanation on non-self-explanatory component should be added, e.g., replication span, connectedness, and concurrence. The explanation for the latter 2 could be added in params (?).
| list(available = FALSE, reason = "not requested (set efficiency = TRUE)") | ||
| }, | ||
| neighbour = if (want_neighbour) { | ||
| .neighbour_balance(df, swap, rc, cc) |
There was a problem hiding this comment.
Reusing layout$nrow and layout$ncol might resolve the warning with buffer thingy if the intention is right.
| } else { | ||
| list(available = FALSE, reason = "not requested (set efficiency = TRUE)") | ||
| }, | ||
| neighbour = if (want_neighbour) { |
There was a problem hiding this comment.
Also check with has_grid here and remove internal check in .neighbour_balance?
2 participants
Add
summary()fordesignobjects (and a leanerprint())What
Adds a
summary.design()method (with aprint.summary.design()) that turns a design object into a statistically meaningful evaluation.print()gives a snapshot of main optimisation results, andsummary()lets you evaluate and defend it. Targetsfeature/summary; bumps to 0.0.9.Why
The single combined score that
print()showed isn't enough to judge a design. Biometricians (and workshop audiences) need to see replication, how the score breaks down, and design-quality diagnostics like connectedness and concurrence without dropping into the object internals by hand.Changes
summary.design— returns a queryable"summary.design"list (printing is formatting-only) in three sections:Self-describing design objects —
speed()results now carry ametadatafield recording each level's swap variable, spatial factors, weights, optimisation settings, achieved score and score components. This is the prerequisite that letssummary()report faithfully rather than re-derive.Key design decisions
componentsvector (the additive pieces that sum to its score), captured into metadata with the exact arguments used. So the decomposition is always faithful — including non-additive objectives (Piepho/factorial) and ring-weighted adjacency — and custom objectives that don't expose components simply show the score with no decomposition.lm(~ spatial factors + block + treatment), counting aliased treatment contrasts), base R, nolme4. It adjusts for whatever the design is stratified by, so incomplete-block designs are assessed the way they're actually analysed. It is size-guarded: skipped for very large designs (where the dense fit isexpensive) unless forced.
efficiency = TRUE) — it is the heaviest metric and assumes a row–column model.add_buffers()are a field-layout convenience, not part of the statistical design, soprint()andsummary()silently ignore them.Tests / docs
New
test-summary.Rplus additions across the objective/efficiency tests; roxygen and runnable examples for the new methods; a new "Evaluating a Design" vignette section; aNEWS.mdentry. Full test suite green.