Correct the keys, help text, figures and report tables

DESCRIPTION

@@ -20,7 +20,7 @@ Description: Provides quantitative tools for substantive and content-oriented
     Krippendorff's alpha as described by Hayes and Krippendorff (2007)
     <doi:10.1080/19312450709336664>. Also provides judge and rater
     heterogeneity analysis following the generalizability-theory treatment of
-    content-validity ratings in Crocker, Llabre and Miller (1988)
+    content-validity ratings in Crocker et al. (1988)
     <doi:10.1111/j.1745-3984.1988.tb00309.x>, content-domain coverage and
     expert-perceived content structure following Sireci and Geisinger (1992)
     <doi:10.1177/014662169201600102>, consensus and stability across Delphi

NAMESPACE

@@ -1,8 +1,19 @@
 # Generated by roxygen2: do not edit by hand
 
+S3method(as.data.frame,contentvalid_agreement)
+S3method(as.data.frame,contentvalid_binom)
 S3method(as.data.frame,contentvalid_component)
+S3method(as.data.frame,contentvalid_cvi)
 S3method(as.data.frame,contentvalid_evidence)
+S3method(as.data.frame,contentvalid_expert_power)
+S3method(as.data.frame,contentvalid_gtheory)
+S3method(as.data.frame,contentvalid_handoff)
 S3method(as.data.frame,contentvalid_report)
+S3method(as.data.frame,contentvalid_reproducibility)
+S3method(as.data.frame,contentvalid_rounds)
+S3method(as.data.frame,contentvalid_signal)
+S3method(as.data.frame,contentvalid_sort_power)
+S3method(as.data.frame,contentvalid_structure)
 S3method(as.data.frame,contentvalid_workflow)
 S3method(plot,contentvalid_delphi)
 S3method(plot,contentvalid_evidence)

NEWS.md

@@ -584,6 +584,86 @@ analysis that matches the cases below.
 * The reporting vignette's congruence guidance reports the index against
   its criterion, with the means and margin as description.
 
+## Keys, help pages, figures and tables: values that change
+
+* **Report intervals are named after their estimates.** `content_report()`
+  returned two columns both named "95% CI" in a relevance report, so
+  `tab$"95% CI"` found only the first. Every interval column is now named
+  after its estimate (`Psa 95% CI`, `V 95% CI`, `I-CVI 95% CI`), and the
+  printed table and the Markdown head it "95% CI", beside that estimate. A
+  column the user renames prints under the new name.
+* **`plot()` takes `xlab`, `ylab`, `xlim`, `ylim` and `main` everywhere.**
+  Most plot methods set these themselves and also passed `...` on, so giving
+  one stopped with "formal argument matched by multiple actual arguments".
+  An argument the caller gives now replaces the method's own, except those
+  the figure's encoding depends on: the frame type, the axes the method
+  draws, and the decision symbols a map's legend keys. A `NULL` leaves the
+  method's value, and an unnamed argument is dropped with a warning. In the
+  distribution views and the evidence profile a title is drawn once, above
+  the legend, and the item labels take the place of a y-axis label.
+* **Long item names no longer stop a figure.** The distribution views and
+  the evidence profile sized the label margin to the longest name, and a
+  long one left no room to draw ("figure margins too large"). Labels are
+  measured on the device; one wider than 40% of the width left for labels
+  is shortened in the middle, keeping its start and end, so names that
+  share an opening stay apart.
+
+## Keys, help pages, figures and tables: other fixes
+
+* `as.data.frame()` works on every result: `cvi()`, `gtheory_content()`,
+  `content_structure()`, `compare_rounds()`, `content_handoff()`,
+  `sort_power()`, `expert_power()` and `panel_agreement()` had no method, and
+  `csv_binom_test()`, `signal_detection()` and `reproducibility_phi()` gave
+  two to four rows for one test. A result holding several tables takes
+  `component`; a single test is one row, with an interval as two columns and
+  a two-by-two table as four named counts. The agreement row names the
+  interval's error rate `ci_alpha`, and the binomial row marks its interval
+  one-sided. See `?"contentvalid-data-frames"`.
+* The key said modified kappa falls below 0 "when agreement is below
+  chance". It does so only when no expert, or one of three, rated the item
+  relevant: 2 of 8 gives .16.
+* The relevance key defines S-CVI/Ave and S-CVI/UA, which the header prints.
+* With `agreement = "ac1"`, the key describes AC1, which is not 0 for
+  independent raters and stays high when ratings concentrate; it repeated
+  the Krippendorff caution that the coefficient "can be low when nearly
+  every rating is the same". The generic agreement entry now describes
+  Krippendorff's alpha alone.
+* The glossary said judge severity is in logits when the model is
+  estimable. Its terms now follow the `results` columns: `severity_raw`,
+  printed as severity, in rating points, and `severity`, printed as logit,
+  from the facets model on the relevant/not-relevant decision, which can
+  differ even in sign. The key gives both.
+* The domain decision meanings state the rules: more than `over_factor`
+  times the expected share, and less than the target share divided by it.
+* Three-author works are cited with "et al." in the `cvi()` printout and
+  help, the legacy sort printout and `?sort_validity` (Yao et al., 2008),
+  the item-sort and reporting vignettes, and DESCRIPTION; "p value" is no
+  longer hyphenated; article numbers read "Article 93".
+* Help pages cite every work they list, by author and date: the interval
+  methods in `?compute_psa`, Colquitt et al. (2019) in `?htd`, Hinkin and
+  Tracey (1999) in `?htc`, Aiken (1980) in `?aikens_v`, Howard and Melloy
+  (2016) in `?simulate_csv_power` and `?sort_power`, Feinstein and
+  Cicchetti (1990) and Wongpakaran et al. (2013) in `?panel_agreement`, and
+  Penfield and Giacobbi (2004), Polit et al. (2007), Hayes and Krippendorff
+  (2007) and Zapf et al. (2016) in `?expert_validity`; the expert-panel and
+  reading-output vignettes likewise.
+* `?contentvalidR` promised `plot()` for all six workflows; judge and domain
+  fits have none, and it now says how to draw a domain fit's content map
+  when similarity data were supplied.
+* The reporting vignettes build their tables with `content_report()` rather
+  than from raw `results` columns, and point to the strongest competitor and
+  the scale-level CVIs, which those tables leave out.
+* `reading-output` says what `Strong` means: the 60th to 79th percentile of
+  the scales Colquitt et al. (2019) collected, not "typical of published
+  work".
+* The walkthrough says seven items, not five, each set a test for one
+  stage; that the fifteen assignments follow from the null probability of
+  .50, not from "two plausible answers"; calls `csv` the substantive validity
+  coefficient; and no longer says the judges "never considered", or "saw
+  only one of", the facets of an item four of them sorted elsewhere.
+* The handoff vignette's nomologR install line names CRAN as well as
+  R-universe, so it works in a fresh library.
+
 ## Other changes
 
 * The reader in nomologR is now tested against handoffs from contentvalidR

R/aikens_v.R

@@ -1,9 +1,9 @@
 #' Aiken's V for expert content-relevance ratings
 #'
 #' @description
-#' Computes Aiken's V per item for bounded ordinal expert ratings. By default,
-#' confidence intervals use the score method described by Penfield and
-#' Giacobbi (2004). Percentile bootstrap intervals remain available for
+#' Computes Aiken's (1980) V per item for bounded ordinal expert ratings. By
+#' default, confidence intervals use the score method described by Penfield
+#' and Giacobbi (2004). Percentile bootstrap intervals remain available for
 #' compatibility and sensitivity analysis.
 #'
 #' @param ratings Matrix/data.frame with judges in rows and items in columns.

R/as_data_frame.R

@@ -0,0 +1,194 @@
+#' Tables from contentvalidR results
+#'
+#' @description
+#' `as.data.frame()` returns a result's table as an ordinary data frame, for
+#' filtering, joining, or writing to a file. A result that holds several
+#' tables returns the one named by `component`; a single test returns one row.
+#' Fitted workflows have their own method,
+#' [as.data.frame.contentvalid_workflow()].
+#'
+#' @param x A contentvalidR result.
+#' @param row.names,optional Accepted for compatibility with
+#'   [base::as.data.frame()] and ignored.
+#' @param component The table to return, where a result holds more than one
+#'   (`NULL`, the default, gives the first listed):
+#'   * [cvi()]: `"item_level"` (default) or `"scale_level"`.
+#'   * [gtheory_content()]: `"variance_components"` (default),
+#'     `"coefficients"`, `"dstudy"`, or `"judges_needed"`.
+#'   * [content_structure()]: `"items"` (default; each item's cluster,
+#'     blueprint cell and coordinates), `"fit"`, or `"cross_tab"`.
+#'   * [compare_rounds()]: `"transitions"` (default), `"summary"`, or
+#'     `"settings_changes"`.
+#'   * [content_handoff()]: `"item_evidence"` (default), `"item_statistics"`,
+#'     or `"panel_statistics"`.
+#' @param ... Not used.
+#'
+#' @return A data frame. [csv_binom_test()], [signal_detection()],
+#'   [reproducibility_phi()] and [panel_agreement()] give one row: an
+#'   interval becomes two columns, and a two-by-two table becomes its four
+#'   counts. For [panel_agreement()] the interval's error rate is `ci_alpha`,
+#'   since `alpha` would read as Krippendorff's; for [csv_binom_test()] the
+#'   one-sided interval is marked by `ci_sides` and `ci_level`.
+#'
+#' @examples
+#' R <- matrix(c(4, 3, 4, 4, 3, 4, 2, 3, 4, 4, 3, 2), 6,
+#'             dimnames = list(NULL, c("I1", "I2")))
+#' as.data.frame(cvi(R >= 3))
+#' as.data.frame(cvi(R >= 3), component = "scale_level")
+#' as.data.frame(csv_binom_test(16, 20))
+#' @name contentvalid-data-frames
+NULL
+
+# One table out of a result holding several.
+.as_df_part <- function(x, component, choices) {
+  component <- match.arg(component, choices)
+  out <- x[[component]]
+  if (is.null(out)) {
+    stop("This result has no `", component, "` table.", call. = FALSE)
+  }
+  out <- as.data.frame(out, stringsAsFactors = FALSE)
+  rownames(out) <- NULL
+  out
+}
+
+# One row from a single test: scalars as they are, an interval as two
+# columns, and a two-by-two table as its four counts.
+.as_df_row <- function(x) {
+  x <- unclass(x)
+  cols <- list()
+  for (nm in names(x)) {
+    v <- x[[nm]]
+    if (is.table(v) || is.matrix(v)) {
+      if (length(dim(v)) == 2L && all(dim(v) == 2L)) {
+        # Each count is named by its row and column: "n_predicted_retain_
+        # actual_not_retained" for a confusion table.
+        dn <- dimnames(v)
+        tidy <- function(z) gsub("[^a-z0-9]+", "_", tolower(z))
+        cells <- if (length(dn) == 2L && !is.null(names(dn)) &&
+                     all(nzchar(names(dn)))) {
+          as.vector(outer(paste(names(dn)[1], dn[[1]]),
+                          paste(names(dn)[2], dn[[2]]), paste))
+        } else {
+          paste(nm, c("11", "21", "12", "22"))
+        }
+        cols[paste0("n_", tidy(cells))] <- as.list(as.vector(v))
+      }
+    } else if (is.atomic(v) && length(v) == 1L) {
+      cols[[nm]] <- v
+    } else if (is.atomic(v) && length(v) == 2L) {
+      stem <- if (identical(nm, "conf.int")) "ci" else nm
+      cols[paste0(stem, c("_low", "_high"))] <- as.list(as.vector(v))
+    }
+  }
+  as.data.frame(cols, stringsAsFactors = FALSE, check.names = FALSE)
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_cvi <- function(x, row.names = NULL, optional = FALSE,
+                                           component = NULL, ...) {
+  if (is.null(component)) component <- "item_level"
+  .as_df_part(x, component, c("item_level", "scale_level"))
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_gtheory <- function(x, row.names = NULL, optional = FALSE,
+                                               component = NULL, ...) {
+  if (is.null(component)) component <- "variance_components"
+  .as_df_part(x, component, c("variance_components", "coefficients",
+                              "dstudy", "judges_needed"))
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_structure <- function(x, row.names = NULL,
+                                                 optional = FALSE,
+                                                 component = NULL, ...) {
+  if (is.null(component)) component <- "items"
+  component <- match.arg(component, c("items", "fit", "cross_tab"))
+  if (component == "items") {
+    # The clusters table already carries each item's coordinates.
+    out <- x$clusters
+    rownames(out) <- NULL
+    return(out)
+  }
+  if (component == "cross_tab") {
+    if (is.null(x$cross_tab)) {
+      stop("This result has no `cross_tab` table: no blueprint was given.",
+           call. = FALSE)
+    }
+    out <- as.data.frame(x$cross_tab, stringsAsFactors = FALSE)
+    names(out)[ncol(out)] <- "n_items"
+    return(out)
+  }
+  .as_df_part(x, component, "fit")
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_rounds <- function(x, row.names = NULL, optional = FALSE,
+                                              component = NULL, ...) {
+  if (is.null(component)) component <- "transitions"
+  .as_df_part(x, component, c("transitions", "summary", "settings_changes"))
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_handoff <- function(x, row.names = NULL, optional = FALSE,
+                                               component = NULL, ...) {
+  if (is.null(component)) component <- "item_evidence"
+  .as_df_part(x, component, c("item_evidence", "item_statistics",
+                              "panel_statistics"))
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_sort_power <- function(x, row.names = NULL,
+                                                  optional = FALSE, ...) {
+  .as_df_part(x, "table", "table")
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_expert_power <- function(x, row.names = NULL,
+                                                    optional = FALSE, ...) {
+  .as_df_part(x, "results", "results")
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_agreement <- function(x, row.names = NULL,
+                                                 optional = FALSE, ...) {
+  out <- .as_df_row(x)
+  # `alpha` here is the interval's error rate, not Krippendorff's alpha.
+  names(out)[names(out) == "alpha"] <- "ci_alpha"
+  out
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_binom <- function(x, row.names = NULL, optional = FALSE,
+                                             ...) {
+  out <- .as_df_row(x)
+  # The interval is one-sided, as the printout says.
+  if ("ci_low" %in% names(out)) {
+    out$ci_sides <- "one-sided"
+    out$ci_level <- attr(x$conf.int, "conf.level")
+  }
+  out
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_signal <- function(x, row.names = NULL, optional = FALSE,
+                                              ...) {
+  .as_df_row(x)
+}
+
+#' @rdname contentvalid-data-frames
+#' @export
+as.data.frame.contentvalid_reproducibility <- function(x, row.names = NULL,
+                                                       optional = FALSE, ...) {
+  .as_df_row(x)
+}

R/compute_psa.R

@@ -12,8 +12,11 @@
 #' @param assignments A data.frame containing item-sort responses.
 #' @param item_col,rater_col,assigned_col,target_col Column names for the item,
 #'   rater, assigned construct, and intended target construct.
-#' @param ci Interval method for Psa: `"wilson"` (default), `"agresti_coull"`,
-#'   `"exact"`, or `"none"`. The methods and the evidence for each are described
+#' @param ci Interval method for Psa: `"wilson"` (default), the score interval
+#'   of Wilson (1927); `"agresti_coull"`, the adjusted Wald interval of Agresti
+#'   and Coull (1998); `"exact"`, the Clopper and Pearson (1934) interval; or
+#'   `"none"`. Newcombe (1998) compared seven methods and recommends score
+#'   intervals over the Wald interval; the evidence for each is described
 #'   under `ci` in [cvi()].
 #' @param alpha Two-sided alpha level for the interval; `0.05` gives a 95%
 #'   interval.

R/content_evidence.R

@@ -449,8 +449,13 @@ plot.contentvalid_evidence <- function(x, type = c("profile", "flow"),
 
   op <- graphics::par(no.readonly = TRUE)
   on.exit(graphics::par(op), add = TRUE)
-  graphics::par(oma = c(0, 1.2 + 0.62 * max(nchar(items)),
-                        if (show_legend) 1.6 else 0.3, 0.5),
+  lab <- .item_labels(items, width_in = graphics::par("din")[1])
+  # A title is drawn once, above every panel, not on each one.
+  dots <- list(...)
+  main <- dots$main
+  graphics::par(oma = c(0, lab$lines,
+                        (if (show_legend) 1.6 else 0.3) +
+                          if (length(main)) 1.6 else 0, 0.5),
                 mar = c(4.1, 0.6, 1.9, 0.6))
   graphics::layout(matrix(seq_len(ns + 1L), 1), widths = c(rep(1, ns), 0.75))
 
@@ -462,14 +467,15 @@ plot.contentvalid_evidence <- function(x, type = c("profile", "flow"),
     stat <- s$statistic[1]
     xlim <- switch(stat, CVR = c(-1, 1), IOC = c(-1, 1), `target IOC` = c(-1, 1),
                    `highest IOC` = c(-1, 1), `IOC margin` = c(-2, 2), c(0, 1))
-    graphics::plot(NA, xlim = xlim, ylim = c(0.4, n + 0.6), xaxt = "n",
-                   yaxt = "n", xlab = stat, ylab = "", ...)
+    .plot_with(list(x = NA, xlim = xlim, ylim = c(0.4, n + 0.6), xaxt = "n",
+                    yaxt = "n", xlab = stat, ylab = ""), dots,
+               protect = c("type", "xaxt", "yaxt", "axes", "main", "ylab"))
     if (stat == "IOC margin") {
       graphics::axis(1, at = seq(-2, 2))
     } else {
       .axis_bounded(1, at = seq(xlim[1], xlim[2], length.out = 5))
     }
-    if (k == 1L) graphics::axis(2, at = y, labels = items, las = 1)
+    if (k == 1L) graphics::axis(2, at = y, labels = lab$labels, las = 1)
     if (length(breaks)) graphics::abline(h = y[breaks] - 0.5, col = "grey80")
     row <- match(s$item, items)
     crit <- unique(s$criterion[is.finite(s$criterion)])
@@ -526,6 +532,10 @@ plot.contentvalid_evidence <- function(x, type = c("profile", "flow"),
     graphics::mtext(paste(parts, collapse = "   "), side = 3, outer = TRUE,
                     line = 0.2, cex = 0.72)
   }
+  if (length(main)) {
+    graphics::title(main = main, outer = TRUE,
+                    line = (if (show_legend) 1.6 else 0.3) + 0.2)
+  }
   invisible(NULL)
 }
 

R/content_handoff.R

@@ -861,7 +861,7 @@
 #'
 #' The four columns are `NA` together when a statistic has no interval. That
 #' happens when the method defines none (Csv, HTC, HTD, CVR, the essential
-#' count, modified kappa, IOC, and p-values), when intervals were switched off
+#' count, modified kappa, IOC, and p values), when intervals were switched off
 #' with `proportion_ci = "none"`, or when the statistic itself could not be
 #' computed. `NA` there never stands for missing data.
 #'