Fix five spec-conformance defects (parser, @input types, layout min, json() lens, CLI stdout)

CHANGELOG.md

@@ -9,6 +9,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/SemVer
 
 ### Added
 - Initial release preparation
+- Regression suite `tests/spec_conformance_regressions.rs` covering the conformance
+  gaps fixed below, including a check that `run --format json` keeps stdout parseable.
+
+### Fixed
+- **Parser**: a comment line with an empty body (a bare `#`) no longer fails with
+  `F003`. Appendix B defines `COMMENT = "#" *(%x20-10FFFF) NL`, i.e. zero or more
+  characters after the marker.
+- **Runtime inputs**: `@input` now validates against the full type system. Composite
+  FTS types — `list<T>`, `map<string, T>`, `struct { ... }`, unions and multimodal
+  assets — previously always failed with `F453`, even for conforming values, because
+  only primitives were matched. `int` is still widened to `float` per §8.4.
+- **Token Box Model**: a `min` larger than the section's own content no longer makes
+  that section undroppable and no longer lets the layout exceed the budget. `min` is
+  clamped to the section size before truncation and drop decisions (§11.3).
+- **`json()` lens**: renders the value as plain JSON instead of the compiler's internal
+  tagged `ValueNode` representation, and honours `indent` both as a positional and as a
+  named argument (Appendix A). Unevaluated nodes are now an explicit error.
+- **CLI**: diagnostics go to stderr, so stdout carries only machine-readable output and
+  `facet-fct run --format json | jq` works. ANSI colouring is disabled when stderr is
+  not a terminal. This makes the log-stripping in `scripts/spec_matrix_examples.sh`
+  redundant.
+- **CLI**: the startup log reported the host default budget and the gas limit as if
+  they were the layout budget; it now reports the effective layout budget after
+  `@context budget` is applied.
 
 ## [0.1.2] - 2026-04-02
 
@@ -206,6 +230,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/SemVer
 
 ### Added
 - Initial release preparation
+- Regression suite `tests/spec_conformance_regressions.rs` covering the conformance
+  gaps fixed below, including a check that `run --format json` keeps stdout parseable.
+
+### Fixed
+- **Parser**: a comment line with an empty body (a bare `#`) no longer fails with
+  `F003`. Appendix B defines `COMMENT = "#" *(%x20-10FFFF) NL`, i.e. zero or more
+  characters after the marker.
+- **Runtime inputs**: `@input` now validates against the full type system. Composite
+  FTS types — `list<T>`, `map<string, T>`, `struct { ... }`, unions and multimodal
+  assets — previously always failed with `F453`, even for conforming values, because
+  only primitives were matched. `int` is still widened to `float` per §8.4.
+- **Token Box Model**: a `min` larger than the section's own content no longer makes
+  that section undroppable and no longer lets the layout exceed the budget. `min` is
+  clamped to the section size before truncation and drop decisions (§11.3).
+- **`json()` lens**: renders the value as plain JSON instead of the compiler's internal
+  tagged `ValueNode` representation, and honours `indent` both as a positional and as a
+  named argument (Appendix A). Unevaluated nodes are now an explicit error.
+- **CLI**: diagnostics go to stderr, so stdout carries only machine-readable output and
+  `facet-fct run --format json | jq` works. ANSI colouring is disabled when stderr is
+  not a terminal. This makes the log-stripping in `scripts/spec_matrix_examples.sh`
+  redundant.
+- **CLI**: the startup log reported the host default budget and the gas limit as if
+  they were the layout budget; it now reports the effective layout budget after
+  `@context budget` is applied.
 
 ## [0.1.0] - 2025-12-09
 

FACET-v2.1.3-Production-Language-Specification.md

@@ -590,13 +590,23 @@ Let `B` be budget in FACET Units and `size[i] = facet_units(content[i])`.
   2. `shrink` descending
   3. original section order ascending
 
+For each flexible section, define the **effective minimum**:
+
+```
+effective_min[i] = min(min[i], size[i])
+```
+
+`min` is a floor on *retained* content. A declared `min` greater than the section's own
+size MUST NOT make the section unshrinkable or undroppable, and MUST NOT permit the
+packed layout to exceed `B`.
+
 Iterate `Flex` in that order while total size > B:
 
 - If `strategy` is set: apply strategy to `content[i]` (Pure Mode: Level‑0 only; else `F801`)
 - Recompute `size[i]` and total
-- If still over budget: truncate deterministically from the end down to satisfy budget but not below `min`
+- If still over budget: truncate deterministically from the end down to satisfy budget but not below `effective_min[i]`
   - truncation MUST NOT split UTF‑8 sequences
-- If still over budget and `size[i] == min`: drop the entire section (unless Critical)
+- If still over budget and `size[i] <= effective_min[i]`: drop the entire section (unless Critical)
 
 Result MUST be deterministic across implementations.
 
@@ -1333,6 +1343,10 @@ If an Execution Artifact is produced during `run` or `test`, it SHOULD be emitte
 
 ## 21. Change History
 
+### v2.1.3 (rev. 2026-08-21 — targeted normative clarification)
+
+- **§11.3** — Defined `effective_min[i] = min(min[i], size[i])` for flexible sections. Previously a `min` larger than the section's own content satisfied neither the truncation condition (`size[i] > min`) nor the drop condition (`size[i] == min`), so a conforming implementation could terminate with a packed layout exceeding `B` — contradicting the resource bound the model exists to provide.
+
 ### v2.1.3 (rev. 2026-02-19 — targeted normative clarifications)
 
 Normative additions within v2.1.3 to close formal gaps identified post-publication:

crates/fct-engine/Cargo.toml

@@ -6,6 +6,7 @@ edition = "2021"
 [dependencies]
 fct-ast = { path = "../fct-ast" }
 fct-std = { path = "../fct-std" }
+fct-validator = { path = "../fct-validator" }
 thiserror = "1.0"
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"

crates/fct-engine/src/box_model.rs

@@ -369,15 +369,20 @@ impl TokenBoxModel {
                 }
             }
 
+            // §11.3: `min` is a floor on *retained* content. A `min` larger than the
+            // section's own size must not make the section unshrinkable and must not
+            // let the layout exceed the budget, so clamp it to the size we actually have.
+            let effective_min = std::cmp::min(section.min, section.current_size);
+
             // If still over budget, truncate deterministically from the end down to `min`.
-            if running_total > self.budget && section.current_size > section.min {
+            if running_total > self.budget && section.current_size > effective_min {
                 let need = running_total - self.budget;
-                let reducible = section.current_size - section.min;
+                let reducible = section.current_size - effective_min;
                 let requested_reduction = std::cmp::min(need, reducible);
                 if requested_reduction > 0 {
                     let target_size = section.current_size - requested_reduction;
                     let (truncated_content, truncated_size) =
-                        self.truncate_content(&section.content, target_size, section.min);
+                        self.truncate_content(&section.content, target_size, effective_min);
                     if truncated_size < section.current_size {
                         section.content = truncated_content;
                         running_total = running_total
@@ -388,8 +393,8 @@ impl TokenBoxModel {
                 }
             }
 
-            // If still over budget and this section is at min, drop it.
-            if running_total > self.budget && section.current_size == section.min {
+            // If still over budget and this section is at its (clamped) min, drop it.
+            if running_total > self.budget && section.current_size <= effective_min {
                 running_total = running_total.saturating_sub(section.current_size);
                 allocated_sections.push(AllocatedSection {
                     final_size: 0,
@@ -567,6 +572,6 @@ impl TokenBoxModel {
 }
 
 fn sort_allocated_by_source(mut sections: Vec<AllocatedSection>) -> Vec<AllocatedSection> {
-    sections.sort_by(|a, b| a.section.source_index.cmp(&b.section.source_index));
+    sections.sort_by_key(|a| a.section.source_index);
     sections
 }

crates/fct-engine/src/r_dag.rs

@@ -502,7 +502,7 @@ impl RDagEngine {
             });
         };
 
-        if !Self::value_matches_runtime_type(&value, declared_type) {
+        if !Self::value_matches_runtime_type(&value, declared_type)? {
             return Err(EngineError::InputValidationFailed {
                 message: format!(
                     "Input '{}' does not satisfy declared type '{}'",
@@ -514,19 +514,24 @@ impl RDagEngine {
         Ok(value)
     }
 
-    fn value_matches_runtime_type(value: &ValueNode, declared_type: &str) -> bool {
-        match declared_type {
-            "any" => true,
-            "string" => matches!(value, ValueNode::String(_)),
-            "int" => matches!(value, ValueNode::Scalar(ScalarValue::Int(_))),
-            "float" => matches!(
-                value,
-                ValueNode::Scalar(ScalarValue::Float(_)) | ValueNode::Scalar(ScalarValue::Int(_))
-            ),
-            "bool" => matches!(value, ValueNode::Scalar(ScalarValue::Bool(_))),
-            "null" => matches!(value, ValueNode::Scalar(ScalarValue::Null)),
-            _ => false,
+    /// Validate a host-supplied runtime value against its declared FTS type.
+    ///
+    /// The full type system is used here (§8): composite types — `list<T>`,
+    /// `map<string, T>`, `struct { ... }`, unions and multimodal assets — are as
+    /// valid in `@input` as primitives are.
+    fn value_matches_runtime_type(value: &ValueNode, declared_type: &str) -> EngineResult<bool> {
+        // `int` is accepted where `float` is declared (widening), matching §8.4.
+        if declared_type.trim() == "float" {
+            if let ValueNode::Scalar(ScalarValue::Int(_)) = value {
+                return Ok(true);
+            }
         }
+
+        fct_validator::runtime_value_matches_type(value, declared_type).map_err(|e| {
+            EngineError::InputValidationFailed {
+                message: format!("invalid type expression '{}': {}", declared_type, e),
+            }
+        })
     }
 
     fn resolve_variable_ref(

crates/fct-parser/src/parser.rs

@@ -27,7 +27,9 @@ fn to_span(input: SpanInput) -> Span {
 }
 
 fn comment(input: SpanInput) -> ParseResult<SpanInput> {
-    recognize(pair(char('#'), is_not("\n\r")))(input)
+    // Spec Appendix B: COMMENT = "#" *(%x20-10FFFF) NL
+    // The comment body is zero-or-more characters, so a bare `#` line is valid.
+    recognize(pair(char('#'), opt(is_not("\n\r"))))(input)
 }
 
 fn eol(input: SpanInput) -> ParseResult<SpanInput> {
@@ -2037,14 +2039,10 @@ fn parse_assertion_from_string(
                             target: "output".to_string(),
                             text: parts[2..].join(" ").trim_matches('"').to_string(),
                         },
-                        "not" => {
-                            if parts.len() >= 4 && parts[2] == "contains" {
-                                fct_ast::AssertionKind::NotContains {
-                                    target: "output".to_string(),
-                                    text: parts[3..].join(" ").trim_matches('"').to_string(),
-                                }
-                            } else {
-                                return None;
+                        "not" if parts.len() >= 4 && parts[2] == "contains" => {
+                            fct_ast::AssertionKind::NotContains {
+                                target: "output".to_string(),
+                                text: parts[3..].join(" ").trim_matches('"').to_string(),
                             }
                         }
                         _ => return None,
@@ -2081,16 +2079,10 @@ fn parse_assertion_from_string(
                     return None;
                 }
             }
-            "sentiment" => {
-                if parts.len() >= 2 {
-                    fct_ast::AssertionKind::Sentiment {
-                        target: "output".to_string(),
-                        expected: parts[1].trim_matches('"').to_string(),
-                    }
-                } else {
-                    return None;
-                }
-            }
+            "sentiment" if parts.len() >= 2 => fct_ast::AssertionKind::Sentiment {
+                target: "output".to_string(),
+                expected: parts[1].trim_matches('"').to_string(),
+            },
             _ => return None,
         }
     };

crates/fct-std/src/lenses/utility.rs

@@ -45,36 +45,89 @@ impl Lens for DefaultLens {
 /// json(indent) - Format value as JSON
 pub struct JsonLens;
 
+/// Convert a FACET value into plain JSON.
+///
+/// Appendix A defines `json(indent: int = 0) -> string`, i.e. the JSON form of the
+/// *value*. Serializing `ValueNode` directly would leak the compiler's internal
+/// tagged representation (`{"kind":"Map","value":...}`) into rendered content.
+fn value_node_to_json(value: &ValueNode) -> LensResult<serde_json::Value> {
+    Ok(match value {
+        ValueNode::String(s) => serde_json::Value::String(s.clone()),
+        ValueNode::Scalar(ScalarValue::Null) => serde_json::Value::Null,
+        ValueNode::Scalar(ScalarValue::Bool(b)) => serde_json::Value::Bool(*b),
+        ValueNode::Scalar(ScalarValue::Int(i)) => serde_json::Value::Number((*i).into()),
+        ValueNode::Scalar(ScalarValue::Float(f)) => serde_json::Number::from_f64(*f)
+            .map(serde_json::Value::Number)
+            .ok_or_else(|| LensError::ExecutionError {
+                message: format!("json(): non-finite float cannot be serialized: {}", f),
+            })?,
+        ValueNode::List(items) => serde_json::Value::Array(
+            items
+                .iter()
+                .map(value_node_to_json)
+                .collect::<LensResult<Vec<_>>>()?,
+        ),
+        ValueNode::Map(map) => {
+            let mut obj = serde_json::Map::new();
+            for (key, val) in map {
+                obj.insert(key.clone(), value_node_to_json(val)?);
+            }
+            serde_json::Value::Object(obj)
+        }
+        other => {
+            return Err(LensError::ExecutionError {
+                message: format!(
+                    "json(): value is not fully evaluated and cannot be serialized: {:?}",
+                    other
+                ),
+            })
+        }
+    })
+}
+
+fn to_json_string(value: &serde_json::Value, indent: usize) -> LensResult<String> {
+    if indent == 0 {
+        return serde_json::to_string(value).map_err(|e| LensError::ExecutionError {
+            message: format!("JSON serialization failed: {}", e),
+        });
+    }
+
+    let spaces = " ".repeat(indent);
+    let formatter = serde_json::ser::PrettyFormatter::with_indent(spaces.as_bytes());
+    let mut buf = Vec::new();
+    let mut ser = serde_json::Serializer::with_formatter(&mut buf, formatter);
+    serde::Serialize::serialize(value, &mut ser).map_err(|e| LensError::ExecutionError {
+        message: format!("JSON serialization failed: {}", e),
+    })?;
+    String::from_utf8(buf).map_err(|e| LensError::ExecutionError {
+        message: format!("JSON serialization produced invalid UTF-8: {}", e),
+    })
+}
+
 impl Lens for JsonLens {
     fn execute(
         &self,
         input: ValueNode,
         args: Vec<ValueNode>,
-        _kwargs: HashMap<String, ValueNode>,
+        kwargs: HashMap<String, ValueNode>,
         _ctx: &LensContext,
     ) -> LensResult<ValueNode> {
-        // Get indent size (default None for compact)
-        let indent = if let Some(ValueNode::Scalar(ScalarValue::Int(n))) = args.first() {
-            Some(*n as usize)
-        } else {
-            None
-        };
-
-        let json_str = if let Some(indent_size) = indent {
-            serde_json::to_string_pretty(&input)
-                .map_err(|e| LensError::ExecutionError {
-                    message: format!("JSON serialization failed: {}", e),
-                })?
-                .lines()
-                .map(|line| " ".repeat(indent_size) + line)
-                .collect::<Vec<_>>()
-                .join("\n")
-        } else {
-            serde_json::to_string(&input).map_err(|e| LensError::ExecutionError {
-                message: format!("JSON serialization failed: {}", e),
-            })?
+        // `indent` may be positional or named; default 0 means compact output.
+        let indent_arg = kwargs.get("indent").or_else(|| args.first());
+        let indent = match indent_arg {
+            None => 0,
+            Some(ValueNode::Scalar(ScalarValue::Int(n))) if *n >= 0 => *n as usize,
+            Some(other) => {
+                return Err(LensError::ArgumentError {
+                    message: format!(
+                        "json(indent=...) expects a non-negative int, got {:?}",
+                        other
+                    ),
+                })
+            }
         };
 
+        let json_str = to_json_string(&value_node_to_json(&input)?, indent)?;
         Ok(ValueNode::String(json_str))
     }