Skip to content

Add Ruby language support - #55

Open
nsmmrs wants to merge 1 commit into
trailofbits:mainfrom
nsmmrs:ruby-support
Open

Add Ruby language support#55
nsmmrs wants to merge 1 commit into
trailofbits:mainfrom
nsmmrs:ruby-support

Conversation

@nsmmrs

@nsmmrs nsmmrs commented Aug 7, 2026

Copy link
Copy Markdown

Summary

Adds Ruby support. Introduces a RubyLanguageEngine that parses and mutates Ruby code using the tree-sitter-ruby grammar.

Mutations

We mapped Mewt's standard mutation operators to Ruby AST nodes and created new operators to support Ruby-specific syntax and control flow.

1. Ruby-Specific Operators (New)

  • UF / UT (Unless False / Unless True): Replaces unless conditions and unless_guard nodes with false or true.
  • ULF / ULT (Until False / Until True): Replaces until loop conditions with false or true.
  • UP (Unpin Pattern): Removes the ^ pin operator in pattern matching expressions (in ^var becomes in var).
  • RBR (Range Bound Replacement): Swaps inclusive (..) and exclusive (...) range operators.
  • SNR (Safe Navigation Removal): Removes the safe navigation operator (&. becomes .).
  • LAOS (Logical Assignment Operator Swap): Swaps the ||= and &&= operators.
  • RMOS (Regex Match Operator Swap): Swaps regex match operators (=~ and !~).
  • CES (Case Equality Swap): Swaps the case equality operator (===) with the standard equality operator (==).
  • EL (Empty Literals): Replaces populated strings, arrays, and hashes with empty equivalents ("", [], {}).

2. Core Operators (Mapped to Ruby)

  • IF / IT (If False / If True): Targets if, elsif, if_guard, and ternary conditional expressions.
  • CR / ER (Constant Replacement / Error Replacement): Replaces statements with nil or raise "mewt". Applies to top-level expressions, method calls, yield, super, redo, retry, break, next, rescue_modifier (inline rescue), and case_match (in clauses).
  • NR (Negation Removal): Removes the ! operator and the not keyword (e.g., not valid?).
  • AS (Argument Swap): Swaps arguments in method calls and named elements in array patterns (e.g., in [a, b, c]).
  • LC (Loop Control): Swaps redo and retry keywords, and replaces break and next with redo or retry.
  • (And all standard operators like arithmetic swaps, boolean replacements, and comparison swaps)

Testing

  • Unit Tests: Added unit tests for each mutation operator in tests/ruby/mutations/ to verify mutant generation across different syntax scenarios.
  • Stress Tests: Tested against open-source Ruby codebases (Discourse, Jekyll, and RuboCop). Testing against the Discourse codebase processed ~5,700 target files and generated ~464,000 mutants in under two minutes on a release build. No parser errors or infinite loops occurred.
  • CI Checks: Code passes just pre-commit formatting and linter checks (cargo fmt, clippy, and typos).

@nsmmrs
nsmmrs requested a review from bohendo as a code owner August 7, 2026 15:30
@CLAassistant

CLAassistant commented Aug 7, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

severity: MutationSeverity::Medium,
},
Mutation {
slug: "RBR",

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We added a custom RBR (Range Bound Replacement) slug to swap Ruby's inclusive (..) and exclusive (...) range operators.

@nsmmrs
nsmmrs marked this pull request as draft August 7, 2026 16:12
@nsmmrs
nsmmrs force-pushed the ruby-support branch 3 times, most recently from 9760eb8 to 40b0887 Compare August 7, 2026 17:18
/// (method/do_block/block) are intentionally excluded so that the
/// outermost-match logic in `patterns` keeps producing one mutant per
/// inner statement rather than a single mutant for the whole body.
const STATEMENT_KINDS: &[&str] = &[

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We include nodes::CASE and nodes::CASE_MATCH but exclude top-level containers like method and class bodies. This creates isolated mutations for innermost statements rather than replacing the entire container.

}
});
}
"UF" => {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We added custom mutation slugs (UF/UT and ULF/ULT) for Ruby's unless and until statements. Standard IF/IT logic creates the opposite behavior because replacing an unless condition with false differs semantically from replacing an if condition with false.

);
}
}
"UP" => {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We created the UP (Unpin Pattern) slug to handle unpinning variables in pattern matching expressions. Removing the ^ pin operator (in ^var) changes the match from an exact-value match to a variable reassignment. We use a dedicated slug for this instead of reusing the NR (Negation Removal) slug, as the caret does not function as a logical negation in this context.

mutants
}

pub fn swap_named_children(root: Node, source: &str, node_kinds: &[&str]) -> Vec<PartialMutant> {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We added this function to support mutating Ruby array patterns (e.g., in [a, b, c]). Tree-sitter treats array elements as standalone named children rather than wrapping them in an arguments list. This prevents the existing swap_args function from traversing them.

Comment thread src/core/cmds/results.rs
info!(
" {:<9} | {}",
&outcome.status.display(),
outcome.status.display(),

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the unnecessary reference (&) here since .display() already returns a type that implements Display. This was done to resolve a clippy::useless_borrow warning.

Comment thread src/core/cmds/results.rs
) -> AppResult<()> {
// If mutant_id is provided, special handling
if filters.id.is_some() {
if let Some(id) = filters.id {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored the if is_some() check that was followed by an unwrap() into an if let Some(id) binding to satisfy clippy::unnecessary_unwrap.

Comment thread .typos.toml

# "varaint" is a typo in the upstream tree-sitter-ruby grammar's comment
# ("command varaint") in grammar.js. We do not control this vendored file.
varaint = "varaint"

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added varaint to the typos ignore list because this typo exists in a comment within the vendored tree-sitter-ruby grammar file, which we do not maintain.

@nsmmrs
nsmmrs force-pushed the ruby-support branch 2 times, most recently from fe669e5 to f3fe683 Compare August 7, 2026 17:55
@nsmmrs
nsmmrs marked this pull request as ready for review August 7, 2026 18:13

@nsmmrs nsmmrs left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed it said I have an unfinished review, so I think my comments have been invisible this whole time.

@nsmmrs

nsmmrs commented Sep 6, 2026

Copy link
Copy Markdown
Author

Not sure why, but I'm suddenly getting a bunch of notifications about failed CI runs on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants