Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/developer/architecture/adr/03_duck_typing.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ expected by the consumers.
- Provide adapter/wrapper helpers (see ADR 06) to normalize foreign node-like objects into the project's
canonical node shape.

The current implementation exposes `NodeProtocol` as the canonical structural contract. Nodes also expose
`parser_kind` and `semantic_kind`; parser-specific mappings remain inside their integration. `PatternKind` is
separate from node classification and represents matcher behavior such as one-node and all-node placeholders.

```python
@runtime_checkable
class NodeMatchProtocol(protocol):
Expand Down
2 changes: 1 addition & 1 deletion docs/developer/architecture/adr/08_pytest_suite.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ Negative:
## Related decisions

- See ADR 09 (Property-based tests) for the use of Hypothesis alongside pytest.
- See ADR 10 (Type hierarchy) for the `SyntacticKind` taxonomy referenced in find-functionality tests.
- See ADR 03 (Duck typing for nodes) and the shared `SemanticKind` vocabulary for kind-based matching.
- See ADR 12 (Patterns are not nodes) for the `Pattern` type used in matching tests.

---
Expand Down
18 changes: 17 additions & 1 deletion docs/developer/architecture/adr/10_type_hierarchy.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 10 - Type Hierarchy

Status: Accepted
Status: Superseded

Date: 2026-03-27

Expand All @@ -25,6 +25,10 @@ Authors:

## Context

> **Superseded by the protocol-based node model.** This ADR records the historical
> class-hierarchy approach. The current architecture uses structural node protocols,
> shared `SemanticKind` values, and parser-local kind maps instead.

the goal of this ADR is to establish a robust and maintainable type hierarchy for AST nodes use in the algorithms
within the Renaissance project and across the languages.

Expand All @@ -35,6 +39,18 @@ own lookup tables. A class hierarchy provides a more robust and idiomatic soluti

## Decision

The class-hierarchy decision in this ADR is no longer the target architecture. It is
retained as historical context for the compatibility code being removed incrementally.

The current decision is documented in [ADR 03](03_duck_typing.md) and uses:

- `NodeProtocol` for the structural node contract.
- `parser_kind` for the exact parser-provided kind.
- `semantic_kind` for shared cross-language concepts.
- parser-local maps and predicates for language-specific concepts.

The remainder of this section describes the superseded approach.

- Follow the Doxygen definition for common node types (e.g., statement, expression, declaration) and use native Python
- types for language-specific or non-standard node kinds.
- Use the class hierarchy to determine the type of a node instead of string-based type name comparisons.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Negative:

- See ADR 01 (Children and properties) for the AST node structure that `Pattern.node` wraps.
- See ADR 03 (Duck typing) for the protocol-based approach used by the matcher to accept `Pattern` objects.
- See ADR 10 (Type hierarchy) for the `SyntacticKind` taxonomy used as the `kind` field.
- See ADR 03 (Duck typing for nodes) for the protocol-based node shape and shared semantic kinds.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/developer/architecture/adr/13_match_pattern.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ Negative:

- See ADR 12 (Patterns are not nodes) for the `Pattern` / `SyntacticKind` design used by the pattern
factory.
- See ADR 10 (Type hierarchy) for the node kind taxonomy referenced by find-by-kind functionality.
- See ADR 03 (Duck typing for nodes) for the protocol-based node shape and shared semantic kinds.
- See ADR 08 (Test architecture) for the test requirements that cover matching, placeholders, and
equivalent-code matching.
- See ADR 11 (Parser with space and comment) for the lossless round-trip required by transformation tests.
Expand Down
2 changes: 1 addition & 1 deletion docs/developer/architecture/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The goal of ADR is to give the developer of new language AST for Renaissance a g
| [07](07_package_management.md) | Use UV for package & environment management | Proposal |
| [08](08_pytest_suite.md) | Test Architecture | Accepted |
| [09](09_property_based_tests.md) | Property-Based Tests | Proposal |
| [10](10_type_hierarchy.md) | Type Hierarchy | Proposal |
| [10](10_type_hierarchy.md) | Type Hierarchy | Superseded |
| [11](11_parser_with_space_and_comment.md) | Parser with Space and Comment | Proposal |
| [12](12_patterns_as_not_nodes.md) | Patterns Are Not Nodes | Proposal |
| [13](13_match_pattern.md) | Match Pattern | Proposal |
Expand Down
5 changes: 5 additions & 0 deletions docs/developer/architecture/code-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@
with basic functionality and a back door:
`get_original_node` to obtain the AST node as provided by the parser.

1. Parser integrations expose both an exact `parser_kind` and a shared `semantic_kind`.
Multiple parser kinds may map to one shared semantic kind, such as Clang's `FunctionDecl` and
`CXXMethodDecl` both mapping to a function. Parser-specific concepts remain available through
`parser_kind` and integration-local predicates; they are not forced into the shared vocabulary.

1. AST Nodes are read only and immutable.

1. AST Nodes are navigable, so parent must be present (except for the ATU / top node) and
Expand Down
6 changes: 3 additions & 3 deletions src/rejuvenation/batch_process_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

from renaissance.integrations.clang import ClangASTNode
from renaissance.integrations.clang.clang_json_ast_node import ClangJsonASTNode
from renaissance.integrations.types import Call
from renaissance.recipes import CleanupRefactoring
from renaissance.syntax_tree import (
ASTFactory,
Expand All @@ -20,6 +19,7 @@
final_action,
recipe_step,
)
from renaissance.syntax_tree.semantic_kind import SemanticKind

example_1 = textwrap.dedent("""
void x(int a) {}
Expand Down Expand Up @@ -109,7 +109,7 @@ def batch_repeat_example():

# remove a function to create more unused variables
def remove_function(ast_processor: ASTProcessor):
[ast_processor.insert_before("// ", node, False, False) for node in ast_processor.find_ast_type(Call)]
[ast_processor.insert_before("// ", node, False, False) for node in ast_processor.find_semantic_kind(SemanticKind.CALL)]

# batch_processor.repeat(simple_codebase_provider, [remove_function])
batch_processor.repeat(
Expand All @@ -134,7 +134,7 @@ def __init__(self):
def store_function_call(self, ast_processor: ASTProcessor) -> Callable[[], None] | None:
# find all function calls and store them, this routing is invoked in parallel!
calls: list[CallInfo] = []
[AnalysisRecipe._add_function_call(node, calls) for node in ast_processor.find_ast_type(Call)]
[AnalysisRecipe._add_function_call(node, calls) for node in ast_processor.find_semantic_kind(SemanticKind.CALL)]
# the resulting lambda is invoked single threaded
# this kind of mechanism is mainly used to store results from multiple processors
# for refactoring operations this is not needed as a refactoring operation is single threaded
Expand Down
6 changes: 3 additions & 3 deletions src/rejuvenation/python_ast_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@

from rejuvenation.python_lst_example import python_lst_smoke_test
from renaissance.integrations.python.ast.factory import PythonFactory, PythonPatternFactory
from renaissance.integrations.types import Call
from renaissance.syntax_tree import ASTRewriter, ASTShower
from renaissance.syntax_tree.ast_finder import find_ast_type
from renaissance.syntax_tree.ast_finder import find_semantic_kind
from renaissance.syntax_tree.match_finder import match_pattern
from renaissance.syntax_tree.semantic_kind import SemanticKind

example_code = """
from module import foo, bar, baz, quux
Expand Down Expand Up @@ -43,7 +43,7 @@ def python_ast_smoke_test():
ASTShower.show_node(atu)

print("_______________simple find____________________________________")
nodes = find_ast_type(atu, Call)
nodes = find_semantic_kind(atu, SemanticKind.CALL)

ASTShower.show_node(nodes[0])

Expand Down
6 changes: 3 additions & 3 deletions src/rejuvenation/python_cst_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
from rejuvenation.python_lst_example import python_lst_smoke_test
from renaissance.integrations.python.ast.cst_node import PythonCstNode
from renaissance.integrations.python.ast.factory import PythonFactory, PythonPatternFactory
from renaissance.integrations.types import Call
from renaissance.syntax_tree import ASTRewriter, ASTShower
from renaissance.syntax_tree.ast_finder import find_ast_type
from renaissance.syntax_tree.ast_finder import find_semantic_kind
from renaissance.syntax_tree.match_finder import match_pattern
from renaissance.syntax_tree.semantic_kind import SemanticKind

example_code = """
from module import foo, bar, baz, quux
Expand Down Expand Up @@ -43,7 +43,7 @@ def python_cst_smoke_test():
ASTShower.show_node(atu)

print("_______________simple find____________________________________")
nodes = find_ast_type(atu, Call)
nodes = find_semantic_kind(atu, SemanticKind.CALL)

ASTShower.show_node(nodes[0])

Expand Down
6 changes: 3 additions & 3 deletions src/rejuvenation/python_lst_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

from renaissance.integrations.python.ast.factory import PythonFactory, PythonPatternFactory
from renaissance.integrations.tree_sitter.lst import LSTNode
from renaissance.integrations.types import Call
from renaissance.syntax_tree import ASTRewriter, ASTShower
from renaissance.syntax_tree.ast_finder import find_ast_type
from renaissance.syntax_tree.ast_finder import find_semantic_kind
from renaissance.syntax_tree.match_finder import match_pattern
from renaissance.syntax_tree.semantic_kind import SemanticKind

example_code = """
from module import foo, bar, baz, quux
Expand Down Expand Up @@ -42,7 +42,7 @@ def python_lst_smoke_test():
ASTShower.show_node(atu)

print("_______________simple find____________________________________")
nodes = find_ast_type(atu, Call)
nodes = find_semantic_kind(atu, SemanticKind.CALL)

ASTShower.show_node(nodes[0])

Expand Down
6 changes: 3 additions & 3 deletions src/rejuvenation/python_rst_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@

from renaissance.integrations.python.ast.factory import PythonFactory, PythonPatternFactory
from renaissance.integrations.python.ast.rst_node import PythonRstNode
from renaissance.integrations.types import Call
from renaissance.syntax_tree import ASTRewriter, ASTShower
from renaissance.syntax_tree.ast_finder import find_ast_type
from renaissance.syntax_tree.ast_finder import find_semantic_kind
from renaissance.syntax_tree.match_finder import match_pattern
from renaissance.syntax_tree.semantic_kind import SemanticKind

example_code = """
from module import foo, bar, baz, quux
Expand Down Expand Up @@ -41,7 +41,7 @@ def python_rst_smoke_test():
ASTShower.show_node(atu)

print("_______________simple find____________________________________")
nodes = find_ast_type(atu, Call)
nodes = find_semantic_kind(atu, SemanticKind.CALL)

ASTShower.show_node(nodes[0])

Expand Down
6 changes: 3 additions & 3 deletions src/rejuvenation/refactor_with_nested_compositions.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
import textwrap

from renaissance.integrations.clang import ClangASTNode, CPatternFactory
from renaissance.integrations.types import Call
from renaissance.syntax_tree import ASTFactory, ASTRewriter, ASTShower
from renaissance.syntax_tree.ast_finder import find_ast_type
from renaissance.syntax_tree.ast_finder import find_semantic_kind
from renaissance.syntax_tree.match_finder import find_all
from renaissance.syntax_tree.semantic_kind import SemanticKind

example_code = """
void f1(int a, int b, int c);
Expand Down Expand Up @@ -82,7 +82,7 @@ def refactor_with_nested_compositions(args):
ASTShower.show_node(pattern1[0], include_properties=True)

# we only want to search the call expression as a pattern so it's searched using the kind
pattern2 = find_ast_type(pattern2, Call)
pattern2 = find_semantic_kind(pattern2, SemanticKind.CALL)

# the replacement code strip indent is used to be agnostic to the indentation of the replacement
pattern1replacement = textwrap.dedent("""
Expand Down
4 changes: 2 additions & 2 deletions src/rejuvenation/walk_compilation_database.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
import targets
from renaissance.integrations.clang import ClangASTNode, CompilationDatabase
from renaissance.integrations.clang.clang_json_ast_node import ClangJsonASTNode
from renaissance.integrations.types import FunctionDef
from renaissance.syntax_tree import ASTProcessor, ASTShower
from renaissance.syntax_tree.semantic_kind import SemanticKind


def main(args):
Expand All @@ -21,7 +21,7 @@ def main(args):
ASTShower.show_node(atu, include_properties=True)
# do something with the factory and atu
ast_refactor = ASTProcessor(atu, factory, in_memory=True)
[print(n.text) for n in ast_refactor.find_ast_type(FunctionDef)]
[print(n.text) for n in ast_refactor.find_semantic_kind(SemanticKind.FUNCTION)]


if __name__ == "__main__":
Expand Down
12 changes: 9 additions & 3 deletions src/renaissance/integrations/clang/c_pattern_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,19 @@
VariableDef,
)
from renaissance.syntax_tree.ast_factory import ASTFactory
from renaissance.syntax_tree.ast_finder import find_ast_type
from renaissance.syntax_tree.ast_finder import find_ast_type, find_nodes
from renaissance.syntax_tree.ast_node import ASTNode
from renaissance.syntax_tree.ast_shower import ASTShower

SHOW_NODE = False


def _matches_kind(node, kind) -> bool:
if isinstance(kind, type):
return isinstance(node.ast_type(), kind)
return kind(node)


def derive_header_text(language: str, ref_node: ASTNode | None):
# collect includes #defines and var decl from the refNode
header = "\n"
Expand Down Expand Up @@ -167,7 +173,7 @@ def create(self, text: str, kind: type[Type] = None) -> ASTNode:
# print(self.header + text)
root = self.factory.create_from_text(self.header + text, "test." + self.language)
if kind:
return first(find_ast_type(root.children[-1], kind))
return first(find_nodes(root.children[-1], lambda node: _matches_kind(node, kind)))
return root

def create_statement(
Expand Down Expand Up @@ -207,7 +213,7 @@ def _create_body(
# node of the specified kind

body = first(find_ast_type(root.children[-1], CompoundStatement)).children
return list(n for n in body if n.is_part_of_translation_unit and first(find_ast_type(n, kind)))
return list(n for n in body if n.is_part_of_translation_unit and first(find_nodes(n, lambda node: _matches_kind(node, kind))))

def _create(self, text: str) -> ASTNode:
atu = self.factory.create_from_text(text, "test." + self.language)
Expand Down
15 changes: 9 additions & 6 deletions src/renaissance/integrations/clang/clang_ast_node.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from clang.cindex import Config, CursorKind, Index, TypeKind
from clang.cindex import TranslationUnit as ClangCindexTranslationUnit

from renaissance.integrations.clang.cpp_utils import matches_kind
from renaissance.integrations.clang.cpp_utils import matches_node_kind
from renaissance.integrations.clang.kinds import CLANG_KIND_MAP
from renaissance.integrations.types import (
KIND_MAP,
Expand All @@ -19,14 +19,13 @@
Definition,
Literal,
MacroDef,
MatchAll,
MatchOne,
Statement,
TranslationUnit,
UnaryOperation,
UnknownType,
)
from renaissance.syntax_tree import ASTFinder, ASTNode, ASTReference
from renaissance.syntax_tree.pattern_kind import PatternKind
from renaissance.syntax_tree.semantic_kind import SemanticKind
from renaissance.utils.ast_utils import match_children, match_props

Expand Down Expand Up @@ -131,6 +130,10 @@ def __init__(
self._kind = insert_kind if insert_kind is not None else self.__derive_kind()
self.parser_kind = self._kind
self.semantic_kind = CLANG_KIND_MAP.get(self.parser_kind, SemanticKind.NODE)
self.pattern_kind = {
"MatchOne": PatternKind.MATCH_ONE,
"MatchAll": PatternKind.MATCH_ALL,
}.get(self.parser_kind)
self.ast_type = KIND_MAP.get(self._kind, UnknownType)
self.indent = ""
# TODO: TextUtils.get_indent(self.content, self._offset)
Expand Down Expand Up @@ -290,7 +293,7 @@ def _is_statement_or_declaration(self):

@override
def matches_kind(self, node: ASTNode) -> bool:
return matches_kind(self.ast_type, node.ast_type)
return matches_node_kind(self, node)

def _derive_properties(self) -> dict[str, int | str]:
result = {}
Expand Down Expand Up @@ -443,9 +446,9 @@ def __derive_kind(self) -> str:
return str(self.node.kind.name)
if self.node.kind.name in ["UNEXPOSED_EXPR", "VAR_DECL", "DECL_REF_EXPR"]:
if self.node.displayname.startswith("$$") and " " not in self.node.displayname:
return MatchAll.__name__
return "MatchAll"
if self.node.displayname.startswith("$") and " " not in self.node.displayname:
return MatchOne.__name__
return "MatchOne"
return str(self.node.kind.name)
except Exception:
return EMPTY_STR
Expand Down
Loading