Skip to content

add function replace_nodes to Tree#1524

Open
zzc0430 wants to merge 10 commits into
lark-parser:masterfrom
zzc0430:feat/add_replace_node_func
Open

add function replace_nodes to Tree#1524
zzc0430 wants to merge 10 commits into
lark-parser:masterfrom
zzc0430:feat/add_replace_node_func

Conversation

@zzc0430

@zzc0430 zzc0430 commented Apr 7, 2025

Copy link
Copy Markdown

This can be used to replace tokens in a Tree that satisfy a specified rule.

This can be used to replace tokens in a `Tree` that satisfy a specified rule.
@erezsh

erezsh commented Apr 7, 2025

Copy link
Copy Markdown
Member

Add tests.

Comment thread lark/tree.py
@zzc0430

zzc0430 commented Apr 8, 2025

Copy link
Copy Markdown
Author

Add tests.

added 388fd80

@erezsh

erezsh commented Apr 8, 2025

Copy link
Copy Markdown
Member

Let's call it replace_tokens.

Also in the tests, you are changing tree2 in-place, which could break other tests if run in a different order.

@zzc0430

zzc0430 commented Apr 8, 2025

Copy link
Copy Markdown
Author

Let's call it replace_tokens.

Also in the tests, you are changing tree2 in-place, which could break other tests if run in a different order.

done 545b808, caea780

@erezsh

erezsh commented Apr 8, 2025

Copy link
Copy Markdown
Member

*deepcopy

@zzc0430

zzc0430 commented Apr 8, 2025

Copy link
Copy Markdown
Author

*deepcopy

5ab7313

@erezsh

erezsh commented Apr 8, 2025

Copy link
Copy Markdown
Member

@MegaIng Do we really need the special behavior for None? The callback can just return the parameter as is, if no changes are necessary. (I don't think it matters in terms of performance)

@MegaIng

MegaIng commented Apr 8, 2025

Copy link
Copy Markdown
Member

True, probably not needed.

To get the type checking to work:

  • replace Token in the signature with the Leaf generic parameter.
  • instead of isinstance(..., Token) just use an else statement.

This slightly changes the semantics, but I think it's more useful.

@zzc0430

zzc0430 commented Apr 9, 2025

Copy link
Copy Markdown
Author

True, probably not needed.

To get the type checking to work:

  • replace Token in the signature with the Leaf generic parameter.
  • instead of isinstance(..., Token) just use an else statement.

This slightly changes the semantics, but I think it's more useful.

c21139c

@erezsh

erezsh commented May 9, 2026

Copy link
Copy Markdown
Member

Tbh I'm not convinced this is a useful method. I can't imagine many situations where it's desirable.

But here's how it should look, if it was ever accepted: (more or less)

    def replace_values(self, replacer: 'Callable[[_Leaf_T], _Leaf_T]') -> None:
        """Replace all leaf values in the tree using the result of ``replacer(value)``.

        The callback receives each leaf and should return its replacement,
        or the same value unchanged if no replacement is desired.

        Example:
            >>> tree.replace_values(lambda v: v.update(value="y") if v == "x" else v)
        """
        for index, child in enumerate(self.children):
            if isinstance(child, Tree):
                child.replace_values(replacer)
            else:
                result = replacer(child)
                if result is not child:
                    self.children[index] = result

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants