Skip to content

fix(jsonschema): emit unicode-safe patterns from concerto regexes - #301

Open
rishabhknowss wants to merge 2 commits into
accordproject:mainfrom
rishabhknowss:fix/jsonschema-unicode-pattern
Open

rishabhknowss wants to merge 2 commits into
accordproject:mainfrom
rishabhknowss:fix/jsonschema-unicode-pattern

Conversation

@rishabhknowss

Copy link
Copy Markdown

Closes #300

JSONSchemaVisitor copied a concerto regex's source straight into pattern. concerto regexes have no u flag, but ajv compiles pattern in unicode mode, so identity escapes like \< (used in agreement.cto) made the generated schema fail with Invalid escape.

Changes

  • add toUnicodeSafePattern(): if the source isn't valid with the u flag, drop backslashes before characters that don't need escaping (they match the same literal either way). \- is kept inside character classes. valid sources are returned unchanged.
  • use it for field/scalar pattern output.
  • verification test now compiles every definition through a $ref, since ajv skips definitions nothing references (this is why agreement passed before). it fails on main with Invalid regular expression: /^[^\<\>]*$/u: Invalid escape and passes with this fix.
  • unit test covering \<\>, \- outside a class, and an already-safe pattern.

Flags

Related Issues

Author Checklist

  • Ensure you provide a DCO sign-off for your commits using the --signoff option of git commit.
  • Vital features and changes captured in unit and/or integration tests
  • Commits messages follow AP format
  • Extend the documentation, if necessary
  • Merging to main from fork:branchname

🤖 Generated with Claude Code

Concerto regexes have no `u` flag, so their source may contain identity
escapes such as `\<` that are invalid in unicode mode, which JSON Schema
validators such as ajv use to compile `pattern`. Drop those redundant
escapes when the source is not valid in unicode mode.

Also compile every definition in the JSON Schema verification test, since
ajv only compiles definitions that are referenced.

Closes accordproject#300

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: rishabhknowss <rishabhrai1515@gmail.com>
@mira-accord-project

Copy link
Copy Markdown

Welcome, and thanks for your first PR to concerto-codegen, and for the clear write-up in #300. A maintainer will review it, and their feedback will appear here. CI has been approved to run for this PR.

Drafted by an Accord Project agent, reviewed by a maintainer.

@mira-accord-project mira-accord-project Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this fix, and welcome to the project. Rewriting identity escapes such as \< so the emitted pattern compiles in unicode mode is the right fix, and the logic reads correctly. Falling back to the original source when the rewrite is still invalid is a safe choice. Strengthening the verification test to compile every definition through a $ref is a good change, because it checks definitions that were never compiled before. I found nothing blocking. One small suggestion on test coverage is below. Once that is considered and a maintainer has approved, this looks ready to merge.

Nothing blocking; some suggestions below. Findings: 1 nit.


Automated review by Mira, an Accord Project agent, at ae74e3c. Findings are labelled blocking, should-fix or nit; a maintainer makes the final call.

expect(schema.properties.myString.pattern).equal('abc.*');
});

it('should generate unicode-safe patterns from regexes with identity escapes', () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit · tests

The new test doesn't reach two branches of toUnicodeSafePattern. The alreadySafe regex is valid in unicode mode, so it returns early and the in-class \- handling (inClass && next === '-') never runs. The fallback that returns the original source when the rewritten regex is still invalid has no test either. Consider adding a case that needs rewriting and also has an escaped dash inside a class (for example /^[\w\-<\>]+$/), and a case that stays invalid after rewriting (for example a pattern containing \a), asserting that the source is returned unchanged. That would protect the edge handling against future changes.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added both in a978654: ^[\w\-<\>]+$ now checks the escaped dash survives the rewrite, and ^\a<\>$ checks the fallback returns the source unchanged. confirmed each one fails if the matching branch is removed. npm test passes (692).

…back

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: rishabhknowss <rishabhrai1515@gmail.com>

@mira-accord-project mira-accord-project Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this fix, and for adding the two extra tests so quickly. I read toUnicodeSafePattern closely: it returns already-valid sources unchanged, and otherwise drops the backslash only from identity escapes of non-alphanumeric characters that are not regex syntax. Those escapes match the same literal character with or without the backslash, so the meaning of the pattern is preserved. It keeps \- inside character classes, where unicode mode requires it, and falls back to the original source if the rewritten pattern is still invalid. The new tests cover each of those branches, and the verification test now compiling every definition through a $ref is a good way to stop this regression coming back. I have no changes to request. The test and verification workflows have not run on this PR yet, so please look out for them once a maintainer approves the runs, and check they pass, since the verification change widens what ajv compiles. After that, the PR needs a maintainer's approval before it can merge.

No problems found.


Automated review by Mira, an Accord Project agent, at a978654. Findings are labelled blocking, should-fix or nit; a maintainer makes the final call.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

json schema pattern from concerto regex fails in ajv (unicode mode)

1 participant