Skip to content

[BUG] banned_strings issue with some strings containing ASCII period #412

@mfiano

Description

@mfiano

OS

Linux

GPU Library

CUDA 12.x

Python version

3.12

Describe the bug

banned_strings=[] OK
banned_strings=["."] OK
banned_strings=[".."] OK
banned_strings=["..."] breaks
banned_strings=[".foo."] breaks
banned_strings=["---"] OK

It seems that any literal string of at least 3 characters which includes a . at in position 0 corrupts the model's output (usually just emitting the stop token, but sometimes gibberish or an incomplete message).

Reproduction steps

Pass banned_strings="..." for a simple reproduction.

I tested with EXL3 and Mistral-Small 24B.

Expected behavior

Expected behavior would be for literal strings to match verbatim instead of getting confused with regex or backtracking.

Logs

No response

Additional context

No response

Acknowledgements

  • I have looked for similar issues before submitting this one.
  • I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
  • I understand that the developers have lives and my issue will be answered when possible.
  • I understand the developers of this program are human, and I will ask my questions politely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions