Skip to content

[trade-off] 泄密检测使用子串匹配,存在假阳性/假阴性 #27

Closed

Description

@ULookup

现状

SecretStore::check_leak_risk() 用 draft_text.find(secret.truth) 做检测,且 truth.size() < 3 时跳过。

假阳性示例:truth="他是好人" → 草稿中"大家都说他是好人"(角色引用谣言)也会触发。

假阴性示例:truth="艾琳是骑士团教官" → 草稿中"艾琳训练过王都骑士"(换说法)检测不到。

当前取舍

  • 设计明确要求不依赖 LLM
  • 精确语义检测需要 NLP 模型,不在范围内
  • size() >= 3 阈值防止单字词海量误报
  • 当前定位是标记可疑片段请作者人工确认(SceneWrapUp.leak_risks),而非自动拦截

升级触发条件

接入本地小模型做语义相似度判断,或使用 embedding 向量比较 truth 和 draft text 的语义距离。

Activity

  1. self-assigned this
    on Jun 3, 2026
  2. added
    trade-offIntentional design trade-off, deferred optimization
    worldbuildingWorldbuilding novel agent feature
    on Jun 3, 2026
  3. added 2 commits that reference this issue on Jun 21, 2026
    33ce541
    66e555b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

trade-offIntentional design trade-off, deferred optimizationworldbuildingWorldbuilding novel agent feature

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions