Skip to content

feat: Upgrade Kiwi to v0.23.2 with new API bindings - #43

Merged
mrchypark merged 4 commits into
codingpot:mainfrom
mrchypark:feat/kiwi-v0.23.2-upgrade
Aug 11, 2026
Merged

feat: Upgrade Kiwi to v0.23.2 with new API bindings#43
mrchypark merged 4 commits into
codingpot:mainfrom
mrchypark:feat/kiwi-v0.23.2-upgrade

Conversation

@mrchypark

@mrchypark mrchypark commented Aug 6, 2026

Copy link
Copy Markdown
Member

Changes

Version Upgrade

  • Kiwi C library: v0.23.0 → v0.23.2

New Features

MatchOption Flags (26 new)

  • OOV detection modes: OOV_RULE_ONLY, OOV_CHR_MODEL, OOV_CHR_FREQ_MODEL, OOV_CHR_FREQ_BRANCH_MODEL
  • Pattern matching: SERIAL, EMOJI
  • Join/split options: JOIN_NOUN_PREFIX, SPLIT_COMPLEX, Z_CODA, etc.

BuildOption Flags

  • Model types: CONG, CONG_GLOBAL
  • Dictionary loading: TYPO_DICT, MULTI_DICT

Config API

  • GetGlobalConfig / SetGlobalConfig - OOV detection settings, thresholds, etc.
  • GetOptionF / SetOptionF - float option access
  • GetOption / SetOption - int option access

Morphset (Blocklist)

  • NewMorphset - create morpheme set for blocking specific morphemes from analysis
  • Use with WithBlocklist in Analyze options

AnalyzeOptions Extensions

  • Blocklist - exclude specific morphemes
  • OpenEnding - control sentence termination
  • AllowedDialects - dialect filtering

Tests

  • Added tests for all new features

Notes

  • Requires Kiwi C library v0.23.2 installation (make install-kiwi)

@mrchypark
feat: upgrade Kiwi to v0.23.2 and add new API bindings
abcfe2a 
- Update KIWI_VERSION from v0.23.0 to v0.23.2
- Add new MatchOption flags (OOV detection, SERIAL, EMOJI, etc.)
- Add new BuildOption flags (CONG model types, typo/multi dict)
- Implement Config struct and GetGlobalConfig/SetGlobalConfig
- Implement GetOptionF/SetOptionF/GetOption/SetOption
- Add Morphset type for blocklist support
- Update AnalyzeOptions with Blocklist, OpenEnding, AllowedDialects
- Add tests for new features
@mrchypark
mrchypark requested a review from kkweon August 6, 2026 19:02
kkweon
kkweon approved these changes Aug 6, 2026
View reviewed changes

@kkweon kkweon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

키위 라이브러리 너무 많이 바뀌어서 따라가기 힘드네요 흙흙.

@kkweon
kkweon enabled auto-merge (rebase) August 6, 2026 19:04
@kkweon
kkweon marked this pull request as draft August 7, 2026 04:54
auto-merge was automatically disabled August 7, 2026 04:54

Pull request was converted to draft

@kkweon
kkweon marked this pull request as ready for review August 7, 2026 04:54
@kkweon kkweon closed this Aug 7, 2026
@kkweon kkweon reopened this Aug 7, 2026
kkweon
kkweon requested changes Aug 7, 2026
View reviewed changes

@kkweon kkweon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 이 리뷰는 Claude Code로 작성했습니다. 변경된 코드를 Kiwi v0.23.2의 capi.h와 대조해 확인했고, 빌드 실패는 실제 CI 실행(run 31150063068)으로 검증했습니다. 나머지 지적 사항은 리뷰어(@kkweon)가 확인 후 함께 올립니다.

0.23.2 API를 폭넓게 반영해 주셔서 감사합니다. Config와 match option 커버리지가 꼼꼼합니다. 다만 approve 당시 CI가 돌지 않고 있었는데, 실행해 보니 빌드가 실패합니다. 머지 전에 이 부분과 API 설계 관련 몇 가지를 함께 봐주시면 좋겠습니다.

블로킹 이슈는 두 가지입니다: kiwi.go:585의 빌드 에러, 그리고 Morphset에 형태소를 추가할 방법이 없는 점입니다.

Comment thread kiwi.go Outdated
Comment thread kiwi.go
Comment thread kiwi.go Outdated
Comment thread kiwi.go Outdated
Comment thread kiwi.go Outdated
Comment thread kiwi.go
Comment thread kiwi.go Outdated
Comment thread kiwi_test.go
Comment thread kiwi_test.go
Comment thread kiwi_test.go
@kkweon kkweon mentioned this pull request Aug 7, 2026
Merged
@mrchypark
fix: address review comments
5f2f51f 
- Fix boolToCInt return type to C.uint8_t for integrate_allomorph
- Add Morphset.Add method with kiwi_morphset_add binding
- Add comments for OOV MatchOption (mutually exclusive 2-bit field)
- Add comments for Model Type BuildOption (single-select field)
- Fix dialect default to use instance setting when not specified
- Add comments for GetOptionF/SetOptionF (future compatibility)
- Improve tests: SetOption, OOV mode comparison, Morphset blocking
- Apply gofumpt formatting
@mrchypark
fix: update tests for v0.23.2
6c2ec5e 
- Update TestKiwiVersion expected value to 0.23.2
- Simplify TestGetSetOption to read-only (KIWI_NUM_THREADS may be read-only)
cypark-conalog

This comment was marked as abuse.

Sign in to view

@codingpot codingpot deleted a comment from cypark-conalog Aug 11, 2026
@codingpot codingpot deleted a comment from cypark-conalog Aug 11, 2026
@mrchypark

mrchypark commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

리뷰 감사합니다. 모든 코멘트를 반영했습니다:

블로킹 이슈 수정

  1. boolToCInt 반환 타입: C.int -> C.uint8_t로 수정 (integrate_allomorph 필드 타입 일치)
  2. Morphset.Add 메서드: kiwi_morphset_add 바인딩 추가, 에러 처리 포함

API 설계 개선

  1. OOV MatchOption 주석: 2비트 상호 배타적 필드임을 명시
  2. Model Type BuildOption 주석: 단일 선택 값임을 명시
  3. Dialect 기본값: AllowedDialects == 0일 때 인스턴스 설정 사용
  4. GetOptionF/SetOptionF: 향후 호환용이라는 주석 추가

테스트 개선

  1. TestGetSetOption: SetOption 호출 추가
  2. TestMatchOptionOOV: OOV 포함 텍스트로 두 모드 결과 확인
  3. TestMorphset: Add 후 차단 검증

코드 스타일

  1. gofumpt 적용: 정렬 문제 수정

@mrchypark
mrchypark enabled auto-merge August 11, 2026 12:20
@mrchypark
mrchypark requested review from cypark-conalog and kkweon and removed request for cypark-conalog August 11, 2026 12:21
@mrchypark
mrchypark dismissed cypark-conalog’s stale review August 11, 2026 12:38

Self-dismissed to allow fresh review

@mrchypark
fix: address oracle code review findings
6427cec 
- Fix CString memory leaks in New, NewBuilder, AddWord, LoadDict
- Add error returns to New and NewBuilder (breaking API change)
- Add thread safety documentation to Kiwi type
- Add test coverage: New failure, WithOpenEnding, WithAllowedDialects
- Fix error handling in all tests (assert.NoError)
- Apply gofumpt formatting
@mrchypark

mrchypark commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

오라클 코드 리뷰 결과를 반영했습니다:

수정 사항

메모리 누수 수정

  • New, NewBuilder, AddWord, LoadDict에서 CString 할당 후 free 추가

에러 처리 추가

  • New, NewBuilder가 에러 반환 (nil 핸들러 검사)
  • 모든 테스트에서 assert.NoError 추가

스레드 안전성 문서화

  • Kiwi 타입에 동시성 주석 추가

테스트 커버리지 보강

  • TestNewFailure: 잘못된 경로 테스트
  • TestWithOpenEnding: 문장 종결 옵션 테스트
  • TestWithAllowedDialects: 방언 설정 테스트

테스트 모두 통과 확인: ok github.com/codingpot/kiwigo 72.277s

kkweon
kkweon approved these changes Aug 11, 2026
View reviewed changes

@kkweon kkweon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 반영 감사합니다! 빌드/테스트 통과 확인했습니다. 🙏

@mrchypark
mrchypark merged commit b869bf7 into codingpot:main Aug 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kkweon kkweon kkweon approved these changes
+1 more reviewer
@cypark-conalog cypark-conalog cypark-conalog left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mrchypark @kkweon @cypark-conalog