Classical Chinese Model needed

[KoichiYasuoka]

I've almost finished to build up UD_Classical_Chinese-Kyoto Treebank, and now I'm trying to make a Classical Chinese model for NLP-Cube (please check my diary). But in my model sentence_accuracy<35 and I can't sentencize "天平二年正月十三日萃于帥老之宅申宴會也于時初春令月氣淑風和梅披鏡前之粉蘭薰珮後之香加以曙嶺移雲松掛羅而傾盖夕岫結霧鳥封縠而迷林庭舞新蝶空歸故鴈於是盖天促膝飛觴忘言一室之裏開衿煙霞之外淡然自放快然自足若非翰苑何以攄情詩紀落梅之篇古今夫何異矣宜賦園梅聊成短詠" (check gold standard here). How do I tune up sentencization for Classical Chinese?

[tiberiu44]

I looked over the corpus, and I see there are no delimiters (punctuation marks) for sentences. Is this ik?

[KoichiYasuoka]

Yes, OK. Classical Chinese does not have any punctuations or spaces between words or sentences. Therefore, in my humble opinion, tokenization is a hard task without POS-tagging, and sentencization is a hard task without dependency parsing...

[tiberiu44]

I think we could go for jointly POS-Tagging and tokenising. Unfortunately, the algorithm we use for dependency parsing requires us to build a NxN matrix for all the words (N), which is likely to cause an out of memory error if we use all tokens. Do you know of any other approach, that does not require dependency parsing for sentence segmentation?

[KoichiYasuoka]

Umm... I only know Straka & Straková (2017) approach using dynamic programming (see section 4.3), but it requires tentative parse trees...

[tiberiu44]

I see. I can imagine joint sentence segmentation and parsing working by using the ARC-system. Whenever the stack is emptied, it implies that a sentence boundary should be generated.

We've finished work for the Parser and Tagger for version 2.0, but we still haven't found a good solution for tokenization/sentence splitting.

I think I will give this new approach a try, but it will take some time to implement. I'll let you know when it's done and maybe you can test it on your corpus.

Thanks for the feedback,
Tibi

[tiberiu44]

@KoichiYasuoka - i haven't had any success with the tokenizer/sentence splitter so far. We are working on rolling out version 2.0 which uses a single model conditionally trained with language embeddings. We have great accuracy figures for the parser and tagger. However, we are still experiencing difficulties with the tokenizer (for all languages).

We tried jointly tagging/parsing and tokenizing, but we simply got the same results as if we would do these two tasks independently. Any suggestions on how to proceed?

[KoichiYasuoka]

Umm... For Japanese tokenisation (word splitting) and POS-tagging, we often apply Conditional Random Fields as Kudo et al. (2004). For Classical Chinese, we also use CRF in our UD-Kanbun.

For sentence segmentation in Classical Chinese, recent progress has been made by Hu et al. (2019) at https://seg.shenshen.wiki/. Hu et al. uses BERT-model, which is trained by enormous Classical Chinese texts of 3.3×10⁹ characters...

[tiberiu44]

@KoichiYasuoka - I hope you are doing well in this time of crisis.

It's been a long time since our last progress update on this issue. We started training the 2.0 models for NLP-Cube and they should be out soon. I saw the Classical Chinese corpus in the UD Treebanks (v2.5). The model will be included in this release. Congratulations and thank you for your work.

I thought you might be interested in the fact that we are also setting up a "model zoo" for NLP-Cube, so contributors can publish their pre-trained models. We will try to make research attribution easy, by printing a banner with copyright and/or citing options for these models.

[KoichiYasuoka]

@tiberiu44 - Thank you for using our UD_Classical_Chinese-Kyoto for your NLP-Cube. We've just finished to add 19 more volumes from "禮記" into https://github.com/UniversalDependencies/UD_Classical_Chinese-Kyoto/tree/dev for the v2.6 release of UD Treebanks (scheduled on May 15, 2020). Enjoy!

[tiberiu44]

Hi @KoichiYasuoka ,

We've finished releasing the current version of NLPCube and we included the classical Chinese model from 2.7. Sentence segmentation seems to be problematic for this treebank. You can check branch 3.0 of the repo to get more info: https://github.com/adobe/NLP-Cube/tree/3.0

If you have any suggestions regarding sentence segmentation, please let me know. Right now we are using xlm-roberta-base for language modeling, but maybe there is some other LM that can provide better results.

Best,
Tiberiu

[KoichiYasuoka]

Thank you @tiberiu44 for releasing NLP-Cube 3.0. But, well, pytorch-lightning==1.1.7 is too old for recent torchtext==0.10.0 so I use pytorch-lightning==1.2.10 instead:

>>> from cube.api import Cube
>>> nlp=Cube()
>>> nlp.load("lzh")
>>> doc=nlp("不入虎穴不得虎子")
>>> print(doc)
1	不入虎穴不得虎子	叔津	PROPN	n,名詞,人,複合的人名	NameType=Prs	0	root	_	_

Umm... tokenization of classical Chinese doesn't work here...

[tiberiu44]

Yes, I see something is definitely wrong with the model. Just tried you example and tokenization did not work. However, on longer examples it seems to behave differently:

Out[13]:
1	子曰學而時習之不亦說乎	子春城	PROPN	n,名詞,人,名	NameType=Giv	2	nsubj	_	_
2	有	有	VERB	v,動詞,存在,存在	_	0	root	_	_
3	朋	朋	NOUN	n,名詞,人,関係	_	2	obj	_	_
4	自	自	ADP	v,前置詞,経由,*	_	6	case	_	_
5	遠	遠	VERB	v,動詞,描写,量	Degree=Pos|VerbForm=Part	6	amod	_	_
6	方	方	NOUN	n,名詞,固定物,関係	Case=Loc	7	obl	_	_
7	來	來	VERB	v,動詞,行為,移動	_	2	ccomp	_	_
8	不	不	ADV	v,副詞,否定,無界	Polarity=Neg	14	advmod	_	_
9	亦	亦	ADV	v,副詞,頻度,重複	_	10	advmod	_	_
10	樂	樂	VERB	v,動詞,行為,態度	_	2	conj	_	_
11	乎	乎	ADP	v,前置詞,基盤,*	_	12	case	_	_
12	人	人	NOUN	n,名詞,人,人	_	7	obl	_	_
13	不	不	ADV	v,副詞,否定,無界	Polarity=Neg	14	advmod	_	_
14	知	知	VERB	v,動詞,行為,動作	_	10	parataxis	_	_

1	而	而	CCONJ	p,助詞,接続,並列	_	3	advmod	_	_
2	不	不	ADV	v,副詞,否定,無界	Polarity=Neg	3	advmod	_	_
3	慍	慍	VERB	v,動詞,行為,態度	_	6	csubj	_	_
4	不	不	ADV	v,副詞,否定,無界	Polarity=Neg	6	advmod	_	_
5	亦	亦	ADV	v,副詞,頻度,重複	_	6	advmod	_	_
6	君子	君子	NOUN	n,名詞,人,役割	_	0	root	_	_
7	乎	乎	PART	p,助詞,句末,*	_	6	discourse:sp	_	_```

I will try retraining the tokenizer with a different LM.

[KoichiYasuoka]

Umm... first eleven characters seem untokenized:

>>> from cube.api import Cube
>>> nlp=Cube()
>>> nlp.load("lzh")
>>> doc=nlp("子曰道千乘之國敬事而信節用而愛人使民以時")
>>> print(doc)
1	子曰道千乘之國敬事而信	子春于	PROPN	n,名詞,人,名	NameType=Giv	2	nsubj	_	_
2	節	節	VERB	v,動詞,描写,態度	Degree=Pos	0	root	_	_
3	用	用	VERB	v,動詞,行為,動作	_	2	flat:vv	_	_

1	而	而	CCONJ	p,助詞,接続,並列	_	2	advmod	_	_
2	愛	愛	VERB	v,動詞,行為,交流	_	6	csubj	_	_
3	人	人	NOUN	n,名詞,人,人	_	2	obj	_	_
4	使	使	VERB	v,動詞,行為,使役	_	2	parataxis	_	_
5	民	民	NOUN	n,名詞,人,人	_	4	obj	_	_
6	以	以	VERB	v,動詞,行為,動作	_	0	root	_	_
7	時	時	NOUN	n,名詞,時,*	Case=Tem	6	obj	_	_