Name		Name	Last commit message	Last commit date
data		data
.gitignore		.gitignore
README.md		README.md
align.py		align.py
align.ted.sh		align.ted.sh
build_dict.sh		build_dict.sh
combine_line.py		combine_line.py
detokenizer.py		detokenizer.py
line_separator.py		line_separator.py
post_tokenize.py		post_tokenize.py
refine.py		refine.py
regex.txt		regex.txt
run_all_joongang.py		run_all_joongang.py
run_all_ted.sh		run_all_ted.sh
run_preprocessing.sh		run_preprocessing.sh
run_preprocessing_clien.sh		run_preprocessing_clien.sh
setup.WSL.sh		setup.WSL.sh
tokenizer.py		tokenizer.py
word_mt.py		word_mt.py

Repository files navigation

Preprocessing Utils

This repo contains python scripts for preprocessing of (Korean) NLP. Also, it contains aligner to build parallel corpora.

Usage

About

Scripts for NLP preprocessing

1 star

1 watching

Report repository

Releases

Packages

Contributors

Languages