BPE tokenization and Word2Vec embeddings on Hadoop, with tuning for embedding size.
scala logback word2vec scalatest hadoop-mapreduce deeplearning4j nd4j apache-hadoop llm bpe-tokenizer jtokkit
-
Updated
Aug 26, 2025 - Scala