Transformer Softmax Bottleneck

An analysis of the effects of Mixture of Softmaxes on the Transformer architecture.

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
.idea		.idea
__pycache__		__pycache__
data		data
.gitignore		.gitignore
Breaking_the_Transformer_Bottleneck.pdf		Breaking_the_Transformer_Bottleneck.pdf
Corpus.py		Corpus.py
LICENSE		LICENSE
README.md		README.md
Table1.png		Table1.png
Table2.png		Table2.png
main.py		main.py
model.py		model.py
requirements.txt		requirements.txt

Provide feedback