How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese

Add code
Jun 16, 2023

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: