toiro 0.0.4
toiro 0.0.4 incorporates the following changes:
- add disable_tokenizers function to
tokenizers.compare
- fix a bug in the initial release.
- fix error for a long input text in Jumanpp
- add 01_getting_started_ja.ipynb to README.md
toiro 0.0.4 incorporates the following changes:
tokenizers.compare