issue when importing BloomTokenizer from transformers in python

Question

I am trying to import BloomTokenizer from transformers

from transformers import BloomTokenizer

and I receive the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'BloomTokenizer' from 'transformers' 
(/root/miniforge3/envs/pytorch/lib/python3.8/site-packages/transformers/__init__.py)

my version of transformers:

transformers                 4.20.1

what could I do to be able to import BloomTokenizer?

Asked By: Marcel

||

Source

Answer 1

BLOOM has no slow tokenizer class. It only has a fast tokenizer. The official documentation is wrong at this point. Use the following instead:

 from transformers import BloomTokenizerFast
 tokenizer = BloomTokenizerFast.from_pretrained("...")

Answered By: cronoik

issue when importing BloomTokenizer from transformers in python

Question:

Answers: