How to use erbacher/Pde1dTokenizerLFQ with Transformers:
# Load model directly from transformers import Tokenizer model = Tokenizer.from_pretrained("erbacher/Pde1dTokenizerLFQ", dtype="auto")
