Tokenizer vocabulary

Gemma 3 SentencePiece

Google's Gemma 3 SentencePiece BPE tokenizer used by Gemini local token counting, with a 262k vocabulary and byte fallback pieces. Vocabulary size, token ranges, and special-token IDs are listed here.

Google

Creator
Google
Mergeable tokens
262,144
Total known tokens
262,144

Browse by type

PunctuationSymbols and marks WhitespaceSpaces, tabs, newlines EmojiPictographs and fragments MultilingualExamples across scripts CodeProgramming tokens Control/special tokensReserved tokenizer values
Open token index