Token category

Gemma 1 & 2 SentencePiece Emoji

Emoji often tokenize as multiple UTF-8 byte fragments or mixed symbol pieces. Skin tones, zero-width joiners, and flags can span several tokens.