Issues

• Add support for readonlyspan parameter for encode and count of tokens

  dmitry-brazhenko/SharpToken#51 · Filter by author ali-hm opened on Nov 18, 2025

• Update models list

  dmitry-brazhenko/SharpToken#50 · Filter by author lyczkop opened on Oct 24, 2025

• Is there a way to get vocabulary size of an initialized tokenizer?

  dmitry-brazhenko/SharpToken#47 · Filter by author zhuzihan728 opened on Apr 21, 2025

• add support for utf8 input and output

  dmitry-brazhenko/SharpToken#46 · Filter by author abdulkareemnalband opened on Sep 13, 2024
  2

• SharpToken.GptEncoding.GetEncoding(o200k_base) takes minutes

  dmitry-brazhenko/SharpToken#45 · Filter by author vovanb opened on Jul 25, 2024
  1

• Gbt4o support

  dmitry-brazhenko/SharpToken#42 · Filter by author daniell0gda opened on May 16, 2024

• cohere support

  dmitry-brazhenko/SharpToken#38 · Filter by author omri-suissa-clearmash opened on Apr 17, 2024