Skip to main content

0.12.0

  • Added OpenAI compatible interface layer
  • Spacelike Speculative Decoding enabled for non-static models. Uses in memory cache for higher generation performance.
  • Support for LLava image to text models.
  • Support for Google's gemma model series