From OpenAI to Open LLMs with Messages API on Hugging Face

Back to Articles From OpenAI to Open LLMs with Messages API on Hugging Face Published February 8, 2024 Update on GitHub Upvote 20 +14 Andrew Reed andrewrreed Follow Philipp Schmid philschmid Follow Joffrey THOMAS Jofthomas Follow David Holtz drbh Follow We are excited to introduce the Messages API to provide OpenAI compatibility with Text Generation Inference (TGI) and Inference Endpoints. Starting with version 1.4.0, TGI offers an API compatible with the OpenAI Chat Completion API. The new Messages API allows customers and users to transition seamlessly from OpenAI models to open LLMs. The API can be directly used with OpenAI's client libraries or third-party tools, like LangChain or LlamaIndex. "The new Messages API with OpenAI compatibility makes it easy for Ryght's real-time GenAI orchestration platform to switch LLM use cases from OpenAI to open models. Our migration from GPT4 to Mixtral/Llama2 on Inference Endpoints is effortless, and now we have a simplified workflow with more control over our AI solutions." - Johnny Crupi, CTO at Ryght The new Messages API is also now available in Inference Endpoints, on both dedicated and serverless flavors. To get you started quickly, we’ve included detailed examples of how to: Create an Inference Endpoint Using Inference Endpoints with OpenAI client libraries Integrate with LangChain and LlamaIndex Limitations: The Messages API does not currently support function calling and will only work for LLMs with a chat_template defined i...