Introducing the Enterprise Scenarios Leaderboard: a Leaderboard for Real World Use Cases

Back to Articles Introducing the Enterprise Scenarios Leaderboard: a Leaderboard for Real World Use Cases Published January 31, 2024 Update on GitHub Upvote 3 Selvan Sunitha Ravi sunitha98 Follow guest Rebecca Qian rebeccaqian Follow guest Anand Kannappan anandnk24 Follow guest Clémentine Fourrier clefourrier Follow Today, the Patronus team is excited to announce the new Enterprise Scenarios Leaderboard, built using the Hugging Face Leaderboard Template in collaboration with their teams. The leaderboard aims to evaluate the performance of language models on real-world enterprise use cases. We currently support 6 diverse tasks - FinanceBench, Legal Confidentiality, Creative Writing, Customer Support Dialogue, Toxicity, and Enterprise PII. We measure the performance of models on metrics like accuracy, engagingness, toxicity, relevance, and Enterprise PII. Why do we need a leaderboard for real world use cases? We felt there was a need for an LLM leaderboard focused on real world, enterprise use cases, such as answering financial questions or interacting with customer support. Most LLM benchmarks use academic tasks and datasets, which have proven to be useful for comparing the performance of models in constrained settings. However, enterprise use cases often look very different. We have selected a set of tasks and datasets based on conversations with companies using LLMs in diverse real-world scenarios. We hope the leaderboard can be a useful starting point for users trying to...