Introducing the Red-Teaming Resistance Leaderboard

Back to Articles Introducing the Red-Teaming Resistance Leaderboard Published February 23, 2024 Update on GitHub Upvote 13 +7 Steve Li steve-sli Follow guest Richard richard2 Follow guest Leonard Tang leonardtang Follow guest Clémentine Fourrier clefourrier Follow Content warning: since this blog post is about a red-teaming leaderboard (testing elicitation of harmful behavior in LLMs), some users might find the content of the related datasets or examples unsettling. LLM research is moving fast. Indeed, some might say too fast. While researchers in the field continue to rapidly expand and improve LLM performance, there is growing concern over whether these models are capable of realizing increasingly more undesired and unsafe behaviors. In recent months, there has been no shortage of legislation and direct calls from industry labs calling for additional scrutiny on models – not as a means to hinder this technology’s progress but as a means to ensure it is responsibly deployed for the world to use. To this end, Haize Labs is thrilled to announce the Red Teaming Resistance Benchmark, built with generous support from the Hugging Face team. In this benchmark, we thoroughly probe the robustness of frontier models under extreme red teaming efforts. That is, we systematically challenge and test these models with craftily constructed prompts to uncover their failure modes and vulnerabilities – revealing where precisely these models are susceptible to generating problematic outputs....