![[2509.19680] PolicyPad: Collaborative Prototyping of LLM Policies](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2509.19680] PolicyPad: Collaborative Prototyping of LLM Policies
Summary
The article presents PolicyPad, an interactive system designed for collaborative prototyping of policies governing large language models (LLMs), particularly in high-stakes domains like mental health and law.
Why It Matters
As LLMs are increasingly utilized in critical areas, establishing effective policies is essential for their safe deployment. PolicyPad enhances collaboration among domain experts, facilitating rapid prototyping and feedback, which is crucial for AI alignment and safety.
Key Takeaways
- PolicyPad allows real-time collaboration among policy designers.
- The system incorporates established UX practices to enhance policy prototyping.
- Workshops showed that PolicyPad improved collaborative dynamics and feedback loops.
- The tool is particularly beneficial for high-stakes domains like mental health and law.
- Findings support the advancement of AI alignment and safety through expert-informed policies.
Computer Science > Human-Computer Interaction arXiv:2509.19680 (cs) [Submitted on 24 Sep 2025 (v1), last revised 18 Feb 2026 (this version, v2)] Title:PolicyPad: Collaborative Prototyping of LLM Policies Authors:K. J. Kevin Feng, Tzu-Sheng Kuo, Quan Ze (Jim)Chen, Inyoung Cheong, Kenneth Holstein, Amy X. Zhang View a PDF of the paper titled PolicyPad: Collaborative Prototyping of LLM Policies, by K. J. Kevin Feng and 5 other authors View PDF HTML (experimental) Abstract:As LLMs gain adoption in high-stakes domains like mental health, domain experts are increasingly consulted to provide input into policies governing their behavior. From an observation of 19 policymaking workshops with 9 experts over 15 weeks, we identified opportunities to better support rapid experimentation, feedback, and iteration for collaborative policy design processes. We present PolicyPad, an interactive system that facilitates the emerging practice of LLM policy prototyping by drawing from established UX prototyping practices, including heuristic evaluation and storyboarding. Using PolicyPad, policy designers can collaborate on drafting a policy in real time while independently testing policy-informed model behavior with usage scenarios. We evaluate PolicyPad through workshops with 8 groups of 22 domain experts in mental health and law, finding that PolicyPad enhanced collaborative dynamics during policy design, enabled tight feedback loops, and led to novel policy contributions. Overall, our work p...
