![[2411.12174] Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2411.12174] Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes
Summary
The paper presents a novel framework, Just KIDDIN, that combines Knowledge Distillation and knowledge infusion to improve the detection of toxic memes, achieving superior performance on benchmark datasets.
Why It Matters
As online toxicity continues to be a significant issue, this research addresses the challenge of detecting harmful content in multimodal environments. By integrating knowledge from various sources, the framework enhances the accuracy of toxicity detection, which is crucial for creating safer online spaces.
Key Takeaways
- The Just KIDDIN framework uses Knowledge Distillation from Large Visual Language Models to enhance toxicity detection.
- Integration of knowledge graphs improves the model's reasoning capabilities in identifying toxic content.
- Experimental results show significant performance improvements over existing state-of-the-art methods.
- The approach combines explicit and implicit contextual cues for better accuracy.
- This research is vital for developing scalable solutions to combat online toxicity.
Computer Science > Machine Learning arXiv:2411.12174 (cs) [Submitted on 19 Nov 2024 (v1), last revised 16 Feb 2026 (this version, v3)] Title:Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes Authors:Rahul Garg, Trilok Padhi, Hemang Jain, Ugur Kursuncu, Ponnurangam Kumaraguru View a PDF of the paper titled Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes, by Rahul Garg and 4 other authors View PDF HTML (experimental) Abstract:Toxicity identification in online multimodal environments remains a challenging task due to the complexity of contextual connections across modalities (e.g., textual and visual). In this paper, we propose a novel framework that integrates Knowledge Distillation (KD) from Large Visual Language Models (LVLMs) and knowledge infusion to enhance the performance of toxicity detection in hateful memes. Our approach extracts sub-knowledge graphs from ConceptNet, a large-scale commonsense Knowledge Graph (KG) to be infused within a compact VLM framework. The relational context between toxic phrases in captions and memes, as well as visual concepts in memes enhance the model's reasoning capabilities. Experimental results from our study on two hate speech benchmark datasets demonstrate superior performance over the state-of-the-art baselines across AU-ROC, F1, and Recall with improvements of 1.1%, 7%, and 35%, respectively. Given the contextual complexity of the toxicity detection task, our approach s...