[2510.05710] FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation

Quantitative Finance > Computational Finance arXiv:2510.05710 (q-fin) [Submitted on 7 Oct 2025 (v1), last revised 19 Mar 2026 (this version, v2)] Title:FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation Authors:Fabrizio Dimino, Abhinav Arun, Bhaskarjit Sarmah, Stefano Pasquali View a PDF of the paper titled FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation, by Fabrizio Dimino and 3 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) are increasingly being used to extract structured knowledge from unstructured financial text. Although prior studies have explored various extraction methods, there is no universal benchmark or unified evaluation framework for the construction of financial knowledge graphs (KG). We introduce FinReflectKG - EvalBench, a benchmark and evaluation framework for KG extraction from SEC 10-K filings. Building on the agentic and holistic evaluation principles of FinReflectKG - a financial KG linking audited triples to source chunks from S&P 100 filings and supporting single-pass, multi-pass, and reflection-agent-based extraction modes - EvalBench implements a deterministic commit-then-justify judging protocol with explicit bias controls, mitigating position effects, leniency, verbosity and world-knowledge reliance. Each candidate triple is evaluated with binary judgments of faithfulness, precision, and relevance, while comprehensiveness is assessed on ...