EMERGE Benchmark Catches AI Failing to Keep Knowledge Graphs Current
EMERGE pairs 233K Wikipedia passages with 1.45M Wikidata edits across seven yearly snapshots (2019-2025). Introduces five Text-driven KG Updating operations: Exists, Add, Mint+Add, Infer, and Deprecate. Accepted at ICLR 2026; benchmarks 13 models including GPT-5.1 with the EDC+ pipeline.
Takeaways
- EMERGE pairs 233K Wikipedia passages with 1.45M Wikidata edits across seven yearly snapshots (2019-2025).
- Introduces five Text-driven KG Updating operations: Exists, Add, Mint+Add, Infer, and Deprecate.
- Accepted at ICLR 2026; benchmarks 13 models including GPT-5.1 with the EDC+ pipeline.
- Model performance degrades as KG deltas grow, revealing weakness on stale graphs.
- Deprecate and Infer operations are hardest, exposing lack of graph-aware reasoning in LLMs.
- Code at github.com/klimzaporojets/emerge , dataset on Hugging Face .
Knowledge graphs drift out of date as organizations merge, people change jobs, new entities appear, and old relationships expire. Information extraction systems usually read text and emit standalone triples without consulting the graph already in storage. EMERGE , a benchmark accepted at ICLR 2026, evaluates the harder task of updating an existing graph from new text.
Researchers from Aarhus University, the University of Amsterdam, and Sapienza University of Rome frame the problem as Text-driven Knowledge Graph Updating, or TKGU. Each update depends on two sources: the claims expressed in a passage and the current state of the knowledge graph. A correct system may preserve an existing edge, add one, create an entity, infer related edges, or retire an obsolete fact.
Five Ways a Graph Can Change
Conventional information extraction can add the triple “person, holds position, CEO” after reading an appointment announcement. A production knowledge graph also needs to resolve the person and company to existing nodes, retire the previous CEO relationship, and avoid duplicating facts already stored.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
