ESGenuine
Evidence-grounded ESG claim extraction — with a deterministic layer that repairs what the LLM gets wrong.
ESGenuine is an evidence-grounded ESG claim extraction and auditing system. It turns a 90-page sustainability report into structured, provenance-carrying claims, then measures how much of that extraction can actually be trusted — with a deterministic post-correction layer that repairs what the language model gets wrong at 0.45 ms per claim and zero inference cost. Every reported number regenerates offline from committed fixtures, with no API key and no network.
ESG disclosures are unstructured and unverified — the same metric appears as a table cell in one report, a sentence in another, and a footnote in a third. LLM extraction from them fails quietly: 31.7% of a 70B model's emitted values do not appear on the page it cites, and ratings from major providers agree at a correlation of only ~0.54.
Docling parses the PDF into sections and tables, an LLM (Llama-3.3-70B via NVIDIA NIM) extracts claims, and a five-stage deterministic repair layer — taxonomy normalisation, fiscal-year column repair, source-value verification, a rule-based gate, and a furniture filter — corrects them before each claim is grounded against its source page and stored with provenance in Supabase. NLI contradiction checks and a greenwashing taxonomy feed a document integrity report in a React dashboard.
- 1Catching silent LLM errors — plausible values that never occur on the cited page.
- 2Measuring recall honestly, against table cells enumerated from the parsed document instead of a gold set sampled from the system's own output.
- 3Different disclosure formats need different repairs — statutory filings and narrative reports respond to different stages.
- Deterministic repair layer: +24.6 points on a statutory filing at zero inference cost.
- Central finding: the dominant repair depends on disclosure format — with non-overlapping 95% CIs.
- Eight evaluation harnesses with bootstrap confidence intervals and a CI gate on ground truth.
- Fully reproducible offline — seeded RNGs and models pinned by commit SHA.