Factuality benchmarks

Evaluate a model on what was actually true as of a given date, using a dated change log rather than a snapshot someone labeled by hand.

Request access

Every item traces to a capture, so a disputed answer has an arbiter.

The job

A benchmark without a date grades the wrong thing.

Ask a model what a company charges and mark it against today's pricing page, and you punish a model that correctly remembers last year while rewarding one that guessed the current number. The label is a point in time, but the eval treats it as timeless.

The record keeps the transitions themselves: when a tier appeared, when language changed, when a certification was first observed. That makes as-of-date evaluation possible, and it makes disagreements resolvable, because each item points at the capture behind it.

What that costs today

  • Hand-labeled sets that go stale the month they ship
  • Evals that cannot distinguish outdated from wrong
  • Disputed items with no artifact to settle them
  • Benchmarks the model may have already trained on the answers to

The API

4 calls cover the job.

Current state is open. History values, claim checks, and evidence trails sit behind a free key. These are the real routes.

GETopen
/v1/companies/{company}/facts

Facts as of a date: `as_of=` is the time machine.

GETmasked · key unmasks
/v1/companies/{company}/history

The dated log of every change; values unlock with a free key.

GETmasked · key unmasks
/v1/changes

Changes across companies in a window, for building eval sets.

GETopen
/v1/companies/{company}/facts/{key}/proof

The proof bundle that settles a contested item.

For agents · MCP

Or let your agent run it.

Connect the Bixel MCP server and any agent, Claude, an internal copilot, a procurement bot, works this job against live structured data, with every value citing the dated capture it came from. The recommendation becomes auditable, not a guess.

See the MCP tool list

You ask your agent

“What did Pinecone's pricing page state in January, and what changed after that?”

Ground truth that carries a date.

A company's dated change log is the raw material for as-of evaluation: see a dated change log →

More in AI & Agents

The answer layer for agents that need company facts with receipts.