How we count

Methodology

Version 1.0.0 (METHODOLOGY_VERSION in lib/taxonomy.ts) Last updated 2026-09-08.

This document is written to be read by a lawyer acting for a company we have named. It states what our numbers are, what they are not, and where they are weak. Nothing here is written to flatter the data.

Every report we publish embeds the version of this document it was generated under, plus the response rate for that reporting window. A number from us without those two things attached is not ours.

1. What we collect

A dispute is one member's grievance against one company, recorded when the member generates a letter. It is stored as a structured record, not free text. The full schema is supabase/migrations/20260908_dispute_structured.sql; the vocabularies are lib/taxonomy.ts and are enforced by database CHECK constraints and by a build-time guard (scripts/check-taxonomy.ts) that fails the build if the code and the database disagree.

Sectors

telco, bank, insurance, energy, airline, landlord, retail, government, other.

Issue types

Scoped per sector. Telco is built out fully because it is the first sector report:

billing_error, charged_after_cancellation, late_fee_disputed, service_not_delivered, outage_compensation, contract_term_change, debt_collection, porting_failure, handset_fault, plan_misrepresentation, other.

Other sectors carry equivalent sets, listed in lib/taxonomy.ts. They should be reviewed before that sector's first report.

The member's own words

disputes.description holds what the member wrote. It is never aggregated and never counted. It is used for two things only: generating their letter, and classification during backfill. When any part of it is rendered publicly it passes through lib/redact.ts first, which strips emails, phone numbers, labelled account and policy numbers, card-length digit runs, street addresses, state/postcode pairs, URLs and self-introduced names.

2. What counts as an outcome

An outcome is a member telling us what happened. Statuses that count as a reported outcome: won, partial, lost, abandoned, no_response.

Two provenance levels, recorded in outcome_source and never conflated:

Level	Meaning	Where it may appear
`self_reported`	The member told us. Nothing was checked.	Aggregate counts and rates
`verified_screenshot`	The member supplied evidence and a human checked it against `docs/WIN_VERIFICATION.md`.	Everything, including the wins board and share cards

A dispute cannot record an outcome without a source. The database rejects it (disputes_outcome_needs_source).

We do not verify losses. Nobody sends a screenshot of being refused. So verified outcomes are structurally biased toward wins, which is precisely why a verified-win count must never be divided by anything to produce a rate. Rates use self-reported outcomes; verification is for the wins board.

3. Density thresholds

No comparative claim about a named company is published below both of:

150 disputes for that company within the reporting window
60 reported outcomes for that company within the same window

This is not a guideline. lib/density.ts returns a discriminated union whose percentage field does not exist on the below-threshold branch, so a page cannot render a rate it is not entitled to — a caller that forgets to check gets a type error or the "insufficient data" branch, never a number. Asserted in tests/guards.test.ts.

Below threshold we say so explicitly, with the real shortfall: "Not enough reported outcomes to publish a resolution rate. 41 reported from 380 disputes; 60 needed." Silence would be less honest than the shortfall.

Why these numbers

They are a judgement, not a derivation, and we would rather say that than dress them up. 150/60 is the point at which a single unusual case stops moving the headline figure by more than a percentage point or two, and it is deliberately set above the level at which we would first be tempted to publish. If we are challenged on them we will show the sensitivity of the rate to n, not assert that 150 is a magic number.

4. Non-response bias, stated plainly

People who win tell us. People who lose or give up mostly do not.

Every resolution rate we publish is therefore an overestimate of how well disputes actually go, and the size of the overestimate is unknown. We do not correct for it, because any correction would be a guess dressed as a method. Instead:

The response rate is published beside every resolution rate, always. lib/density.ts returns them together in the same object so they cannot be separated by accident. Response rate = reported outcomes ÷ disputes.
We count our prompting. outcome_prompts records every prompt sent and whether it was answered, so the response rate is a measured number, not an estimate.
A resolution rate quoted without its response rate is not a figure from us, and we will say so publicly if one is reproduced that way.

If the response rate is 30%, then a 70% resolution rate means "70% of the 30% who told us". We will write it that way.

5. Reporting windows

A report covers a calendar window stated on the report itself.
A dispute belongs to the window containing its created_at.
An outcome belongs to the window containing its dispute's created_at, not the outcome's own date. Otherwise a dispute and its outcome fall in different windows and the rate is computed across mismatched denominators.
Outcome capture rate excludes letters sent in the last 30 days: they have not had time to produce an outcome, and including them makes the rate move for the wrong reason.

6. Attribution to companies

A dispute is attributed to a company by company_id. That is set by matching the extracted institution string against companies.name and companies.aliases (supabase/migrations/20260712_company_aliases.sql).

Unmatched disputes are excluded from company-level figures entirely. They are not bucketed into "other" and they are not guessed at. The count of unmatched disputes is reported alongside every company table so the size of the excluded pool is visible.

Free-text institution values with no company_id are shown in the admin view marked "unlinked free text" and are never used in a published report, because "Telstra" and "telstra corp" would count as two companies.

Duplicates

One member opening several disputes against the same company for the same issue within a window counts once for company-level rates. Duplicate detection is currently not implemented — see BUILD_REPORT. Until it is, company dispute counts should be read as "disputes", not "distinct complainants", and the report says so.

7. Backfill accuracy

Historical disputes predate this schema. scripts/backfill-dispute-taxonomy.ts maps them forward conservatively: anything not confidently recognised becomes other rather than being guessed into a sector, because a wrong classification silently corrupts an aggregate while an absent one merely shrinks it.

The script prints the percentage that landed in other for both sector and issue type. Those percentages must be recorded here after the first production run and quoted in any report that includes pre-2026-09 disputes.

Backfill run: 2026-09-09 against production, 64 disputes. sector = other: 18.8% (12 of 64) · issue_type = other: 98.4% (63 of 64)

issue_type was not written. At 98.4% unclassifiable, stamping other across the corpus would have permanently closed the door on a better classifier — the backfill only fills nulls, so a written value is final as far as it is concerned. Historical rows therefore carry a null issue_type, the report counts and prints them as unclassified, and no issue-type breakdown may be published for pre-2026-09 disputes.

Sector was written and is usable, with the 18.8% other share quoted wherever a sector figure includes historical rows.

Company attribution, measured 2026-09-09

	Disputes	Share
Total	64	100%
Matched to a company	40	62.5%
Unmatched: no institution string was ever recorded	17	26.6%
Unmatched: institution string was an addressee title, not an entity	7	10.9%

24 of 64 disputes — 37.5% of the corpus — are excluded from every company-level figure. §6 excludes unmatched disputes entirely rather than bucketing them into "other", so that 37.5% is not distributed anywhere. It is absent.

Any report quoting historical data must show this figure.

Every dispute that carries an institution string is now matched: 40 of 40, across 17 distinct strings. The excluded 37.5% is therefore entirely disputes where the company was never captured, not disputes we failed to resolve.

The 7 addressee rows were Complaints Officer, Customer Relations Manager, Internal Dispute Resolution Manager, Customer Relations Team, Internal Dispute Resolution Officer and Review Officer — job titles scraped from the letter salutation by a regex that has since been replaced. Their institution field was set to null on 2026-09-09 so they can never be matched or counted as a company. isGenericAddressee in lib/extract-institution.ts catches all seven today, so this is closed, not recurring.

The 17 blank rows were left untouched. They are honestly unknown.

If issue_type = other is high, issue-type breakdowns are suppressed for historical rows rather than presented as if they were classified.

8. Member-reported versus regulator data

These are different things and are never mixed in one figure.

Member-reported: everything in our database. Self-selected, unaudited except where marked verified. Always labelled in the UI by a component, not by copy, so it cannot be forgotten.
Regulator data: published complaint statistics from bodies such as the TIO or AFCA, ingested separately. Cited to source and dated.

Where a report puts them side by side, it is explicitly a comparison of two different populations, not a validation of one by the other. Our population is people who found us and chose to act. Theirs is people who escalated to a regulator. Neither is the general population.

9. Corrections policy

Any error of fact in a published report or on a company page is corrected within 5 business days of being substantiated.
Corrections are made in place, with a dated note on the same page. We do not silently edit. The original figure stays visible in the note.
A correction that changes a headline figure by more than 2 percentage points, or that changes which company leads a table, is additionally posted as a standalone note and linked from the report index.
A log of all corrections is kept publicly. If you believe something here is wrong, the fastest route is the contact address on the site; we would rather be corrected than defended.
Reports are never retracted quietly. A withdrawn report is replaced by a page explaining why.

10. What we will not do

Publish a resolution rate below the density thresholds.
Publish a resolution rate without its response rate.
Present verified wins as a rate.
Attribute a dispute to a company we have not matched.
Characterise a company's conduct in our own voice on the strength of member-reported data alone.
Take money from a company that appears in a ranking. See /who-pays-us.