MATTHEWLAB
AI workflows

Your token count isn’t your AI bill.

Before comparing AI usage, check what the report counts, where the day begins, and which records are missing.

Paper tokens and a ledger illustrating the scope of a usage report.

A token report looks precise. Rows line up, totals add up, and a larger number seems to imply a larger bill. But precision inside a table does not tell you whether the table describes the question you are trying to answer.

Start with a smaller question: what, exactly, was counted?

Give the report a boundary

Imagine working on the same project from two computers. A report built from one machine’s local logs is a report about that machine. Calling it “total project usage” silently changes its meaning. An empty row can mean no activity, but it can also mean missing or unreadable records.

A useful report header names the project, included devices, start and end dates, time zone, and last available record. Preserve the original timestamps so that a different reporting boundary can be applied later.

Count responses, not repeated lines

Consider a synthetic log with three entries: response A, a repeated entry for response A, and response B. There are three lines and two distinct responses. The right deduplication key depends on the log format; similar text alone is not a reliable identity.

Keep unreadable or excluded records visible in the summary. A correctly summed subset should not be described as complete.

Keep the unit attached

The cc-token-ledger README, for example, describes a weighted relative measure and explicitly distinguishes it from money and subscription limits. That distinction belongs beside the number, not in a footnote the reader has to discover.

A comparison can be useful without being a bill. Apply the same weights, scope, and time boundary to both sides. If the calculation changes, record that change before interpreting the difference.

Pair usage with an actual outcome

A smaller total can come from unfinished work. A larger total can include extra verification. Neither result, by itself, proves that a workflow became more efficient.

Next to the usage total, record the task that was actually completed and the checks that were actually run. If those are unknown, leave the conclusion open. The practical improvement is often a clearer label before it is a smaller number.

한국어 글: 토큰 사용량 표를 보고, 뭘 알 수 있을까