An AI that drafts tax returns without hallucinating the maths

Tax is a strange domain for a language model. The law is deterministic: given the numbers, the answer is not a matter of opinion, and a cent out is a wrong return, not a rounding quirk. But the inputs are the opposite of deterministic. They arrive as a photographed receipt, a bank CSV with a bespoke column order, a PDF payslip that scanned badly. So the shape of the problem is clear before you write a line: use the model for the fuzzy part, the reading and the judgement, and never, ever let it do the arithmetic.

Capitalogic drafts Australian personal tax returns, and that separation is its whole architecture.

The maths lives outside the model

The tax engine is deterministic Python that never sees a token of model output as a number it trusts. Every monetary value is a Decimal, not a float, because floating point cannot represent money and I am not going to explain to anyone why their refund is off by a cent that came from binary rounding. Brackets, offsets, the Medicare levy, the thresholds: all encoded as rules, all tested. The model can decide that a line item is a work-related deduction. It does not get to decide what that does to taxable income. That is a function call with a known answer.

This is the part people skip when they wire an LLM into anything financial. They let the model produce the final figure, and it produces one that looks right, which is worse than one that looks wrong. Keeping a hard wall between the probabilistic reader and the deterministic calculator is the single decision the rest of the system is built to protect.

An agent, not a prompt

The reading side is a genuine agentic loop, not a single call. The model works through a return the way a person would: pull the documents, extract what each one is, ask what is still missing, decide which rule applies, and call a tool when it needs a fact or a calculation. It has real tools, and the loop continues until the return is complete rather than stopping at the edge of one context window.

Document extraction is the case I am most careful about. Rather than asking the model to describe a document in prose and parsing that back, I use tool-use as structured output: the model calls a tool whose schema is the shape I need, and the arguments come back typed and validated. A receipt becomes a typed line item or it fails the schema and gets flagged, which means the failure is visible instead of a plausible sentence hiding a wrong number.

Retrieval that respects the source of truth

Tax questions need the actual ATO position, not the model’s memory of it, so there is a retrieval layer over an ATO corpus. It is hybrid on purpose. BM25 handles the exact statutory phrasing, the section numbers and defined terms where a dense vector will happily return something semantically close and legally wrong. pgvector handles the neighbourhood, the cases where the user’s wording and the ruling’s wording share no keywords. Dense-only retrieval quietly drops precision exactly where a compliance domain cannot afford it, so the lexical index earns its place.

Some answers depend on current published figures, so the agent can fetch live. That is also the most dangerous tool in the system, because a fetch tool pointed at a URL is a server-side request forgery waiting to happen. So it sits behind an SSRF guard: the target is validated before the request goes out, internal ranges and metadata endpoints are refused, and the model does not get to talk the guard out of it. Giving an agent a live fetch without that is how you turn a helpful feature into an internal-network scanner.

Why I built it this way

None of this is exotic. It is the same instinct the day job runs on, applied to a personal project: work out where the system must be deterministic and defend that boundary, use the probabilistic parts only where fuzziness is the actual requirement, and treat every tool the agent holds as attack surface. Compliance-grade rigour and security-by-design are not heavier for a tax agent than for a bank migration. They are the same discipline at a different scale, which is rather the point of building things at night. The muscle stays warm.