All work

Multi-provider AI credits system

Turning heterogeneous AI consumption — tokens, pages, seconds — into a single internal currency the business can meter, price and resell.

Role
Software Engineer — system design & build
Period
2024 — Present
Company
Digital marketing agency (confidential)

5

providers with incompatible usage units unified

1

internal currency for all AI consumption

Capped

per-user consumption — no unlimited burn

#Context

As the agency's platform added AI features — chatbots, transcription, scraping, document parsing, video analysis — each capability billed differently: OpenAI charges per token, Firecrawl per scraped page, Unstructured per page/size, Whisper per transcription, Vertex per video second. The business wanted to sell 'AI usage' to clients with a margin, but there was no way to even account for what a user consumed, let alone price it.

#Problem & constraints

  • Every provider reports consumption in incompatible units — tokens, pages, seconds, documents — with different pricing tiers that change over time.
  • Without metering, a single heavy user could burn unbounded provider spend — unlimited consumption was the default.
  • The resale price needed a configurable margin over real provider cost, auditable per operation.

#Architecture

AI operationchat · scrape · transcribeRedisbalance pre-checkMetering gatewayusage → creditsRate tablescost + marginLedgerappend-onlyPostgreSQLbalance settle
Every AI operation is metered, converted to credits and settled against the user balance.

Every AI operation in the platform passes through a metering gateway: the response's usage payload (tokens, pages, seconds — whatever the provider reports) is normalized by a per-provider cost table into credits, the platform's single internal currency. Each conversion applies a configurable margin multiplier, so one credit always maps to a known resale price. Credits are debited atomically per operation against the user's balance in PostgreSQL (with Redis for fast balance checks at request time), and every debit writes a ledger entry — provider, raw usage, rate applied, margin — making consumption auditable end to end. When a balance hits zero, AI features stop gracefully instead of generating unbounded provider bills.

platform.internal/credits
balance4,850 cr
opproviderusagecr
chat.completionOpenAI1,204 tok-12
scrape.pageFirecrawl3 pages-6
transcribeWhisper94 sec-4
doc.parseUnstructured12 pages-9

* Interface recreated with synthetic data — the production system holds client-confidential data.

#Key decisions & tradeoffs

01One internal currency over per-provider pricing

Decision
Normalize everything into credits at the metering boundary; the rest of the platform only ever sees credits.
Alternatives
Track usage per provider and price each feature separately (token-based chat pricing, page-based scraping pricing, etc.).
Why
Product, sales and users think in one unit: 'you have N credits'. Per-provider pricing would leak infrastructure detail into every pricing conversation and make bundles impossible.
Cost of being wrong
The cost table is a liability: when a provider changes pricing, credits must be re-rated or the margin silently erodes.

02Append-only ledger for every debit

Decision
Every credit movement writes an immutable ledger row with the raw provider usage and the rate applied.
Alternatives
Just decrement a balance column and keep aggregates.
Why
Money-adjacent systems get disputed. When a client asks 'why did I consume 4,000 credits on Tuesday?', the ledger answers with per-operation detail — and it makes margin reporting a query, not a project.
Cost of being wrong
Storage grows forever and writes double (balance + ledger) — acceptable for a system whose entire value is accountability.

03Pre-flight balance checks with post-hoc settlement

Decision
Check balance (Redis) before allowing an operation; debit the true cost (PostgreSQL) after the provider responds.
Alternatives
Estimate-and-pre-debit before the call, or fully synchronous single-source debiting.
Why
True usage is only known after the provider responds (a chat answer's token count, a document's page count). Pre-checks stop insolvent users instantly; post-settlement charges the exact amount — no refunds, no overcharging.
Cost of being wrong
A balance can briefly go negative between check and settlement under concurrency — bounded by per-request cost caps.

#Results

Auditable

every credit traceable to raw provider usage

Margin

controlled resale margin on all AI features

No surprises

unlimited consumption structurally impossible

#What I'd do differently

I'd version the rate table from day one — when a provider re-priced, historical ledger entries stayed correct but 'current cost' comparisons got confusing until we pinned rates to time ranges. I'd also add per-operation cost alerts earlier; the first anomalous bill was discovered in a monthly review, not by a monitor.

#Stack

Node.jsTypeScriptPostgreSQLRedisOpenAIFirecrawlUnstructuredWhisperGoogle Vertex