Architecture

How Veri is built

An identity-verification service for Pakistan, built on open-source vision models rather than a document vendor, and shaped end to end by one constraint: document images are never kept. This is the request path, the infrastructure it runs on, and the reasons behind both.

← API guide AWS infrastructure One document, end to end

The whole picture

Every box is a resource the Terraform declares; every arrow a call that happens. One account, one region: ap-southeast-1, the region with the GPU quota.

Compute
Networking
Storage
Database
Machine learning
Messaging
Security
Observability

Read it left to right. The applicant’s browser loads the app from Amplify and talks to the API through the load balancer — the only thing in the VPC with a public address. Everything that touches a document runs in the private subnets: the API, the worker, the database and the model endpoint. The services on the right are AWS-managed and sit outside the VPC, reached over private endpoints rather than the internet. And the dashed green line along the top is the one that matters: the document itself goes from the browser straight into S3, and the API never sees its bytes.

The shape of it

Four moving parts. The applicant's browser talks to two of them, and the file itself only ever touches one.

The request path

The app

Your app, or our web flow

API

Django on Fargate

Queue

SQS

Worker

Reads documents

Models

vLLM on one GPU

The document itself never takes this path: it goes from the browser straight into storage.

The file takes a different route from everything else. The app asks the API for a presigned upload link and puts the bytes straight into S3, so a document image never passes through the API tier at all. The worker pulls it from S3, reads it in memory, writes the record, and deletes the image.

Why not one server that does everything?

Reading a document takes seconds and needs a GPU; answering “how is my verification going?” takes milliseconds and must never be blocked behind it. Splitting them across a queue is what lets the API stay responsive on a 3G connection while a GPU works through a backlog — and what lets the GPU be switched off entirely without taking the product down.

One document, end to end

What happens between a user tapping the shutter and a compliance officer seeing a result.

`1`	The app asks for an upload link	API 2. The API presigns a PUT into the ingest bucket, valid 15 minutes, and records a pending document.
`2`	The browser uploads the file	API 3, direct to S3. The bytes never reach the API tier.
`3`	The app confirms	API 4. The API checks the object arrived, then puts a job id on the queue — never the image, never a URL.
`4`	The worker claims the job	Each worker task runs four loops, each taking one message at a time, with a visibility timeout long enough for a cold GPU.
`5`	Quality gates, before any GPU call	Blur, glare, darkness, crop, and for a CNIC or passport whether this is a photo of the original rather than a scan, a photocopy or a screenshot. A bill or statement may be a scan or a copy. A selfie must show one face, close enough to see; its sharpness is judged on the face, not the wall behind it. A bad photo is answered with what to fix, with no model called.
`6`	The model says what it is	Before anything is read, the vision model is asked what the upload shows and how it was made — without being told what the user chose. Rules send back the wrong document, a copy of an identity document, a screenshot, a card cut off, a statement with pages missing, or a selfie that is not one real person's whole face.
`7`	The models read it	An OCR pass and a vision-language pass, on one endpoint. The document type decides the prompt and the schema. A selfie is not read.
`8`	Deterministic checks run	Document type, CNIC structure, passport MRZ check digits, address match, statement arithmetic, recency. Code decides, never the model.
`9`	The record is committed	Extracted fields, confidences, model and prompt versions, check results — the CDD record.
`10`	Only then, the image is deleted	In that order. Deleting first and failing the write would lose the document and make the applicant start again. An upload that was sent back is deleted too.

Inside the worker, per document

Fetch

S3 → memory

Gates

No GPU yet

Inspect

What is it?

Read

OCR + VLM

Check

Rules, not models

Commit

CDD record

Delete

The image goes

The order is load-bearing: the record is committed before the image is deleted, never the other way round.

The four invariants

Everything above follows from these. Breaking one is a compliance incident, not a bug.

Invariant	How it is held
Raw images live only in the ingest bucket	Presigned uploads keep them out of the API tier; the worker holds them in memory and writes nothing to disk. They are never in the database, a backup, a log or a queue message.
Nothing read from a document is ever logged	Logging uses an explicit allowlist of field names. Errors name fields and codes, never values — the same rule the API's 400s follow.
Images are deleted once the record is committed	In that order, on every path — including a document that failed its quality gate, so failure cannot become a retention loophole. A one-day bucket lifecycle rule is the backstop if the worker ever fails to delete.
The model never decides	It extracts and explains. Deterministic rules and humans decide. Every check records which policy version judged it, so a decision can be defended under the rules of the day.

Two things deliberately absent

The ingest bucket has no object lock, because a lock would prevent the deletion the whole design turns on. And SageMaker Data Capture is off — it writes raw inference inputs to S3, which would silently recreate the document store we just deleted. Both absences are load-bearing.

How a document is read

Two models on one endpoint, and a routing rule that is about failure modes rather than confidence scores.

Model	Role
`PaddleOCR-VL 0.9B`	Transcribes the page. An OCR model driven by task prompts, not an instruction follower: asking it for JSON does not work.
`Qwen3-VL-8B-Instruct-FP8`	Structures the fields from the image, grounded on that transcript so digits are read rather than guessed. Constrained decoding holds it to the schema for that document type.

Routing is on the failure mode, not a single confidence number. A blurry photo is a retake, not a second model: sending it to a general vision model turns an honest low-confidence answer into a confident wrong one. Where the pixels are fine but the fields are ambiguous, the reasoning model gets another pass; where a schema comes back invalid, it is retried and then sent to review.

Two documents are read twice, on purpose

The back of a CNIC prints its address only in Urdu, and a passport carries 88 characters of machine-readable code. Asked to read and structure those in one constrained answer, the model returned empty fields. Both are now transcribed in a pass of their own before the fields pass — and for the passport, the reading whose ICAO check digits verify is the one kept.

What decides the outcome

Code, and then a person. The models are evidence.

Each check is recorded as its own row with a result, a score, an explanation and the policy version — never blended into a single number. A blended score is useless to debug and misleading to a reviewer. Checks that could not run are recorded as UNAVAILABLE rather than omitted, so an absent authoritative check never reads as a pass.

Two ceilings are deliberate. A CNIC number has no check digit at all, so a well-formed one passes only “with conditions”. A passport’s check digits are real arithmetic, but the algorithm is public — passing proves the zone was read correctly, not that the passport was issued. Neither is identity; an authority check (NADRA Verisys) is, and it is shown as unavailable until one is connected.

What runs where

One AWS account, one region — ap-southeast-1 (Singapore), the region with the GPU quota. All of it is Terraform in infra/terraform.

Piece	AWS	Notes
Frontend	Amplify Hosting	The applicant flow and these docs, exported as static files. Built from `main`.
API	ECS Fargate behind an ALB	Django REST, 1 vCPU / 2 GB per task, autoscaled. HTTPS only, with the certificate on the load balancer.
Worker	ECS Fargate	The same image, run as a queue consumer. 2 vCPU, and core dumps disabled: a core dump is a memory image, and this process holds document bytes.
Inference	SageMaker endpoint	One `ml.g6e.xlarge` (an L40S, 48 GB), running both models under vLLM in our own container.
Database	RDS PostgreSQL	`db.t4g.medium`, encrypted, private subnets, automated backups. Holds CDD records, checks and the audit trail — never a document image.
Queues	SQS ×3	Processing, a dead-letter queue kept for 14 days, and a reconciliation queue that sweeps for documents that should have been deleted.
Images	ECR	API, worker and inference images, tagged by commit.

Where data lives

Two buckets with opposite lifetimes, and that is the point.

Store	Holds	Lifetime
Ingest bucket	Document images, in flight	Deleted by the worker in seconds. A one-day lifecycle rule expires anything it missed. Versioning off, public access blocked, encrypted with its own key.
Audit bucket	The evidence trail	Object-locked — `GOVERNANCE` in dev, `COMPLIANCE` in production — so it cannot be rewritten.
PostgreSQL	CDD records, checks, corrections, the mobile number	Five years from the end of the relationship, not from onboarding — expiry is driven by a closure event.

What survives a document is deliberately narrow: the extracted fields, a SHA-256 of the bytes that proves which file was seen, and a perceptual hash that lets the same document be recognised across cases — spotting a reused document without keeping it.

Network and keys

A VPC with public and private subnets across two availability zones. The API sits behind the load balancer; the worker, the database and the model endpoint have no public address. Traffic to ECR, KMS, SageMaker and the rest goes over interface endpoints rather than the internet, and S3 over a gateway endpoint.

Three KMS keys, split by lifetime rather than for the sake of it: documents (ephemeral), audit (five years) and logs. Separate keys mean the document key can be rotated or revoked without touching the record that must be kept.

The GPU, and its bill

One accelerator instance is the single largest cost, so it is treated as something that can be switched off.

The endpoint can be deleted and recreated on demand — nothing is lost, because the model weights live in S3 and the container in ECR. Recreating it takes about 10–15 minutes, which is why the worker returns a document to the queue rather than failing it when the endpoint is not running, and why the API tells the app ai.state: "starting" with an estimate instead of an error.

The same admin control can run it always-on, on a schedule, on demand, or off. An applicant never sees a broken flow: the quality gates and the checks that need no GPU still run, so a blurry photo is still answered immediately.

Deploys

Push to main: GitHub Actions assumes a role through OIDC — no long-lived AWS keys in the repo — builds the API, worker and inference images, pushes them to ECR, registers new task definitions, runs database migrations as a one-off task, and rolls the services. Amplify rebuilds the frontend from the same commit. Infrastructure changes go through Terraform separately, so a deploy cannot quietly reshape the account.

Data residency

Why the inference layer is a plain container rather than a managed model API.

Pakistan’s SBP mandates cloud localisation for core material workloads for regulated fintechs, and Bangladesh’s BFIU requires customer data to stay in-country. Neither country has an AWS region today. That does not change the build, but it decides one thing: the models run in our own vLLM container, with object storage behind an interface, so SageMaker is a hosting choice that can be reversed rather than an architecture. The same stack redeploys to an in-country data centre — which is also why the models are small enough to run on a single GPU.

The short version

Small open models, run by us, on one GPU, with the images deleted as soon as the record is written. That is the whole design; everything else is a consequence of it.