3 interfaces, 1 shared engine — hosted page, local app, and CLI all produce identical output
Now a real backend API: SQLite job storage, a fair multi-user queue, millisecond re-scoring on review
22 data-validation checks, built into the JSON output, not just a UI badge
109 automated tests, offline, run on every GitHub push — 0 API key needed
A broken balance check forces REFER regardless of score — never a silent APPROVE
A full product blueprint & MVP PRD — market research, personas, roadmap, 5 differentiating bets
Tools used: Node.js 18, pdf.js (digital-PDF text layer), Tesseract.js (OCR for scans/images), xlsx (Excel ingestion), SQLite (job/result storage), the Claude API via forced tool use for extraction and classification. Three shells — a hosted claude.ai Artifact (rules-only, since a static page can't hold an API key), a local Node backend (LLM-enabled, now with a REST API), and a CLI — all run the same pipeline, so the JSON a CLI run produces matches what the UI and the API show.
Executive summary
LedgerLens reads a raw bank statement and returns a 0–1000 credit-risk score, a decision, and the reasons behind it — automating the three things a credit officer does by hand (read the layout, judge what each transaction really is, catch tampering) without losing auditability. Rules classify first; an LLM only fills what rules can't; every model output is forced through the same balance check as the statement itself. A real multi-user backend (SQLite, job queue, instant re-scoring on review) and 109 automated tests are already built. Status: a tested proof of concept — the scoring model is not yet calibrated against real loan outcomes.
The Pipeline
| Stage | What it does |
|---|---|
| 1. Convert | pdf.js text layer for digital PDFs; Tesseract OCR for scans/images; Excel sheets → CSV; password PDFs asked once, never stored |
| 2. Validate order | Checks "Page X of Y" footers per file, or date continuity if there are none — stops cleanly on jumbled or missing pages instead of silently scoring a broken sequence |
| 3. Batch extraction | 3 pages per batch; with the LLM on, one model call per batch returns structured rows, cross-checked against the rule parser and the running balance before being trusted |
| 4. Enrich | Counterparty parsing (mandatory — never blank), channel, IFSC→bank, merchant aliasing, foreign-currency fields — built from real Indian bank narration formats (HDFC, ICICI, SBI, Axis, Kotak, UPI/NEFT/RTGS/IMPS/NACH) |
| 5. Classify | A 37-category taxonomy, resolved in order: user rules → structural rules → narration keywords → LLM for leftovers → OTHER, with a confidence and method stored on every row |
| 6. Score | 6 weighted components → a composite 0–1000 score, rating band, and decision, run against 22 data-validation checks |
Backend API — Built for a Team, Not One Analyst
- Persistent. Jobs are rows in SQLite, not in-memory state — a server restart loses nothing.
- Isolated. Each job runs in its own worker process, so one stuck OCR page can't block anyone else.
- Fair. The next job always comes from the user with the fewest jobs already running, not whoever queued first.
- Instant correction. A reviewer's fix re-scores from a saved snapshot in milliseconds — no re-read, no OCR, no model call — with every prior version and a full audit trail retained.
Positioning Against a CIBIL Score
- A CIBIL score shows whether someone has repaid credit before. LedgerLens shows whether they can afford a new loan today.
- CIBIL can't see a salary that stopped last month; a statement can't see a loan already repaid elsewhere — lenders need both, and the app says so rather than positioning itself as a bureau-score replacement.
- An in-app Knowledge Sharing tab explains the LLM's role, the scoring math (worked through on a real sample), and a 21-item FAQ for non-specialists.
Key Design Decisions
- Rules first, model second, human last. Deterministic rules handle what's structured and auditable; the model only fills what rules can't; a human only sees what can change the decision.
- Integrity over coverage. Balance arithmetic is checked on every row. A broken balance forces REFER regardless of score — never a silent APPROVE.
- The model proposes, code verifies. Every LLM row must pass the same balance check as the statement; a failing batch retries once, then falls back to rules.
- Not every credit is income. Only SALARY, BUSINESS_INCOME, INTEREST, and RENTAL_INCOME count — P2P, refunds, loan disbursals, and own-account transfers are excluded.
- Decision = band + hard overrides. No verifiable income or FOIR above 65% forces DECLINE; tampering, structuring, or circular flows force REFER — every override's reason is listed.
- Correction ≠ re-run. A reviewer's fix re-scores from a stored snapshot, not the raw file — fast, and every prior version stays readable.
Proof the Safety Net Works
| Finding | OCR misread ₹1,532.40 as ₹1,632.40 on a scanned test statement. |
| Result | The balance check caught it. Decision forced to REFER (906) instead of a wrong APPROVE. |
| Why it matters | The architecture's core guarantee held under a real failure, not just in a spec. |
Testing Discipline
- 109 automated tests, 0 API key required, run on every push.
- A deliberately-wrong mock model (every balance off by ₹1,000) proves the fallback-to-rules path actually engages.
- 18 tests exercise a live backend: a real SQLite database, 2 live worker processes, multi-user isolation, and fair scheduling.
- 13 example statements, each checked against its own documented expected result.
Quick recall
| Section | Summary |
|---|---|
| Problem | A credit officer reviewing statements by hand has to read inconsistent bank layouts correctly, judge what each transaction really is, and catch tampering/structuring — all without a system that's actually auditable for a lending decision. |
| Research steps | Chose rules-first classification specifically because Indian bank narrations are semi-structured enough to make it viable → designed the LLM's extraction output to be forced through the same balance check as the statement itself, so a model error can never silently pass → built 13 example statements and 109 offline tests, including a deliberately-wrong mock model and a real backend run against a temporary database, to prove the guardrails actually engage rather than just existing on paper → caught a real OCR digit misread during testing and confirmed the safety net correctly forced REFER instead of a wrong APPROVE → wrote a full product blueprint and MVP PRD (market research, personas, roadmap, 5 differentiating bets) once the engine itself was proven. |
| Solution | A 6-stage pipeline (convert → validate order → extract → enrich → classify → score) now backed by a real multi-user API — SQLite job storage, a fair queue, millisecond re-scoring on review, and a full audit trail — sharing one engine across a hosted demo, a local LLM-enabled app, and a CLI. |
| Conclusion | The architecture, guardrails, and now the multi-user backend are real and tested end to end — but honestly, LLM accuracy still hasn't been measured on real (non-synthetic) statements, and the scoring weights remain illustrative, not calibrated against historical loan outcomes. A proof of concept that's grown real infrastructure, not yet a production credit model. |
Which feature is used at which step
| Feature | Used at |
|---|---|
| Page-order validation | Right after conversion, before extraction even starts — stops a jumbled or missing-page statement with a clear error instead of scoring a broken sequence |
| Running-balance cross-check | Inside batch extraction — every LLM-returned row has to agree with the statement's own running balance, or the batch retries once then falls back to the rule parser |
| Mandatory counterparty | Enrichment — a row with no identifiable counterparty is never left blank; it falls back to UNIDENTIFIED and is always routed to the Review queue |
| Confidence + method per row | Classification — every row stores whether RULE, LLM, or MANUAL produced its category, and at what confidence, so a reviewer can see how a label was produced, not just what it is |
| Smart review triage | Before scoring — only high-impact flagged rows (large amounts, recurring payments, loan-related, possible bounces, no counterparty) reach a human; low-impact ones auto-accept |
| Hard policy overrides | Last, after the composite score — tampering, structuring, no verifiable income, or FOIR above 65% can force REFER/DECLINE regardless of how high the score is |
| Review-decision re-scoring | After a reviewer acts in the queue — re-scores from a saved snapshot in milliseconds, saves a new result version, and logs the change to the audit trail, all without touching the original file again |
| Fair job scheduling | In the backend queue, when multiple users submit jobs at once — the next job always comes from whoever has the fewest jobs already running, so one heavy user can't starve everyone else |
