Case Studies — full list

// published as each is genuinely scoped

A curated subset of the fully built flagships also lives on the main portfolio page; this page has the complete list. T5 (Flight Delay Prediction Dashboard) and T8 (Booking Funnel Analysis) are written to the same depth. T28 (Weft) and T29 (Career Copilot) are real, shipped products pulled in from their own GitHub repos, not travel-tech scope docs. T1–T17 are travel tech; T18–T27 are cross-industry practice categories: feature reverse engineering, user psychology, metrics, strategy, and more. The rest publishes here as each is genuinely scoped and written up, not before.

Product Teardown

// reverse-engineering how existing products work, end to end
T1

Flight Booking Product Teardown

View the files on GitHub →

CIRCLES method HEART framework (Google) OKR-aligned recommendation RICE + MoSCoW Regulatory & financial-filing research

Frameworks used: CIRCLES, RICE, MoSCoW, HEART (Google), OKRs — written analysis, no build.

A note on evidence: as an outside analyst I don't have MakeMyTrip's internal funnel or A/B data, so this teardown doesn't invent conversion percentages to sound precise. Every claim about the booking flow below is sourced to a public regulator (CCPA, CCI), an enforcement action against a named company, or MakeMyTrip's own financial filings — linked at the bottom of each section. Where something is an allegation rather than a proven fact, it's labelled as one.

C — Comprehend the Situation

The clarifying questions I'd ask before touching the product: which platform (web vs. app: both), domestic or international (domestic only), and an existing product or a new one (existing, mature product). Scope: MakeMyTrip's flight search → booking flow specifically, narrowed further to the fee/pricing-transparency question, because that's the one part of the flow with real, public evidence behind it — a regulator, an enforcement precedent, and the company's own disclosed accounting, not a guess at a funnel I can't see.

Business model, from MakeMyTrip's own FY25 filing: the flight vertical monetizes through airline commissions/overrides, a convenience fee layered on the base fare (₹249–499 on MakeMyTrip vs. ₹0–399 among lower-cost competitors), and ancillary add-ons (seat selection, insurance, baggage). MakeMyTrip discloses $301.6 million in "customer inducement costs" for FY25 — discounts, cashbacks, and loyalty spend — booked as a reduction of revenue, not a separate expense line. That accounting choice matters for this teardown specifically: it means the fee/add-on mechanics below aren't incidental UX decisions, they're revenue mechanisms with a disclosed, nine-figure scale. Removing them has a real, quantifiable cost — see Risks & Trade-offs below.

The regulatory backdrop, which is what actually makes this teardown tractable: India's Central Consumer Protection Authority (CCPA) published Guidelines for Prevention and Regulation of Dark Patterns, 2023, naming 13 specific patterns — including "drip pricing" (revealing charges late in the flow) and forced/pre-selected add-ons — as unfair trade practices. In 2026, SpiceJet was fined under this exact framework for practices in the same category this teardown examines: a real, recent enforcement precedent in Indian travel booking, not a hypothetical risk. Separately, India's Competition Commission (CCI) fined MakeMyTrip and Goibibo ₹223.48 crore in 2022 for hotel price-parity abuse — a different specific practice, but the same underlying pattern: pricing mechanics that regulators have already acted on, in this company's own history.

Sources: MakeMyTrip FY25 20-F (SEC) · CCPA Dark Patterns Guidelines, 2023 · CCI price-parity order, 2022 (₹223.48 Cr)

I — Identify the Customer

Three illustrative personas, built to span the segmentation axes that predict booking behavior — price sensitivity, time pressure, and party complexity — not measured MakeMyTrip user data:

  • "Value Rakesh" (28, IT professional, 4–5 domestic trips/yr): price-sensitive, compares multiple apps before booking on the final price paid, not the headline price.
  • "Business Priya" (34, consulting manager): books last-minute, values reliability and rebooking flexibility over price.
  • "Family Planner Anita" (41): books for 4+ pax, needs group-seat clarity and is most exposed to an unclear total-fare breakdown across multiple travellers.

R — Report the Customer's Needs

Journey: Home search widget → results list (filters/sort) → fare review (baggage/meal add-ons) → traveller details → payment → confirmation + "My Trips." Logged-out users are funnelled into account creation at payment, not upfront; that reduces search friction but adds a late-stage form-fill tax.

Retention loop: "My Trips" functions as a lifecycle retention engine inside a post-booking utility tab — it's why a user reopens the app after booking at all (rebooking, cancellation, web check-in), feeding the same repeat-usage loop the company's own loyalty program (MMTBLACK, restructured in 2024) is built to reinforce.

Needs uncovered, mapped to the journey above and grounded in the regulatory evidence, not a modeled funnel:

  • Trust/transparency need: "drip pricing" — the base fare shown at search differing from the total revealed at fare review — is one of the 13 patterns CCPA explicitly names as unfair, and is the same category of practice SpiceJet was fined for. Pre-selected, opt-out add-ons (insurance, seat) sit in the same named category.
  • Time-efficiency need: filter fatigue (many dimensions, no smart default) with no persistent price comparison across dates forces users to manually reopen search in a new tab — a UX observation, not a regulatory one.
  • Clarity/control need: baggage policy buried a tap deep is a plausible driver of post-booking support contact, consistent with (though not proof of) the "black-box numbers" pattern flagged in the Verification Gap research (T1) elsewhere on this site.

Funnel: Search → Results → Fare Review → Traveller Info → Payment → Confirmation. I can't see MakeMyTrip's actual conversion numbers at each stage — no outside analyst can — so this teardown doesn't invent one. What's verifiable instead: the specific mechanic named by CCPA (late fee reveal between Results and Fare Review) is the exact shape of pattern the regulator and the SpiceJet enforcement precedent describe, which is reason enough to treat that transition as the highest-risk point in the flow without needing a fabricated percentage to justify it.

Journey map from Search to Confirmation, annotated with pain points, persona impact, and which recommendation targets each stage

Click to open the full interactive Journey Map (also available in dark mode). Note: the funnel percentages in the linked map are an illustrative RICE-scoring exercise, not measured MakeMyTrip data — see the methodology note above.

Show more: Cut, List Solutions, RICE scoring, Risks, Summarize, HEART metrics, OKRs (the full CIRCLES back half)

Competitive and regulatory context: the 2022 CCI order named MakeMyTrip and Goibibo together, and a separate CCI probe (2019) examined MakeMyTrip and OYO on related unfair-practice complaints — meaning this isn't a single-company problem, it's a pattern regulators are watching across the category. A March 2026 short-seller report (Morpheus Research) made further allegations — including that MakeMyTrip continued price-parity practices after the 2022 order, and used "mandatory" insurance opt-outs and late-revealed convenience fees. These are allegations from a party with a financial interest in the stock falling, not independently verified facts, and I'm treating them accordingly — worth naming because they're part of the public record, not because they're proven.

C — Cut Through Prioritization

Of the three personas, I'd prioritize Value Rakesh's need first. Price-sensitive comparison shoppers are both a large segment of OTA search traffic and the segment most exposed to a late fee reveal, since they've already anchored on the headline price shown at search. Business Priya's reliability need and Family Planner Anita's group-booking complexity are real, but lower-frequency, worth a later cycle rather than this one. Everything below is scoped to fixing Rakesh's need first.

L — List Solutions

  1. All-in fare display at the results level: address the "drip pricing" pattern CCPA specifically names, before a regulator or a repeat of the SpiceJet outcome forces it.
  2. Opt-in (not opt-out) add-ons at fare review — directly responsive to CCPA's forced-add-on pattern.
  3. Persistent "compare dates" strip on results instead of reopening search.
  4. Baggage/meal policy inline on the fare card, not a tap away.
  5. One-tap "hold my seat 15 min" for indecisive high-intent users.
  6. Smart filter defaults ranked by price + duration + layover, not price alone.
  7. Payment retry with saved card + biometric re-auth to reduce late-stage abandonment.
  8. Proactive delay-risk badge on results (feeds the Flight Delay Dashboard (T5)).
  9. Group-booking mode for 4+ pax with a shared itinerary link.
  10. Post-booking "manage trip" nudges (web check-in, gate-change alerts) via push.

E — Evaluate Trade-offs

RICE-scored, ranked by priority. Reach, Impact, and Confidence here are my own analyst judgment for illustrating prioritization method, not measured MakeMyTrip metrics — the two regulatory-motivated items are ranked P1 regardless of the exact score, because they're the two the CCPA framework and the SpiceJet precedent most directly bear on:

Feature (RICE-scored, illustrative)Reach × Impact × ConfidenceEffortRICE / priority
Opt-in add-ons at fare review8 × 2 × 0.9114.4 · P1 / Must
All-in fare display at results10 × 3 × 0.9213.5 · P1 / Must
Baggage/meal policy inline9 × 1.5 × 0.8110.8 · P1 / Must
Delay-risk badge on results10 × 2 × 0.634.0 · P2 / Should
Payment retry + biometric re-auth5 × 3 × 0.733.5 · P2 / Should
Persistent compare-dates strip6 × 2 × 0.732.8 · P3 / Could

MoSCoW: Won't-have this cycle: the full loyalty-program overhaul (separate project, see Travel Loyalty Program Redesign below); it's a retention lever, not a fee-transparency fix.

Risks & Trade-offs

Every recommendation above has a real cost. Naming it here, before launch, not after — and grounding the scale in MakeMyTrip's own disclosed numbers, not a guess:

  • All-in fare display: the honest total can look higher than a competitor's headline price at the results stage, hurting click-through from price-comparison meta-search and paid search ads. The fix that builds trust downstream may cost impressions upstream.
  • Opt-in add-ons: directly reduces ancillary revenue per booking. MakeMyTrip's own FY25 filing shows $301.6M in customer-inducement-related revenue reduction already — a real, disclosed number that shows this isn't a marginal trade-off; it needs a revenue-per-booking floor agreed with Finance before shipping, not just a UX sign-off.
  • Baggage/meal policy inline: minor engineering cost, no material revenue risk. The lowest-risk item on the list, consistent with its RICE ranking.
  • Delay-risk badge: risks eroding trust in a different direction if the underlying prediction (see the Flight Delay Dashboard, T5) isn't well-calibrated. A wrong badge is worse than no badge.

Mitigation: ship all-in pricing and opt-in add-ons behind an A/B test, with revenue-per-booking as a guardrail metric alongside conversion, not conversion alone. A win on Fare Review conversion that quietly craters ancillary revenue is a loss dressed up as a win — and given the CCPA/SpiceJet precedent, shipping this proactively is materially cheaper than being the next enforcement case.

S — Summarize Recommendation

Ship all-in fare display and opt-in add-ons first: RICE 13.5 and 14.4, the two highest-scored and lowest-effort items on the list, and the two most directly responsive to the CCPA dark-patterns framework. Everything else on this list sequences behind those two.

Success Metrics: the HEART Framework

Using Google's own HEART framework (Rodden et al.) instead of a single catch-all "engagement" number:

  • Happiness: CSAT specifically on "pricing clarity" (post-booking survey); NPS delta before/after the all-in pricing rollout.
  • Engagement: searches per session; % of sessions using the new compare-dates strip.
  • Adoption: % of eligible searchers who see and interact with the all-in fare display / opt-in add-on toggle in month one.
  • Retention: repeat booking rate within 90 days, segmented by whether a user's first booking hit the old opt-out add-on flow or the new opt-in one.
  • Task success: Results→Fare Review conversion (the primary metric this cycle targets — MakeMyTrip would measure this against its own internal baseline, which isn't externally visible), search-to-confirmation completion rate, time-to-book.

Tied to an OKR

Objective: Bring fare-review pricing in line with the CCPA dark-patterns framework, ahead of enforcement rather than in response to it.

  • KR1: Establish an internal Results→Fare Review conversion baseline (not something an outside analyst can know) and target measurable lift after removing the late fee reveal.
  • KR2: Reduce "hidden fees" CSAT verbatims by 50%.
  • KR3: Ship both P1/Must recommendations this quarter, validated against an A/B test rather than shipped on judgment alone.

Sources: Morpheus Research report (allegations, not verified fact) · BusinessToday coverage · CCI probe, MakeMyTrip & OYO, 2019

Quick recall

SectionSummary
ProblemMakeMyTrip's flight search → booking flow has a fee/pricing-transparency gap with real, public evidence behind it — not a guessed-at funnel: CCPA's 2023 dark-patterns framework, a 2026 SpiceJet enforcement precedent in the same category, and MakeMyTrip's own ₹223.48 Cr 2022 CCI fine for a related pricing practice.
Research stepsScoped to domestic flight booking and grounded the business model in MakeMyTrip's own FY25 filing ($301.6M "customer inducement costs") → built 3 personas around price sensitivity, time pressure, and party complexity → mapped the journey and named drip-pricing as the highest-risk transition, sourced to CCPA's named pattern rather than an invented conversion number → prioritized Value Rakesh's need first → listed 10 candidate solutions → RICE-scored and MoSCoW'd them, flagging which scores are illustrative judgment vs. which priority calls are regulator-driven regardless of score.
SolutionShip all-in fare display and opt-in (not opt-out) add-ons first — the two highest-RICE, lowest-effort items, and the two most directly responsive to the CCPA framework — behind an A/B test with revenue-per-booking as a guardrail metric, not conversion alone.
ConclusionEvery recommendation has a named, disclosed-scale cost (opt-in add-ons cut into the same $301.6M revenue line MakeMyTrip itself reports) — not hidden trade-offs. Shipping ahead of enforcement, given the SpiceJet precedent already exists in this exact category, is the actual OKR, not a vague "improve trust" goal.
T41

Business Model & Strategy Deep Dive

Revenue & pricing power Moat & network effects Governance & ownership Five Forces Regulatory & risk mapping Growth strategy

Framework used: a 21-point business model checklist applied end to end — analysis of a real, public company, not something I built. Not a product teardown (see the Pricing Transparency & Dark Patterns Teardown above for that); this is the kind of research I'd do before joining a company, going deep on the actual mechanics of how it makes money, defends its position, and where it's exposed.

A note on evidence: every figure below is sourced to a SEC filing, an earnings release, a named regulator, or dated news coverage — linked inline and in the source lines. Where a claim is a third party's allegation rather than a verified fact (short-seller reports, disputed incidents), it's labelled as one, not stated as settled. Where a figure is my own estimate rather than a disclosed number (CAC, DPO), it's marked "derived," not presented as company-reported.

Snapshot

MetricValueNote
Founded / IPO2000 / NASDAQ 2010One of the first Indian internet companies on NASDAQ
FY26 revenue$1.04BFirst time crossing $1B
FY26 gross bookings (GBV)$10.4B+10.4% YoY, a deceleration from FY25's +25.9%
Overall Indian OTA market share60%+MMT + Goibibo + redBus combined
BrandsMakeMyTrip, Goibibo, redBusPremium / value / bus — distinct behavioral segments, not cosmetic branding

Sources: FY26 results coverage · Market share data

Money In — Revenue by Segment (FY25/FY26)

SegmentFY25 adjusted marginFY26 YoY growth
Air Ticketing$373.1M+13.4%
Hotels & Packages$429.5M+15.7%
Bus Ticketing (redBus)—+29.3%
Others (corporate + fintech)—+37.1%

The fastest-growing lines are the ones furthest from the original flight-commission business. MMTBLACK loyalty covers only 13% of customers. Pricing power sits on a disputed foundation:

ClaimFigureStatus
Real flight commission~4% of gross bookingsAllegation (Morpheus Research, Mar 2026)
Reported "Adjusted Margin" take rate~6.4%Company-reported metric
Customer inducement costs, FY25 (discounts/cashbacks)$301.6MCompany-disclosed, booked as revenue reduction, not an expense

Sources: MakeMyTrip FY25 20-F · Morpheus Research report (allegations)

Money Out — Cost Structure, Unit Economics & Float

Line itemFY25YoY
Service cost$274.3M+27.6%
Personnel$160.1M+8.5% (mainly annual wage increases, not seasonal flex)
Marketing & sales promotion$165.3M+34.1%
Net finance costs (Q2 FY26)$35.9Mup from $0.5M — interest + FX losses, the actual driver of the quarter's net loss despite Adjusted Operating Profit growing
MetricValueStatus
Blended CAC~$18.4/customerDerived (marketing spend ÷ new customers) — likely understated, not split from retention spend
LTVNot disclosedGap — no company or third-party figure found
Cash + term deposits (FY25)$763.3MCompany-disclosed
Contract liabilities (advance customer payments)$31.9M (2021) → $73.9M (2023)Company-disclosed — real evidence of float
Estimated Days Payable Outstanding~114 daysDerived, not a disclosed metric
FX exposure$22.6M profit swing per 10% USD moveCompany-disclosed (FY26 20-F)

The float funds a real fintech layer: TripMoney/BookMyForex (forex, operationally profitable since 2025, $250M+ annual volume), Capital Float (BNPL/travel loans), and Happay (corporate expense management, acquired from CRED, Nov 2024).

Customers — Segments, Value Prop, Channels, Retention

Segment / channelScaleNote
Corporate travel (MyBiz + Quest2Travel)10–12% of GBV, $1B+Structurally different, less seasonal, strongest genuine switching cost
Myra (AI assistant, built on OpenAI)85,000+ daily conversations45%+ from Tier 2/3 cities
Organic search47.83% of website trafficLargest but least controllable channel — Google is also building a competing agentic-booking feature
App81.5M MAU / 8.7M DAU—
Repeat booking rate70%+Third-party NPS inconsistent: -20 to +19 depending on source
AI-agent purchase trust (industry-wide)2–8% willing~70% still prefer a trusted brand over an AI agent — arguably favors Myra's brand-wrapped approach

A real share of the 70%+ repeat rate is plausibly driven by MyCash — a refund/cashback wallet usable only for future bookings, non-cashable — a financial lock-in mechanic, not pure earned loyalty. One documented case: a single refund-processing delay erased 6–7 years of one customer's trust.

Relationships & Structure — the Moat, Stress-Tested

Running every candidate through a "could a funded competitor replicate this in 2 years" test:

Moat candidate2-year replicable?Verdict
82M-user behavioral data (feeds Myra)NoReal moat
Corporate workflow integration (MyBiz/Quest2Travel/Happay)NoReal moat
Brand trustPartiallyReal, but actively contested (CCPA/CCI/Morpheus)
Scale (60%+ share)No, vs. domestic rivalsDoesn't prevent margin loss to global rivals
Price Lock featureYesNot a moat — licensed from Hopper
Myra's AI engineYesNot a moat — built on OpenAI
Loyalty / co-branded cardsYes — already copiedNot a moat — every major competitor runs the same playbook
Supplier relationshipsYesWeaker than they look — Marriott shifted share to Booking
Regulatory access (IRCTC etc.)Yes — already sharedNot a moat — same access for Yatra, ixigo, EaseMyTrip, Paytm, Cleartrip

Network effects are genuinely weak on both sides of the marketplace: 73% of hotels globally use channel-manager software specifically to multi-home across 5+ OTAs, and Indian travelers demonstrably compare prices across apps before booking. The 2022 CCI fine (below) reads as MakeMyTrip trying to manufacture lock-in through restrictive contracts precisely because the natural network effect wasn't strong enough on its own.

Ownership & Governance

Holder / structureDetail
Trip.com Group stake49% (2019–2025) → 16.9%, via ~$5.7B in buybacks across two transactions (2025–2026)
Convertible senior notes$1.25B, 0.00% coupon, due 2030, conversion price ~$121.50/share, redeemable from July 2028
Insiders / institutions~16.66% insiders, 76.91% institutional (largest: Baillie Gifford, 14.39%)
Board composition10 directors, only 4 independent
Audit Committee2 voting members — below Nasdaq's default minimum, via a "home country practice" exemption
Trip.com's residual influenceA Compensation Committee seat (James Jianzhang Liang) + observer roles on two committees, even after the stake cut
Response to Morpheus allegationsOne-line public dismissal — no disclosed independent review

Sources: Trip.com stake reduction (Skift) · Convertible notes terms · Hotel multi-homing data

Competitive Position — Five Forces

ForceVerdictEvidence
Supplier powerHighHotels/airlines multi-home freely; rate parity blocks price-based exclusivity
Buyer powerHigh (most segments)Low switching cost, active cross-app comparison; low only for corporate
Threat of substitutesHigh, wideningGoogle agentic AI Mode, Paytm Travel (via 46%+ UPI dominance), direct supplier booking (OYO)
Threat of new entrantsModerateBarrier is specifically data/scale, not the whole business — niche entrants (ixigo) succeed more easily
Rivalry intensityHighHotel commission compressed 35%→18%; Marriott share lost to Booking (38%→31%)

Market share: 60%+ overall OTA, ~30% air travel, 56.9% within airline booking specifically (vs. Cleartrip 13.7%, EaseMyTrip 13.4%). Share is growing against domestic rivals and eroding against global ones — two different trends hiding inside one aggregate "market leader" number.

Regulation & Risk

Regulator / eventStatusKey fact
CCI (Competition Commission)Live / unresolved₹223.48 Cr fine (2022) for price-parity abuse; a further DG investigation opened Mar 2024, findings "expected early 2026" — no public resolution found
CCPA (dark patterns)Industry-wide risk2023 guidelines, 13 named practices; 2026 enforcement precedent set against SpiceJet, not MakeMyTrip directly
DPDP Act (data protection)Deadline aheadPhased compliance through May 2027, penalties up to ₹250 Cr for security failures
BookMyForex security incidentLive / disputedFeb 2026: ~₹2.55 Cr in fraudulent transactions, 5,000 unauthorized charges approved; company denies it was a "breach"
FDI / Press Note 3StructuralLand-border-country investment (incl. China) needs government approval — directly relevant to the Trip.com stake history

Sources: CCI investigation & Morpheus coverage (Skift) · BookMyForex incident

What Could Break It, Ranked by Evidence Strength

  1. The BookMyForex incident — already happened, disputed, unresolved.
  2. The 2030 convertible notes — a dated obligation, binary outcome (dilution vs. cash redemption) depending on share price.
  3. The open CCI DG investigation — live, no public resolution found.
  4. Google's agentic AI Mode — MakeMyTrip absent from Google's named booking partners, while Google is simultaneously its top acquisition channel.
  5. A compounding scenario — demand shock + note-maturity window + unresolved trust incident landing together, close to what 2026 itself already delivered.

Growth & Change — M&A Track Record and Seasonality

DealPrice / yearOutcome
Ibibo Group (Goibibo + redBus)$960M, 2016$272.2M goodwill impairment (FY2020, COVID-triggered) — but Goibibo remains a genuinely distinct value brand (25% customer overlap) and redBus is a clear category Star
Hotel Travel Group2012$14.6M impairment — a clean failure, no COVID factor
"MGH" investment~2016Impairment — clean failure
BookMyForex, Happay, Savaari, Flamingo Transworld2022–2026No disclosed impairments yet
QuarterSeasonal pattern
Q3 (Oct–Dec)Strongest — Diwali, festival, winter travel
Q2 (Jul–Sep)Weakest — monsoon; but corporate travel is structurally less seasonal, quietly smoothing the overall profile
International segmentFollows the same domestic calendar (outbound Indian travelers) — reinforces the Q3 peak rather than diversifying away from it

International travel now makes up 27% of total revenue, growing 27–45% YoY — a fact that received little proportionate strategic emphasis relative to its actual scale. FY26 GBV growth (10.4%, down from FY25's 25.9%) is substantially explained by a real sequence of 2026 shocks: the Pahalgam terror attack, an India-Pakistan military escalation that cancelled 80% of bookings in the affected window, an Air India crash, and monsoon flooding that displaced 500,000+ people in Assam — not primarily competitive weakness.

The market hasn't rewarded the transformation yet: MMYT trades at roughly half the revenue/EBITDA multiple of Booking Holdings and Expedia despite faster growth, and the stock is down ~38% over the trailing year — partly the West Asia conflict's hit to international travel, partly a broader re-rating of "India-exposed consumer-discretionary" stocks. Leadership has been explicit about the transformation ("beyond aviation" as the next growth frontier); the market is still pricing MakeMyTrip closer to its regional risk profile than to the differentiated platform story being told.

Sources: "4 Shocks That Broke India's Travel Momentum" (Skift) · MMT India IPO filing · Valuation vs. peers

Quick recall

SectionSummary
ProblemNot a UX teardown — the question here is how MakeMyTrip actually makes money, defends its position, and where it's exposed, the kind of research worth doing before joining a company rather than after. FY26 growth decelerated from +25.9% to +10.4% YoY and the stock trades at roughly half Booking/Expedia's multiple despite faster growth — something had to explain that gap.
Research stepsWorked a 21-point checklist end to end: revenue by segment and the disputed real-take-rate question (Money In) → cost structure, derived CAC, and the fintech layer the float actually funds (Money Out) → customer segments and the MyCash lock-in mechanic (Customers) → stress-tested every moat candidate against "could a funded competitor replicate this in 2 years" (Relationships & Structure) → ownership, the Trip.com stake history, and the audit-committee exemption (Governance) → Five Forces → every live regulatory and security risk, ranked by evidence strength → the real M&A track record and seasonality, including the 2026 shock sequence behind the growth deceleration.
SolutionNot a product fix — the deliverable is the analysis itself: which of the claimed moats are real (behavioral data, corporate workflow integration) vs. easily replicated (Myra's AI, Price Lock, loyalty), and a ranked, evidence-graded list of what could actually break the business.
ConclusionThe market hasn't re-rated the "beyond aviation" transformation yet — it's still pricing MakeMyTrip closer to its regional risk profile. The strongest moats (data, corporate integration) are real; most of what looks like a moat (AI engine, loyalty, supplier relationships, regulatory access) is already matched or easily copied by at least one named competitor.
T11

Flight Search & Price Comparison Engine

The System

Search Systems — Google Flights, Skyscanner, Kayak. The engine that turns "Delhi to Bangkok, next week" into a ranked, comparable set of fares across airlines and OTAs. What PMs build here is meta-search (aggregating other platforms' inventory and referring out) vs. direct search (querying and selling your own inventory): a fundamentally different product and monetization model from a booking system, even though the search box looks identical to a user.

Key Concepts

  • Meta-search vs. OTA economics: Skyscanner/Kayak don't sell tickets, they refer (CPC or referral fee); a booking platform earns margin on the fare. Same search box, opposite business model.
  • Real-time fare staleness: GDS/airline APIs rate-limit and cache; a shown fare can go stale between search and click-through. "Fare no longer available" is a core UX problem here, not an edge case.
  • Ranking trade-offs: cheapest-for-user vs. highest-referral-fee-for-business; how a meta-search product decides what "best result" even means.
  • Flexible-date search (±3 days, whole-month calendar view) as a distinct product surface from single-date search, with a very different backend query pattern.

Open Questions

How does Google Flights' ranking differ from Skyscanner's? What's the actual fare-staleness recovery flow when a user clicks through to a dead price? How is a meta-search product's North Star (clicks-to-partner) different from an OTA's (completed bookings)?

Skills Demonstrated

Marketplace/aggregator business models, ranking algorithm trade-offs, real-time data staleness handling — distinct from the single-inventory booking-flow thinking in T1.

T12

Travel Marketplace Platform

The System

Marketplace Systems — Airbnb, GetYourGuide. Two-sided platforms connecting hosts/suppliers with guests/travellers, where trust and liquidity are the core problems, not booking UX. What PMs build here is everything a single-inventory OTA (T1) doesn't need: supply-side onboarding, trust and safety systems, and a matching/ranking model that has to weigh host quality alongside price.

Key Concepts

  • Two-sided liquidity: no guests without hosts, no hosts without guests. Which side do you subsidize first, and how do you measure "liquidity" as a real metric, not a slogan?
  • Trust & safety as core product: reviews, ID verification, host guarantees, guest screening — the actual mechanism that makes a stranger's home bookable at all.
  • Marketplace ranking: host quality signals vs. price vs. availability vs. novelty; very different from ranking a fixed airline-fare inventory.
  • Take-rate design: host fee vs. guest fee vs. blended, and the transparency trade-offs each creates in the checkout UX.

Open Questions

How does Airbnb's search ranking balance host quality vs. price vs. novelty? What's the trust-building flow a first-time host actually goes through? How does GetYourGuide's supply-side onboarding differ from Airbnb's, given smaller/more numerous suppliers?

Skills Demonstrated

Two-sided marketplace dynamics, trust & safety product design, supply-side (not just demand-side) product thinking.

T13

Ride-Sharing & Mobility Booking

The System

Mobility Systems — Uber, BlaBlaCar. Real-time matching between riders and drivers/co-travellers, dynamic pricing, and ETA-driven trust — a live-match product, not a browse-and-book one. What PMs build here is dispatch/matching algorithms and pricing mechanics that respond in seconds, not the search-and-compare flows every other system on this list uses.

Key Concepts

  • Real-time supply/demand matching: the product surface is a live match, not a list of results to browse; latency itself is a feature.
  • Surge/dynamic pricing: the most studied and most controversial pricing mechanism in travel-tech; balancing driver-supply incentive against rider price sensitivity and perceived fairness.
  • ETA accuracy as trust: a wrong ETA is a broken promise, not a UX nitpick; it's the primary trust mechanic in a product with no browsable inventory to build confidence from.
  • Marketplace vs. peer-to-peer: Uber owns dispatch and pricing outright; BlaBlaCar is closer to a trust-and-coordination layer over independently-arranged car-share trips.

Open Questions

How would you redesign surge pricing to reduce rider anger while preserving driver supply incentives? What's BlaBlaCar's trust model for a multi-hour shared car ride, versus Uber's for a 10-minute trip?

Skills Demonstrated

Real-time marketplace/dispatch systems, dynamic pricing design, trust mechanics in a zero-inventory-browse product.

T14

Tours & Activities Marketplace

The System

Experience Marketplace — Viator, Klook. Discovery and booking for bookable experiences (tours, tickets, activities): a catalog/inventory problem more than a real-time-matching one. What PMs build here is supply-side tooling for thousands of small local operators (not a handful of airlines), and the cross-sell integration point where "book a flight" surfaces "things to do at your destination."

Key Concepts

  • Perishable inventory at small scale: an unsold tour slot today is gone forever, the same yield-management problem as an airline seat, but per-operator volume is tiny compared to an airline.
  • Long-tail supply: thousands of small local operators instead of a handful of airlines; supply-side tooling and quality control look completely different from T1's world.
  • Instant confirmation vs. request-to-book: some experiences confirm immediately, others need the operator to manually accept; a real trust/UX trade-off, not just an engineering detail.
  • Cross-sell placement: where in a booking flow (like T1's) should experience upsells actually appear, and does that belong to this system or the booking system it's bolted onto?

Open Questions

How does Klook's supply-side operator tooling differ from a single-inventory OTA's? Where in the funnel (see T8) should an experience upsell actually be placed, and what does it cannibalize if placed too early?

Skills Demonstrated

Long-tail supply management, perishable-inventory yield thinking at small scale, cross-sell/attach-rate product design.

T15

Travel Discovery & Reviews Platform

The System

Travel Discovery Systems — Tripadvisor, Yelp. Pre-booking inspiration and trust — reviews, rankings, and content that influence a decision made and completed somewhere else entirely. What PMs build here is a non-transactional product: most Tripadvisor traffic never books on Tripadvisor. The PM problem is monetizing influence, not completing a purchase.

Key Concepts

  • Monetizing a non-transactional product: referral/ads revenue instead of booking margin; a fundamentally different business model than every OTA on this list.
  • Review fraud & fake-review detection: the core trust problem: how do you know a 5-star review is real, without over-flagging genuine negative ones?
  • Content ranking vs. e-commerce ranking: a "best restaurant" ranking has no purchase event forcing a decision; different signals matter than in a booking-funnel ranking (T8).
  • SEO as the primary growth channel: discovery platforms live or die on organic search traffic in a way booking platforms (T1) don't have to.

Open Questions

How would you detect and penalize fake reviews without over-flagging real negative ones? What's Tripadvisor's actual North Star metric if it isn't bookings?

Skills Demonstrated

Non-transactional business models, trust/fraud detection systems, content ranking, SEO-driven growth strategy.

AI & Agentic Product Design

// designing new AI-driven systems beyond a wrapped prompt
T4

AI Trip Planner — PRD

View the files on GitHub →

Goals & Non-Goals P0/P1/P2 requirements Phased launch plan Open questions, not just answers

Frameworks used: 7-field PRD structure, P0–P2 prioritization — written PRD, no build.

Doc typePRD
StatusDraft v1.2
OwnerBhawna Mangla (PM)
Last updatedJul 2026

TL;DR

"Wayfare AI": a conversational trip-planning assistant that turns "plan me a trip" into a bookable, weather- and budget-aware itinerary in one session. MVP scope: single-destination domestic trips only, 3–7 days. Ships itinerary generation, a budget optimizer, and weather-aware sequencing; explicitly does not attempt multi-city routing, group itineraries, or live disruption reflow in v1; see Goals & Non-Goals below for why.

Problem Statement

Planning a multi-day trip today means 8–12 open tabs (flights, hotels, blogs, weather, maps) and hours of manual cross-referencing. Users abandon planning mid-way and either overpay on a generic package or under-plan and lose time on-trip re-deciding.

Goals & Non-Goals

Goals (v1):

  • G1: Cut time-to-first-itinerary to under 90 seconds for a first-time planner.
  • G2: Convert at least one booked leg per generated itinerary, at a target rate set post-launch (the North Star, below).
  • G3: Let a budget-conscious traveller swap any single item without manually recalculating the rest of the plan.

Non-Goals (explicitly out of scope for v1):

  • Multi-city or multi-destination routing.
  • Shared or group itineraries.
  • Live disruption reflow: needs a flight-status API integration not yet built; sequenced for v2 once MVP itinerary quality is validated, not because it's unimportant (it's Reactive Rhea's entire use case, see below).
  • Replacing a travel agent for complex international multi-leg trips: this is a single-destination planning aid, not a full-service booking replacement.

Users & Use Cases

  • "First-timer Farah": plans 1 international trip/yr, wants 2–3 easy choices, not raw options.
  • "Optimizer Omar": books 6+ trips/yr, wants full control and only trusts AI suggestions he can see the reasoning behind.
  • "Reactive Rhea": mid-trip, needs the plan to adapt live when a flight delays or weather turns (her use case is explicitly a Non-Goal for v1; see above).

User stories:

  • As a first-time planner, I want to type "5 days in Vietnam under ₹60k" and get a full day-by-day itinerary, so I don't have to research every leg myself.
  • As a budget-conscious traveller, I want to see how each day's plan affects my total spend, so I can swap a paid activity for a free one without recalculating manually.
  • As a returning user, I want the assistant to remember my dietary and pace preferences, so I don't re-explain them every trip.
  • As a skeptical planner, I want to see why a restaurant or attraction was recommended, so I can trust or override the suggestion.

Requirements

Prioritized P0 (must ship for MVP) through P2 (explicitly post-MVP):

RequirementPriorityNotes
Destination recommendation from loose intentP0Parses budget band, trip length, vibe from a single input
Day-by-day itinerary generationP0The core deliverable; every other requirement supports it
Budget optimizerP0Reallocates spend across flight/hotel/activities to hit a target
Weather-aware sequencingP1Outdoor-heavy days scheduled against the best forecast window
Restaurant/attraction recommendationP1Rating + proximity + price band, with visible reasoning
Preference memory (dietary, pace)P1Persists across sessions for returning users
Disruption re-planning on delayP2Non-Goal for v1; needs flight-status API integration
Multi-city routingP2Non-Goal for v1
Shared/group itinerariesP2Non-Goal for v1

Design: Key Flows

  • Intent capture: single chat-style input + 3 quick-pick chips (budget band, trip length, vibe), not a 12-field form.
  • Itinerary review: day-by-day vertical timeline, each activity swappable via a "Why this?" expand showing rating/distance/price reasoning.
  • Budget bar: persistent sticky bar showing spend-to-date vs. target, updating live as items are swapped.
  • Book/hold: itinerary converts to a bookable checklist (flight, hotel, key activities) with per-leg deep links.
Four low-fidelity wireframes for Wayfare AI: Intent Capture (chat input plus quick-pick chips), Itinerary Review (day-by-day timeline with a Why this? expand), Budget Bar (sticky spend-vs-target progress bar), and Book/Hold (per-leg booking checklist)

Click to open the full wireframe set (also available in dark mode).

Show more: Success Metrics, Launch Plan, Risks & Open Questions, Reusable Insight (the rest of the PRD)

Success Metrics

North Star: itineraries that convert to at least one booked leg. Supporting: time-to-first-itinerary (target <90s, ties to G1), swap rate per itinerary (a trust signal on first-pass suggestion quality: a high swap rate means the first draft is missing the mark), budget-target adherence rate, D7 return rate for pre-trip re-edits.

Launch Plan

  • Phase 1 (MVP): single-destination domestic trips, 3–7 days. Itinerary generation + budget optimizer + weather sequencing only: the three P0 requirements.
  • Phase 2: multi-city routing + persistent preference memory, gated on Phase 1 hitting its swap-rate and time-to-itinerary targets. Routing multiple cities is premature if the single-city itinerary still isn't trusted.
  • Phase 3: live disruption reflow (Reactive Rhea's use case), gated on a flight-status API integration that doesn't exist yet.

Risks & Open Questions

Risk: hallucinated venue details (wrong hours, permanently closed) need a live data-freshness check; training-time accuracy alone won't catch them. Recommendation logic must stay auditable and explainable so it doesn't over-fit to popular/sponsored listings (same scoring discipline as the Hotel Recommendation Engine project, still on the roadmap).

Open questions this doc doesn't resolve:

  • Should the budget optimizer be allowed to suggest a different destination than requested if it saves >30% of budget, or does that override user intent in a way that breaks trust?
  • What's the fallback UX when no itinerary can be generated within the stated budget: degrade gracefully with a "closest match," or refuse and ask the user to adjust?
  • Does surfacing restaurant/attraction rankings require a sponsored-placement disclosure, and who owns that legal review before launch?

Why Trust Is the Real Problem

Generating an itinerary is the easy half. The real PM problem is the trust layer around it: the "why this?" reasoning, the swap-not-regenerate interaction, whatever lets a user hand over planning without feeling like they lost control of the trip.

Quick recall

SectionSummary
ProblemPlanning a multi-day trip means 8–12 open tabs and hours of manual cross-referencing across flights, hotels, blogs, weather, and maps — users abandon mid-way, overpay on a generic package, or under-plan and lose time re-deciding on-trip.
Research stepsDefined 3 personas spanning first-timer, power-user, and mid-trip-reactive needs → wrote explicit Goals and Non-Goals, naming what's deliberately cut from v1 and why (e.g. live disruption reflow needs a flight-status API that doesn't exist yet) → prioritized P0–P2 requirements → designed the 4 key flows around a "why this?" trust mechanic rather than a black-box suggestion → named the open questions this PRD doesn't resolve instead of pretending it's complete.
Solution"Wayfare AI" — single-destination domestic trip planning (3–7 days) with itinerary generation, a budget optimizer, and weather-aware sequencing, phased so multi-city routing and live disruption handling wait until the core itinerary is trusted.
ConclusionGenerating an itinerary is the easy half — the real PM problem is the trust layer around it. The North Star (itineraries that convert to a booked leg) and the swap-rate metric both measure trust in the first draft, not just output volume.
T3

Customer Support AI Copilot

View the files on GitHub →

Live interactive demo Agentic AI, not just Q&A 3 core conversation flows Escalation & guardrail architecture Deflection vs. safety trade-off

Tools used: HTML/CSS/JS — self-contained interactive demo, no framework or backend.

Where the Volume Comes From

The support queries that pile up fastest at an OTA ("cancel my flight," "how much baggage can I bring," "can I change the travel date") are also the most repetitive and rule-governed. A human agent resolving them one ticket at a time is expensive and slow at OTA volumes; a plain FAQ chatbot that only answers questions can't actually act on a booking. The gap this project scopes is the one in between: an assistant that can both answer and execute (cancel a booking, initiate a refund, apply a fare rule), which is a materially different (and riskier) design problem than a Q&A bot, because it can now make a costly, sometimes irreversible mistake on its own.

Scope & Non-Goals

In scope: cancellations, refund recommendations, baggage-allowance questions, and date/name booking changes: the four query types responsible for most repetitive support volume. Explicitly routed to a human, never handled autonomously: visa legal advice (a compliance and liability question, not a support one; the bot can point to the airline/embassy's own documentation but never advise), disputes and fraud claims, and any refund above a fixed autonomous-action ceiling (see Guardrails below).

Where Today's Industry Falls Short

Before designing the flows, it's worth naming the pattern this design is deliberately not repeating. Travel and OTA support bots in production today tend to fail in one of four recurring ways. This is a pattern visible across the category, not a claim about any specific company's product:

  • Answer-only, can't act: most bots can tell you the cancellation policy but can't actually cancel anything; the moment a user needs an action taken, they're back in a queue for a human, which defeats the point of automating the repetitive cases in the first place.
  • Black-box numbers: the few bots that do quote a refund or fee rarely show the rule behind it, so a user has no way to check it's right, which is the same trust erosion the fare-shock problem in the Flight Booking Teardown (T1) causes at the booking stage.
  • Silent or absent confirmation: bots that can act either skip a confirmation step entirely (risky) or are so conservative they add friction to low-risk actions too (annoying): few calibrate the gate to the actual risk of the specific action, the way Flow 1 vs. Flow 2 do below.
  • Context loss at handoff: "please repeat your issue" after being passed to a human is one of the most common support complaints in the category; the bot's understanding of the problem evaporates the moment escalation happens.

The wireframes further down (Screens 1–4) are the direct, one-to-one answer to these four gaps, not generic chat-UI decoration; each screen is annotated with the specific failure mode it closes. But rather than leave it at mockups, the architecture below is also built and actually running:

Interactive demo of the Travel Support AI Copilot: a chat panel where a user asks to cancel a flight, the copilot shows its reasoning, a confirmation gate appears before the irreversible action executes, and the confirmed cancellation and refund are logged in the Agent Action Log panel

Click to open the live, interactive demo (also available in dark mode): a rule-based simulation of the flows and guardrails below, running entirely in your browser. A public static site can't safely hold an API key to call a real LLM, so this demonstrates the architecture working end to end (confirmation gate, escalation handoff, audit trail), not a claim of production-grade language understanding; labeled as exactly that on the demo page itself.

Flow 1: Cancellation + Refund Recommendation (highest-risk flow)

The flow with the most at stake: it ends in an irreversible action (a cancelled booking) and moves real money. Representative flow, illustrating the decision points, not a transcript from a live system:

TurnSpeakerMessageSystem
1User"I need to cancel my Delhi–Mumbai flight tomorrow"Intent: cancel_booking (0.96) → PNR lookup
2Agent"Found your DEL–BOM booking. Cancelling inside 24hrs of departure means a partial refund under this fare's rules — you'd get back part of what you paid. Want the exact amount?"Tool call: refund_policy_engine(fare_class, hours_to_departure)
3User"Yes, how much"—
4Agent"You'd get back ₹3,200 of ₹5,800 paid. Confirm cancellation?"Confirmation gate: irreversible action, requires explicit user confirmation before executing (guardrail, not optional)
5User"Confirmed"—
6Agent"Done — booking cancelled, ₹3,200 refund initiated, arrives in 5–7 business days. Reference CXL-48213."Tool calls: cancel_booking() + issue_refund(): both logged to the audit trail below
Four wireframes for the Travel Support AI Copilot: Reasoning You Can See (visible confidence and fare-rule reasoning), Confirm Before It's Irreversible (a confirmation gate before cancel/refund actions), Context Follows You to a Human (a structured handoff summary instead of a repeat-yourself transcript), and Audit Trail for Support Ops (every autonomous action logged with a confidence score and review flag)

Click to open the full wireframe set (also available in dark mode): each screen is annotated with the specific industry gap it closes.

Show more: Flows 2 & 3, agent architecture, guardrails, evaluation, metrics, risks (the safety-critical half)

Flow 2: Baggage Policy Question (lowest-risk flow)

The contrast case: a pure information lookup with no action taken and nothing to confirm, so it needs none of Flow 1's guardrails.

  • User: "How much check-in baggage do I get on my booking?"
  • Agent (intent: baggage_query, 0.99 confidence) → tool call: fetch fare rules for the booking's specific fare class → "Your Economy Saver fare includes 15kg check-in. Extra baggage is ₹500/kg if you add it now, or ₹800/kg at the airport — want me to add extra baggage to this booking?"

Fully autonomous end-to-end: answered and resolved the likely follow-up (add baggage) without a human, because nothing here is irreversible or high-value enough to need a confirmation gate.

Flow 3: Booking Date Change (medium-risk flow)

Sits between the two above: it's an action, so pure information-lookup rules don't apply, but a correctable one, so it gets a lighter version of Flow 1's gate: confirm the fare difference, but no separate human-escalation trigger unless the fare-difference lookup itself fails. Intent: change_booking_date → tool call: check seat availability + fare difference on new date → agent states the fare difference and change fee → single confirmation → executes change_booking() and emails the updated ticket.

Agent Architecture

Four layers, in order: (1) Intent router: classifies the incoming message into one of the in-scope intents above, or "unclassified" → human. (2) Tool-calling layer: each intent maps to a fixed set of callable tools (PNR lookup, refund_policy_engine, fare-difference calculator, cancel_booking, issue_refund, change_booking): the agent can only call tools on this allow-list, it cannot invent new actions. (3) Confirmation gate: any tool call that is irreversible or moves money (cancel_booking, issue_refund) requires an explicit user confirmation turn before executing, never fires on the first pass. (4) Escalation router: triggers on low intent-confidence (<0.85), a tool call failing twice, explicit user frustration signals, or any query touching a hard non-goal (visa advice, disputes): handed to a human agent with the full conversation transcript attached, not restarted from zero.

Guardrails

This is the part of the design that actually differentiates a support copilot from a support chatbot, and the part a scope doc for an agent that can take real actions can't skip:

  • Autonomous-action ceiling: the agent may autonomously execute cancellations/refunds up to a fixed rupee ceiling; above it, the same flow runs but ends in a human-approval step instead of auto-executing.
  • Confirmation before irreversible actions: never optional, shown in Flow 1 above; a wrong autonomous cancellation is far more costly to trust than a wrong autonomous baggage-fee quote.
  • No autonomous visa/legal advice: the intent router treats this as a hard non-goal, not a confidence threshold; it escalates every time, regardless of how confident the model is in an answer.
  • Full audit trail: every tool call the agent executes, conversations included, is logged with the triggering intent, confidence score, and outcome, reviewable by support ops, the same operational-audit discipline as the PCMS payout reconciliation work (see Technical Execution, T27).

Evaluation Approach

Deflection rate alone is a misleading headline metric here: a bot can "deflect" a ticket by giving an answer that satisfies the user in the moment but is wrong, which shows up as a good deflection number and a bad outcome. Evaluation needs three lenses, not one: resolution accuracy (a sampled human audit of closed bot-handled tickets weighed against the bot's own confidence score), wrong-action rate: the safety metric, tracking autonomous actions a human reviewer would have executed differently, weighted far more heavily than any convenience metric, and escalation-appropriateness, checking the router escalates when it should as often as it deflects when it shouldn't.

MetricWhat it's really measuringWhy it's on this list
Containment rate% of in-scope tickets resolved without a humanThe headline efficiency number, but never read alone (see below)
Wrong-action rateAutonomous actions a human audit would reverse or flagThe safety guardrail metric: this one has veto power over containment rate
CSAT on bot-resolved ticketsSatisfaction specifically on containment, not overall support CSATA ticket can be "contained" and still leave the user unhappy
Escalation-appropriatenessDid the router escalate the tickets it should haveCatches an overconfident agent as readily as an under-confident one
Time-to-resolutionEnd-to-end, bot-handled vs. human-handled ticketsThe business-facing efficiency number, once the safety metrics above clear

Business Impact (modeled, not measured)

Illustrative, not a real support-cost figure: if cancellation, baggage, and date-change queries together are a modeled ~35% of ticket volume, and the copilot fully contains half of those (the baggage-query type especially, since it needs no guardrail and resolves in one turn), that's roughly a sixth of total support volume handled at near-zero marginal cost instead of a human agent's fully-loaded cost-per-ticket. The number that actually matters for a launch decision, though, is the wrong-action rate at that containment level; a model here would need real historical ticket data to size properly, which is exactly why this stays a scope doc and not a "built" project, unlike the modeled-but-labeled dataset in the Flight Delay Dashboard (T5).

Risks

The risk that actually matters isn't "the bot gives a bad answer": a Q&A chatbot has that risk too, and it's recoverable. It's the bot takes a bad action: an autonomous cancellation or refund that shouldn't have happened costs real money and leaves a damaged booking behind, well past an unhelpful chat log's worth of harm. Every guardrail above exists to keep that specific failure mode's blast radius small and recoverable (confirmation gates, action ceilings, audit trails) rather than trying to prevent it outright, which no confidence threshold can fully guarantee.

What Carries Over From T18

The same lesson as the ChatGPT Memory Feature teardown (Feature Reverse Engineering, T18) shows up again here in a higher-stakes form: the hard PM problem in agentic AI isn't language understanding or intent classification (those are largely solved). It's designing the guardrail and escalation layer that keeps the agent's inevitable mistakes cheap and reversible instead of expensive and permanent.

Quick recall

SectionSummary
ProblemRepetitive, rule-governed support queries (cancel, change date, baggage allowance) are expensive to handle one ticket at a time, but a plain Q&A bot can't actually act on a booking — and an agent that can act introduces a new, riskier failure mode: a costly, sometimes irreversible mistake made autonomously.
Research stepsNamed 4 recurring ways support bots fail across the category (answer-only, black-box numbers, silent confirmation, context loss at handoff) → scoped in/out which query types get autonomy vs. mandatory human routing → designed 3 flows at 3 different risk levels (cancellation+refund, baggage lookup, date change) to show the guardrails actually flex with risk, not apply uniformly → built a working rule-based demo proving the architecture (confirmation gate, escalation handoff, audit trail) runs end to end → defined a 3-lens evaluation approach specifically because the obvious headline metric (deflection rate) can look good while hiding a bad outcome.
SolutionA 4-layer agent architecture — intent router → tool-calling layer (allow-list only, can't invent actions) → confirmation gate on anything irreversible or money-moving → escalation router with full context handoff — plus an interactive browser demo and 4 annotated wireframes, each mapped to a specific industry failure mode it closes.
ConclusionThe risk that matters isn't a bad answer (recoverable) — it's a bad action (not). Every guardrail exists to keep that specific failure's blast radius small, not to pretend it can be prevented outright. The business-impact number is explicitly modeled, not measured, and said so rather than presented as a real support-cost saving.

Which feature is used at which step

FeatureUsed at
Intent routerFirst touch on every message — classifies into an in-scope intent or routes straight to "unclassified → human"
Tool-calling allow-listOnce an intent is classified — the agent can only call the fixed tools mapped to that intent, never an invented action
Confirmation gateImmediately before any irreversible or money-moving tool call (cancel_booking, issue_refund) — never fires on the first pass, shown explicitly in Flow 1
Autonomous-action ceilingInside the confirmation gate — above a fixed rupee threshold, the same flow ends in human approval instead of auto-executing
Escalation routerTriggered by low confidence, repeated tool failure, frustration signals, or a hard non-goal (visa/legal, disputes) — hands off with full transcript, not a restart
Audit trailLogged on every executed tool call, used after the fact by support ops and by the wrong-action-rate evaluation metric
T28

Weft — Workflow-to-Automation Copilot

Chrome MV3 extension, fully local Multi-session workflow merge Automation-brief + ticket export ROI tracked, not just discovered Privacy-by-structure architecture

Tools used: Vite, React, TypeScript, @crxjs/vite-plugin, Chrome Extension APIs (content script + service worker + side panel).

The Gap Nobody's Closing

Three gaps stack on top of each other, and most tools only fix the first: nobody actually knows how work gets done. The real process lives in muscle memory across a dozen browser tabs, and documented SOPs are fiction. The insight dies in a slide, too: the distance from "here's an inefficiency" to someone actually building the fix is where most optimization effort quietly expires. And nobody proves it worked: optimization gets bought on ROI, but almost nobody closes the loop from "we changed X" to "it saved ₹Y / Z hours." Process-mining incumbents like Celonis and UiPath attack the first gap with heavy, IT-led, six-figure deployments, and stop there. Weft attacks all three, lightly, from the browser.

The Mechanism: Observer, Not Agent

Weft is a browser-side observer, not a tool that logs in for you. The user is already logged in; Weft watches while they work, with no credentials collected or stored, no form values, no page content, no screenshots. It captures the structure of the workflow (which step, which system, what kind of action) using stable element roles and labels, never the data on screen. Anything textual that's retained passes a PII-redaction pass, and a privacy preview shows the user exactly what will be stored before it saves, closing a real hole found during a later security review, where structural ARIA roles on a data grid could otherwise leak on-screen content (a customer name in a table cell) even though form values were already masked.

The Closed Loop

Most task-mining tools stop at a map. Weft's actual product thesis is closing the loop all the way to a measured outcome:

StepWhat happens
1. CaptureUser records a task once, and a few more times, because one recording is one person's happy path
2. MergeSessions align into a single map: main path plus variant branches, with how often each occurs
3. AnalyzeEach step is classified against a step-signature taxonomy and gets an AI/automation suggestion with an estimated saving
4. SpecEach kept opportunity becomes an automation brief: problem, systems touched, approach, effort estimate
5. Hand offThe brief exports as a groomable Jira/Linear-ready ticket instead of a dead PDF
6. MeasureOnce shipped, the realized saving is tracked and rolled into an ROI view: hours and money saved, estimate vs. actual

Principles (Product Constraints, Not Marketing)

Because this product watches people work, four rules are enforced structurally, not just promised: capture is worker-initiated only, on the user's own click, never covert or remotely triggered; it's worker-owned: the person reviews and can redact or delete what was captured before anything is shared; Weft never produces individual performance metrics: timing data finds process bottlenecks, never rates a person; and it's private by structure, meaning the sensitive part is designed to never be captured, rather than protected after the fact. These also defuse the biggest adoption and legal risk: recording workers is regulated (EU works councils; varying rules across India, Singapore, Dubai), and worker-initiated, worker-owned, local-only capture sidesteps most of it.

Show more: architecture, build status, business model, and the validation plan (the technical and strategic half)

Architecture

Vite + @crxjs/vite-plugin + React + TypeScript, content script + service worker + side panel, all local storage: no backend, no accounts, no network calls. Reconstruction and merge are rules-based and client-side (segment by URL/submit/domain change/idle time; classify against the taxonomy; merge by step identity rather than literal position, so a single inserted or skipped step in one session doesn't break alignment for every step after it): chosen over ML because a wrong-but-explainable map the user can fix beats an opaque one, and there's no training data yet to justify the complexity.

Build Status

All 8 MVP milestones are built and gate-verified, end to end: scaffold → capture + masking + PII redaction → reconstruction spike → privacy preview → multi-session merge → map + analysis + register → brief + ticket export + ROI tracker → consent + metrics + polish. A subsequent hardening pass (an internal security/architecture review, not a third-party pen test) found and fixed a real capture-time privacy leak, redaction gaps, a cross-tab data-bleed bug, and a service-worker resilience issue where a single failure could permanently break the extension.

The AI-Opportunity Layer

Each captured step is classified by a step-signature taxonomy that maps to a likely intervention. This is the differentiated core, not the map itself:

Step signatureLikely intervention
Repetitive + rule-basedRPA / scripted automation
Copy between systemsIntegration / API bridge
Judgment on unstructured textLLM assist
Lookup / verificationAutomated validation
Wait / handoffProcess redesign, not tech
Approval / decisionPolicy automation + exception routing
Exception branch (from merge)Often the highest-value target, only visible once a workflow's been captured more than once

Business Model & Moat

Land free/low-cost self-serve for a single workflow; expand paid per-team, priced against value delivered (opportunities implemented / hours saved) rather than seats, so price stays aligned with the ROI the buyer already cares about. The wedge is mid-market teams Celonis/UiPath structurally can't serve without cannibalizing their enterprise motion. Task mining itself is already a commodity; the moat is the compounding outcome-data asset built on top of it: every captured session, every thumbs-up/down, and every implemented-and-measured result teaches Weft which step patterns lead to interventions that actually saved money, which incumbents can't see because their tools stop at discovery.

Validation Plan: Riskiest Assumption First

Tests are sequenced by what would kill the product, cheapest first, with kill criteria set before each test, not "ship and see":

Riskiest assumptionCheapest honest testProceed if
A click-stream becomes a readable, correct mapReconstruct 10 real sessions by hand; show them back≥8/10 rated "that's right"
Merging sessions surfaces real variants, not noiseHand-merge 3 captures of one workflowVariants found match what practitioners confirm exist
Teams will actually act on a briefHand a generated brief to an eng/ops team≥2 briefs become real, owned tickets
Realized savings can be measured crediblyTrack estimate vs. actual on 2 implemented itemsEstimates land within a defensible range; team trusts the number
There's a buyer with budgetPitch the ROI view; ask "pay how much, from whose budget?"≥2 credible "yes, from [budget]"

What the Scoping Discipline Bought

The disciplined part of this build wasn't cramming every good idea into the MVP; it was sorting six review-raised gaps by altitude. The ones that made the product invalid or unsellable without them (multi-session merge, page-content privacy) earned MVP slots; the ones with an expensive half (implementation hand-off, suggestion quality) got right-sized MVP features with the expensive part deferred; and the ones that were really principles or strategy, not build (worker-owned consent, the moat) got decided, not coded. Knowing which gap belongs at which altitude is the actual PM work, well beyond reacting to every piece of feedback the same way.

Quick recall

SectionSummary
ProblemThree stacked gaps nobody closes together: nobody actually knows how work gets done (documented SOPs are fiction), the insight dies in a slide before anyone builds the fix, and nobody proves the fix worked after shipping. Incumbents like Celonis/UiPath fix only the first gap, and only with heavy, six-figure, IT-led deployments.
Research stepsDesigned capture as structure-only (never page content/credentials) after identifying what makes recording workers both legally risky and personally invasive → built reconstruction and merge as rules-based and client-side, deliberately choosing explainable-but-sometimes-wrong over an opaque ML model with no training data to justify it → ran an internal security review that found and fixed a real capture-time privacy leak before shipping → sorted 6 review-raised feedback items by altitude (MVP-blocking vs. deferrable vs. a decision, not a build) instead of reacting to all of them equally → sequenced a 5-test validation plan by which assumption would kill the product first, with kill criteria set in advance.
SolutionWeft — a fully local Chrome MV3 extension that captures a workflow's structure, merges multiple sessions into one map with variant branches, classifies each step against an intervention taxonomy, exports a groomable automation brief/ticket, and tracks realized savings against the original estimate — closing the loop incumbents stop short of.
ConclusionAll 8 MVP milestones are built and gate-verified, not just scoped. The real moat isn't the task-mining itself (already a commodity) — it's the compounding outcome-data asset: knowing which step patterns actually led to interventions that saved money, which a tool that stops at discovery can never learn.

Which feature is used at which step

FeatureUsed at
Content script + PII redactionCapture (Step 1) — records structure only, redacts anything textual retained before it's ever stored
Privacy previewBefore any session saves — shows the user exactly what will be stored, the fix for the ARIA-role leak the security review caught
Merge-by-step-identityMerge (Step 2) — aligns multiple sessions into one map even when one session has an inserted or skipped step
Step-signature taxonomyAnalyze (Step 3) — classifies each step and attaches a likely intervention with an estimated saving
Automation brief + ticket exportSpec → Hand off (Steps 4–5) — turns a kept opportunity into a groomable Jira/Linear-ready ticket instead of a dead PDF
ROI trackerMeasure (Step 6) — once shipped, tracks estimate vs. actual saving, the step most tools skip entirely
T29

Career Copilot — 18-Agent Job Application System

18 specialized agents Live, try-it-yourself app Narrated video walkthrough too Planner / Critic / Supervisor orchestration Structured JSON contracts, no free-text parsing Evidence-traceability as the core design bet Non-fabrication enforced structurally, not just prompted

Tools used: Python, Claude API, an 18-agent pipeline architecture (pipeline.py, AGENT_REGISTRY).

The Actual Time Cost

A serious job application takes 6–8 hours of fragmented, error-prone manual work: decoding what a JD actually screens for, researching a company beyond stale memory, honestly assessing fit, tailoring a resume without breaking ATS parsing, building interview answers, and drafting several pieces of outreach copy, each from scratch, with no consistent quality bar. Generic AI chat makes this faster but not safer: it produces confident, plausible-sounding output that's frequently wrong in ways the candidate can't detect (invented metrics, inflated scope, interview answers built on facts not in the candidate's real history). In an interview, a fabricated claim doesn't fail quietly; it fails in front of the person deciding whether to hire you.

The Core Design Bet

The bet that shaped every architectural decision: a job-application AI system's most important property is evidence-traceability, well ahead of fluency: every claim it produces must resolve to something the candidate's actual resume states or a cited, real source, and the system must say "I don't have evidence for this" rather than invent something plausible. That single constraint drove everything from per-field null-ability in the schemas to a dedicated review agent whose entire job is catching violations of it.

Architecture

18 agents across four categories, wired together by pipeline.py in real dependency order (topologically sorted, not a hardcoded sequence):

CategoryAgentsJob
ExtractionResume Analysis, JD Intelligence, ATS Optimization, Resume Rewrite, Gap Analysis, Skill Evidence, Recruiter Simulation, Interview Coach, LinkedIn Optimization, Application AssetsRead the resume/JD and re-derive findings independently, so a later agent can cross-check them instead of trusting one chain blindly
ResearchCompany Intelligence, Learning & Certification, Career StrategyTwo-phase: a search-enabled research call, then a separate non-searching structuring call, so a wrong fact and a malformed JSON field are never the same failure
SynthesisPortfolio Recommendation, Final ReportCompose other agents' already-structured findings into a new artifact, rather than re-reading the source documents
OrchestrationPlanner, Critic, SupervisorReason about the other 15 agents, not resumes or JDs; grounded in one shared, machine-readable registry

The Dashboard

A production-quality Streamlit dashboard sits on top of the pipeline, well past a generic agent-builder UI, only this project's 18 agents. Demo mode (default, no key needed) replays the project's own hand-verified sample outputs so anyone can exercise the full app (upload, live execution graph, history, and analytics) at zero cost. Live mode unlocks real API calls behind an admin-set passcode, with a hard spend ceiling (LIVE_MODE_COST_LIMIT_USD): cumulative estimated cost is checked after every single agent call, and the run stops immediately (no further agents, no Critic, no Supervisor, since those would themselves spend more) the moment the ceiling is met, with a clear "stopped early" banner and whatever completed still saved. The agent catalog itself is scanned live rather than hardcoded, from career_copilot/agents/*.py, so a new agent shows up in the execution graph and Settings page without touching any dashboard file.

▶ Watch the 2:16 narrated demo: the real dashboard running the live pipeline

A genuine screen recording (Demo mode, no live API calls), not a mockup: uploading a resume/JD, the execution graph filling in live, then a tour of every history/analytics page.

Non-Fabrication as a Structural Constraint

Early on, non-fabrication was enforced entirely through system-prompt instructions ("never invent facts"). That's necessary but not sufficient: a model instructed to be helpful will still often fill a gap with something plausible rather than leave it empty, because empty feels unhelpful. The fix that actually held under review was structural: giving fields explicit null-ability and bracketed-placeholder conventions ([Recruiter Name]) as part of the schema itself, so "I don't have this" has a designated, expected place to go instead of competing with "make something up" as an implicit default.

Verification Approach

Every numeric claim across all 18 agents' sample outputs (ATS score arithmetic, gap-analysis row counts, skill-evidence confidence distributions, placeholder counts) was independently recomputed with a script against the underlying JSON, not just asserted in prose. Critic Agent's first real review pass came back almost entirely clean (14 of 15 agents approved or approved-with-minor-notes): read correctly as a red flag, not a green light, since a critic that always says "looks good" is indistinguishable from one that skipped the check entirely. The fix wasn't forcing it to find more issues artificially; it was giving it one concrete, mechanical task it couldn't fake: recompute every stated total against the numbers it's supposedly summarizing. That's what caught the one real bug in the project (Learning & Certification's stated total didn't match the sum of its own line items).

Show more: the orchestration layer and lessons the build actually surfaced (the review-and-verdict half)

The Orchestration Layer

Planner, Critic, and Supervisor don't analyze resumes, JDs, or companies themselves; they reason about the other 15 agents. Their shared ground truth is AGENT_REGISTRY, a machine-readable catalog of every agent's required/optional inputs and real dependencies, built directly from how each agent is implemented, with every agent_id and depends_on reference in a Planner-generated plan programmatically checked against it before acceptance. This makes an entire class of hallucination (inventing an agent, or a dependency, that doesn't exist) structurally impossible rather than just less likely. Supervisor is the top-level entry point: it cross-checks Planner's execution plan against what actually ran, cross-checks every Critic-flagged issue against a named, specific fix rather than a general assurance, and renders a single go/no-go verdict while carrying forward every limitation the other 17 agents raised, instead of letting the verdict imply more confidence than the system actually earned.

What This Build Actually Surfaced

A shared registry beats trusting an agent's self-description. The tempting shortcut was describing the other agents inside each orchestrator's own system prompt, but that drifts the moment an agent's real required fields change and nobody updates three separate prompt descriptions; one machine-readable source of truth doesn't drift. Two-phase research beats one-phase "search and format": combining "search the web" and "structure the answer" into a single model call conflates two different failure modes (wrong facts vs. wrong JSON), so splitting them means a review can tell which one actually broke. And "insufficient evidence" has to be a first-class schema value, not a prompt suggestion; that's the lesson underneath the core design bet itself.

What Actually Made This Hard

Getting one agent to produce good output turned out to be the easy part. The hard part of building a trustworthy multi-agent system is designing the layer that keeps 18 agents honest about what they don't know, and makes that layer's own claims checkable rather than just plausible. Verification-by-recomputation, structural null-ability, and a shared dependency registry are three different tools solving the same underlying problem: trust that has to be earned by the system's design, not asserted in its prompts.

Quick recall

SectionSummary
ProblemA serious job application takes 6–8 hours of fragmented, error-prone work, and generic AI chat makes it faster but not safer — it produces confident, plausible-sounding output that's often wrong in ways a candidate can't detect, and in an interview a fabricated claim fails in front of the person deciding whether to hire you.
Research stepsCommitted to one core design bet — evidence-traceability over fluency — and traced it into every architectural decision → split 18 agents into 4 categories wired by real dependency order → split research into a two-phase search-then-structure call specifically so a wrong fact and a malformed JSON field are never the same failure to debug → moved non-fabrication from a prompt instruction (found insufficient) to a structural schema constraint (explicit null-ability) → independently recomputed every numeric claim in all 18 agents' sample outputs against the underlying JSON → read a near-perfect first Critic pass as a red flag, not a pass, and gave Critic one mechanical, unfakeable task instead.
SolutionAn 18-agent pipeline (Extraction → Research → Synthesis → Orchestration) with a Planner/Critic/Supervisor layer grounded in one shared, machine-readable agent registry, plus a production dashboard with a free Demo mode and a cost-capped Live mode that stops mid-run the instant a spend ceiling is hit.
ConclusionGetting one agent to produce good output was the easy part — the hard part was designing the layer that keeps 18 agents honest about what they don't know, and makes that layer's own claims checkable. The recomputation check caught one real bug (a mismatched total), proving the verification approach actually works rather than just sounding thorough.

Which feature is used at which step

FeatureUsed at
Structural null-ability / bracketed placeholdersInside every agent's output schema — gives "I don't have this" a designated place instead of competing with "make something up"
Two-phase research callThe Research category (Company Intelligence, Learning & Certification, Career Strategy) — search and structuring run as separate calls so a wrong fact is never confused with a malformed field
AGENT_REGISTRYChecked by Planner before any plan is accepted — makes inventing a nonexistent agent or dependency structurally impossible, not just less likely
Critic's recomputation taskAfter each agent runs — recomputes every stated total against its own underlying numbers; the step that caught the one real bug in the project
Supervisor go/no-go verdictPipeline exit — cross-checks the executed plan against the intended one and every Critic-flagged issue against a named fix, carrying forward every limitation instead of implying more confidence than earned
Live-mode spend ceilingChecked after every single agent call in Live mode — stops the run immediately when hit, with partial results still saved

Hosted free on Streamlit Community Cloud, which sleeps the app after a period of no traffic. The first open after a while shows a "this app has gone to sleep" screen with a one-click button to wake it back up (~30–60s), not a broken link.

T6

Hotel Recommendation Engine

Scope

Design how hotel recommendations are generated using budget, location, ratings, amenities, and past bookings — plus how I'd evaluate recommendation quality. fill in: weighted scoring model across the 5 inputs, and the offline (precision@k) vs. online (click-through, booking rate) evaluation approach

Skills Demonstrated

Recommendation system design, explainability — the same auditability standard called out in the AI Trip Planner's guardrails above. Outcome/insight: fill in: once built

T34

OpsPilot — AI Operations Copilot

View the files on GitHub →

Confidence-based auto-approval, live threshold slider 6-stage pipeline: ingest → classify → parse → rules → review → case PII redaction + fraud/value guardrails Full audit log, every decision explained

Tools used: HTML/CSS/JS — self-contained interactive prototype, no framework or backend.

Why This Exists

Every other AI project on this site is either a data-analysis dashboard (SQL, Amplitude, Power BI) or a customer-facing conversational copilot (the Travel Support AI Copilot). Neither one demonstrates the specific shape of AI product I actually shipped-adjacent work in at Bajaj Finance: an internal operations tool that triages a high-volume inbound queue, extracts structured data from unstructured text, and decides — with a human safety net — what can be auto-processed versus what a person has to look at. That's a real, common enterprise-AI pattern, and it wasn't represented here until now.

The Scenario

"Meridian Financial" is a fictional NBFC (no real company, named or implied). Its partner-operations inbox receives a constant stream of emails: partners disputing a payout amount, resubmitting a rejected KYC document, or escalating an unresolved claim. Today that's a person reading every email, deciding what it is, pulling out the relevant details, and deciding what to do — the exact kind of repetitive, rule-governed decision that AI operations tooling is built for, and the same category of problem PCMS solved for payout calculation at Bajaj Finance, applied here to payout correspondence instead.

My Approach

Built the full pipeline as a working prototype rather than a static mockup: an LLM Classifier stage that categorizes each email (Payout Dispute / KYC Resubmission / Claim Escalation / Irrelevant), a Parser stage that extracts structured fields, and a Business Rules Engine that decides auto-approval based on two independent signals: classifier confidence against a configurable threshold, and three hard guardrails that override confidence entirely (a payout above ₹2,00,000, a partner ID with an active fraud flag, or a duplicate claim reference number). The confidence threshold slider on the live demo is fully functional — dragging it re-runs the routing logic against all 14 sample cases in real time, the same lever an ops lead would actually pull in production to trade off automation coverage against review volume.

Key Design Decisions

  • Confidence alone isn't the approval gate — confidence AND guardrails are. A classifier can be 97% confident about a case that's still too risky to auto-approve (a large payout, a flagged partner). Modeling these as two separate, independently-overridable checks — not one blended score — is the difference between a demo that looks safe and a system that actually is.
  • PII gets redacted before it reaches any surface a human doesn't strictly need it on — logs, dashboards, this demo included. The full unmasked value only ever exists inside the secure case record itself, not in anything an ops analyst scans while triaging a queue.
  • Every routing decision is explained, not just logged. The audit log doesn't just say "approved" or "reviewed" — it states the exact reason (which threshold was cleared, or which specific guardrail fired), because an ops lead auditing a month of automated decisions needs to trust the "why," not just the "what."

Recommendations

What I'd push for next if this were a real system, not a portfolio prototype:

  • Track guardrail-override rate as its own metric, separate from auto-approval rate. A rising share of high-confidence cases getting caught by guardrails is an early signal worth watching on its own — it might mean fraud patterns are shifting, not just that guardrails are "working."
  • Route human review decisions back into a feedback loop. Every case a human overturns or confirms is a labeled example; without feeding that back into the classifier, the system never gets better at exactly the cases it's currently worst at.
  • Set the threshold per category, not globally. A 90% bar might be too conservative for well-formed KYC resubmissions and too loose for payout disputes — a single global slider is the simplest version of this control, not the final one.

Where the Real Risk Sits

Same discipline as every other AI project here: the risk isn't the classifier being wrong sometimes — it's a wrong decision being irreversible and unexplained. The guardrails exist specifically to catch the cases where being wrong is expensive (a large payout, a flagged partner), and the audit log exists so a wrong auto-approval is never a mystery when someone has to unwind it. A system that's merely accurate isn't good enough here; it has to be accurate and legible.

OpsPilot dashboard: a 6-stage architecture pipeline (Email Ingestion through Automated Case Creation), KPI tiles (14 processed, 7 auto-approved, 7 to review, 88% avg confidence), a live confidence-threshold slider, a 14-row case queue with category badges and status pills, PII redaction and fraud-guardrail panels, and a full audit log

Click the screenshot to open the live, interactive demo — drag the confidence threshold and click any row to expand its routing reason (also available in dark mode).

Quick recall

SectionSummary
ProblemA partner-operations inbox gets a constant stream of emails (payout disputes, KYC resubmissions, claim escalations) that a person reads, categorizes, and extracts data from one at a time — the same repetitive, rule-governed decision pattern AI operations tooling exists for, and a real shape of AI product not yet represented elsewhere on this site.
Research stepsBuilt the full 6-stage pipeline (ingest → classify → parse → rules → review → case) as a working prototype rather than a static mockup → deliberately modeled approval as two independent, non-blendable checks (confidence threshold AND hard guardrails) instead of one combined score → redacted PII before it reaches any surface a human doesn't strictly need it on → made every routing decision state its specific reason, not just its outcome, since an ops lead auditing a month of decisions needs the "why."
SolutionOpsPilot — a confidence-gated triage pipeline with a live, functional threshold slider, 3 hard guardrails that override confidence entirely (large payout, flagged partner, duplicate claim), and a full audit log explaining every auto-approval or review routing.
ConclusionThe real risk was never the classifier being wrong sometimes — it's a wrong decision being irreversible and unexplained. A system here has to be accurate and legible; accuracy alone isn't enough when someone eventually has to unwind a bad auto-approval.

Which feature is used at which step

FeatureUsed at
LLM ClassifierStage 2 — categorizes each inbound email (Payout Dispute / KYC Resubmission / Claim Escalation / Irrelevant)
ParserStage 3 — extracts structured fields from the classified email
Confidence threshold sliderStage 4 (Business Rules Engine) — the lever an ops lead would actually pull to trade off automation coverage against review volume
3 hard guardrailsStage 4, alongside confidence — override auto-approval regardless of confidence score on a large payout, a flagged partner, or a duplicate claim reference
PII redactionBefore anything reaches a dashboard, log, or this demo — the unmasked value only exists in the secure case record
Audit logStage 6 (case creation) — states the exact reason behind every routing decision for later review
T35

The Verification Gap — Research in Progress

View the files on GitHub →

24 problem statements mapped and sorted by who actually owns each one 12+ organizations traced worldwide, across 6 sectors India-specific: Supreme Court zero-tolerance ruling, July 2026 Primary sources only — no blog paraphrase without a receipt Ran the check myself: a real, unflagged Deloitte report — 100 citations, 49 sourced, 51 not 16-step verification framework → a 12-stage, 3-track platform architecture

Status: this is a live research trail, not a finished case study — it's the groundwork behind a future build (see OpsPilot's confidence-gate pattern, applied one level up — to the deliverable itself, not just a case queue). I'm adding to it as the research goes deeper, so treat the dated entries below as a running log, not a finished table of contents.

Why This Exists

This started as market research for a Deloitte-facing project — "what problem is a company like this actually working on, that a PM could credibly help with?" It turned into something more useful than a single pitch: a documented pattern, verified across dozens of primary sources, about what actually breaks when companies plug generative AI into real client-facing work — and, just as importantly, what doesn't fix it. I'm keeping this section live and adding to it as the research goes deeper, rather than freezing it as a one-time writeup.

Current thesis (updated as this develops)

A vendor's AI platform is built to solve a broad problem well, for millions of possible customers. It was never built to know whether it got a specific client's specific facts right in a specific deliverable. That depth is structurally the buyer's job, not the vendor's — no procurement checklist, certification, or contract clause changes that. The gap isn't a bad vendor choice; it's a missing internal layer that has to exist regardless of which vendor a company picks.

Full log — every question asked, every answer given

Kept verbatim, unedited, for authenticity — this is the actual research conversation, not a cleaned-up summary of it. Sources are the same links cited live during the research; a few of the longer tables below are reformatted from chat markdown into this site's table style, but the words and numbers are unchanged.

Show more: Q1–Q26 (the original Deloitte site scan, the PM/data-management filtering, the build decision, the plain-language explainer, then the full source-hunting, global-pattern, India, platforms, regulation, and procurement thread, plus a completeness re-check and the Canada watchdog's non-investigation)

Q1 — "i want to make fresh project for deloitte company- analyze the whole company website and various other live websites and let me know on which problem statements they are working on?"

(deloitte.com and deloittedigital.com turned out to be blocked for direct fetching in my research environment, so this was built from indexed content, press coverage, and third-party analysis — not a raw site crawl.)

  • Scaling agentic AI — and the governance gap behind it. Deloitte launched Zora AI (with Nvidia) — deployable agents for finance, HR, supply chain, procurement, customer service — already running internally on expense management (claimed 25% cost cut, 40% productivity gain). Their own Tech Trends 2026 report admits only 1 in 5 companies have a mature governance model for autonomous AI agents.
  • AI output verification — and it's not theoretical for them. In 2025, Deloitte Australia delivered a 237-page government report with fabricated citations, a made-up court quote, and nonexistent academic references — and refunded ~$290,000 USD to the Australian government.
  • Service orchestration at scale. TrueServe (Deloitte Digital) — an agentic service-acceleration platform already running 100+ projects across 60+ clients.
  • Financial services clients are telling Deloitte their top pressures are regulatory/compliance complexity, AI adoption vs. stablecoin/digital-asset disruption, and workforce resistance to new tools.
  • India-specific: Project Bharat (expansion into tier-2/3 cities), Union Budget policy advisory, and real open Product Manager roles in Bengaluru/Hyderabad/Pune.

Sources: Zora AI launch · Tech Trends 2026 · OECD.AI incident record · Computerworld · Fortune · Deloitte Digital / TrueServe · Financial services predictions · Deloitte India press room

Q2 — "product managers are working on which projects out of these"

Filtered the list down to what an actual Product Manager — not a strategy consultant, compliance advisor, or HR/infra team — would own:

  • Clearly PM-owned: AI output verification/hallucination risk in client deliverables; governance maturity gap for autonomous AI agents; scaling agentic AI from pilot to production (Zora AI); AI-first process redesign vs. automating human-shaped workflows; tooling broken/suboptimized processes instead of redesigning them; service orchestration/personalization at scale (TrueServe); content supply chain fragmentation (Content Intelligence Hub); safely deploying GenAI in regulated industries (Agentforce accelerators).
  • Borderline: translating AI ROI into enterprise financial impact (analytics/strategy usually owns the answer); AI ambition vs. stablecoin disruption in banking (becomes PM work only once it's a specific product).
  • Not PM work (excluded): infra/security engineering, HR/L&D, org design, macro policy, compliance advisory.

Of the 8 clearly-PM ones, #1 (AI output verification) is the strongest fit — it's the one with a real, sourced, named incident behind it rather than a generic trend.

Q3 — "data management team might work on which project out of these"

Filtered for what a Data Management team (governance, quality, pipelines/ETL, master data, privacy/access) would own — not the PM layer on top:

  • Primary fits: content supply chain fragmentation (Content Intelligence Hub — a named content pipeline built with Adobe/AWS); service orchestration/personalization (TrueServe's underlying "customer data platforms"); data-privacy regulation compliance; safe GenAI deployment in regulated industries (data classification/access governance before an agent can act); AI output verification (the Deloitte incident was fabricated citations — at root, a data provenance/lineage failure); scaling Zora AI (agents are only as good as the data feeds wired into them).
  • Borderline: translating AI ROI (data management owns the reporting pipeline feeding it, not the answer itself); re-architecting tech orgs (data architecture is one input among several).

Interesting pattern: #21, #20, #17, #24, #1, and #3 all share the same underlying skill — building and governing the data layer under an AI/personalization product.

Q4 — "what project i should work on"

Build #1: AI Output Verification & Provenance — a guardrail product for the exact failure Deloitte already had. Reasoning given: it's not speculative (a named report, fabricated court quotes, a real refund, not a guessed trend); it bridges both the PM lens and the data-management lens at once; it extends an existing case study instead of starting from zero; and it's honest to build without pretending to have worked with Deloitte data.

Q5 — "consider me a layman person and in human centric language without considering what project i have already done- focus on deloitte and make me understand the requirement, problem statement."

Explained as a story: Deloitte Australia was hired to review a welfare compliance system; AI was used to help draft the report; nobody properly checked the AI's work before it shipped; the AI invented a quote and attributed it to a real judge in a real court case, and cited academic papers that don't exist. An outside researcher found it. Deloitte had to admit it, rewrite the report, and refund the government close to $290,000.

The real reason: not that the AI is "bad" — AI tools are known to occasionally invent plausible-sounding facts. The failure was a human process failure — whoever was supposed to verify every fact and quote either didn't do it carefully, or there was no system forcing that check to happen before the document went out. Like a newsroom publishing a fabricated quote because the editor didn't fact-check it.

What's actually needed, in plain terms: a reliable checkpoint that (1) flags anything AI-generated that hasn't been independently confirmed, (2) makes it easy to trace every fact/quote/citation back to something real, (3) refuses to let a document ship until that check has happened, and (4) keeps a record of what was checked and by whom.

Q6 — "where can i research about these things- share me links"

Grouped into: (1) the Deloitte incident itself — AI Incident Database entry, CFO Dive's two articles, Horses for Sources' industry take, with an explicit flag that outlets disagree on the refund amount ($63K–$439K depending on what's being described); (2) the general hallucination problem — Infomineo, Hatchworks, Digital Divide Data; (3) how people design the actual fix — Indium and Knostic on human-in-the-loop review, with the caveat that reviewers often approve AI output without truly verifying it when volume is high; (4) the official rulebooks — NIST AI RMF and the EU AI Act's official text, framed as "cite these, don't just cite blogs."

Sources: AI Incident Database #1193 · CFO Dive · CFO Dive (refund) · Horses for Sources · Infomineo · Hatchworks · Digital Divide Data · Indium · Knostic · NIST AI RMF · EU AI Act

Q7 — "so the incident occured in 2024, deloitte admitted in 2025, what measure steps currently deloitte is taking"

Corrected timeline: DEWR commissioned Deloitte Dec 2024; report delivered Jul 2025; a University of Sydney researcher spotted fabrications Sep 2025; Deloitte admitted, corrected, and partially refunded Oct 2025; a second, separate incident surfaced in Canada in Nov 2025 — Newfoundland & Labrador paid ~$1.6M CAD for a 526-page healthcare-staffing report also containing fabricated citations and real researchers credited with papers they never wrote.

What Deloitte actually did: in Australia — corrected the report, added a disclosure statement, partial refund. In Canada — called the fake citations "incorrect," stood behind the report's recommendations anyway, promised a citation review; no refund confirmed; a provincial accounting watchdog opened its own investigation.

The part that matters most: no public evidence of a real fix between incident one and incident two — Canada was delivered and flagged only weeks after Australia went public. Deloitte also already sells a product for exactly this problem — "aiStudio: Large Language Model Controls," including a literal "Citation Retriever" and "RAG Evaluation" control, part of "Deloitte Trustworthy AI™" — with nothing in public reporting suggesting it was used on their own report-writing.

Sources: CBC News · Fortune (Canada report) · FINRA 2026 coverage · The Independent (watchdog) · aiStudio LLM Controls fact sheet

Q8 — "which companies faced similar situations all over world- give me list"

SectorEntityWhat happened
ConsultingDeloitte (Australia)Fabricated citations, invented court quote — partial refund
ConsultingDeloitte (Canada)Fake academic citations in a $1.6M healthcare report — watchdog investigating
ConsultingEY (Canada)16 of 27 sources fabricated/broken — report pulled
ConsultingKPMG40 of 45 citations fabricated or wrong in a flagship AI report
LawSullivan & Cromwell~28 erroneous citations in a bankruptcy motion — public apology to the Chief Judge
LawEllis George LLP & K&L Gates LLP9 of 27 citations fake — jointly fined ~$31,000
AviationAir CanadaChatbot invented a bereavement-fare policy — held legally liable
GovernmentNYC "MyCity" chatbotFeeds business owners fabricated legal information — still live
MediaChicago Sun-Times10 of 15 books in a "Summer Reading List" were fabricated

Scale: as of April 2026, 1,313 documented court cases worldwide involve AI-fabricated content, 496 involving licensed attorneys; sanctions up ~11× in 18 months. Caveat given at the time: almost everything well-documented here comes out of the US, Canada, and Australia — most likely a visibility bias (public court records, active press), not proof the problem is smaller elsewhere.

Sources: Medium (KPMG) · LawSites · Forbes (Air Canada) · Open.cx · HAQQ

Q9 — "in india?"

WhereWhat happenedOutcome
Supreme Court (Essel Infraprojects/NCLT)3 of 6 judgments relied on didn't exist; 3 real ones had invented paragraphs/wrong titlesOrder quashed; zero-tolerance ruling, 2 Jul 2026
Bombay High Court (KMG Wires)Tax officer cited 3 non-existent decisions₹27.91 crore tax order quashed
Bengaluru tax tribunal4 non-existent case citations in an order~₹669 crore order recalled
Delhi High CourtPlea drafted via ChatGPT, fake quotes/casesDismissed; lawyer pulled up
Manipur High Court (2024)Government pleader used ChatGPT for researchReliability concern recorded

The distinctive thing about India's version: the cases cluster in courts and government tax administration — not, so far, a named Indian consulting firm the way Deloitte/EY/KPMG got caught globally. Either they haven't been caught yet, or India's exposure is currently concentrated wherever the paper trail is public.

Sources: Medianama · The Wire · iPleaders · ThePrint (Delhi HC)

Q10 — "other than consulting firms which sectors and which companies made these mistakes all over world"

Tech: Google Bard's factual error in its public launch demo wiped ~$100B off Alphabet's market cap in a day; Microsoft Bing Chat gave wrong Gap/Lululemon financial figures live in its own launch demo. Media: CNET paused AI-written finance articles after errors; Gannett paused AI sports recaps after public mockery; Sports Illustrated's CEO was fired after AI-generated fake bylines. Retail/automotive: a Chevrolet dealership chatbot was manipulated into "agreeing" to sell a $76,000 Tahoe for $1. Logistics: DPD's chatbot was coaxed into swearing at a customer and calling DPD "the worst delivery firm in the world." Aviation: Air Canada held legally liable for its chatbot's invented refund policy. Government: NYC's "MyCity" chatbot still gives business owners fabricated legal information. Healthcare: a lawsuit alleges ChatGPT offered reassurance instead of urging emergency care during a serious health event. Banking: no named scandal yet, but analysts explicitly call it "not if, but when."

Pattern noted: two different failure modes — "fabrication" (Deloitte, EY, KPMG, Sports Illustrated, CNET: AI invents authoritative-looking content nobody checked) versus "loss of control" (Air Canada, DPD, Chevrolet: AI was manipulated or ungoverned in a live conversation the company had already called binding). Both point at the same missing checkpoint between "the AI said it" and "we stand behind it."

Sources: Slashdot (Google Bard) · NPR (Sports Illustrated) · AI Incident DB (Chevy Tahoe) · TechInformed (DPD) · Forbes (ChatGPT health lawsuit) · BizTech (banking risk)

Q11 — "we have noticed gemini not giving correct output or other platforms....currently which platforms are actually responsible for all these issues that happened in deloitte and other companies"

IncidentPlatformConfirmed?
Deloitte (Australia)Microsoft Azure OpenAI — GPT-4oYes, disclosed in the reissued report
NYC "MyCity" chatbotMicrosoft Azure + OpenAIYes
Microsoft Bing ChatOpenAI GPT-4Yes
Google BardGoogle's own Bard (Gemini's predecessor)Yes — different, older product
Chevrolet dealership chatbot"ChatGPT-powered" per reportingReasonably
Deloitte Canada, EY, KPMGNot disclosedNo

Direct answer on Gemini: Gemini is not behind any of the major professional-services scandals. Every disclosed one points to the same platform — OpenAI's GPT family via Microsoft Azure OpenAI — most likely because Azure OpenAI had the largest enterprise footprint during 2023–2025, a deployment-volume story, not necessarily a worse-model story. Pointed to the Vectara HHEM leaderboard as the real apples-to-apples comparison rather than incident anecdotes, and noted every current model still hallucinates on open-ended factual questions, Gemini included — but the paper trail on scandals specifically points at GPT-4/GPT-4o via Azure, not Google.

Q12 — "how many such ai platforms are launched till date and their names."

No official registry exists, so no exact count — but listed the ~19 organizations the industry treats as genuinely distinct foundation-model makers: OpenAI (GPT/ChatGPT), Google DeepMind (Gemini), Anthropic (Claude), Microsoft (Copilot, built on OpenAI), Meta (Llama), xAI (Grok), Amazon (Titan/Nova), Mistral AI, Cohere (Command), Perplexity, DeepSeek, Alibaba (Qwen), Baidu (Ernie), Tencent (Hunyuan), 01.AI (Yi), Zhipu AI (GLM), Stability AI, AI21 Labs (Jamba), Databricks (DBRX), IBM (Granite) — with hundreds of significant variants below that, and thousands of consumer products built on top of this handful, not separate platforms in the same sense.

Q13 — "what is india doing"

Regulation: India's Feb 2026 "AI Governance Guidelines" ("seven sutras") — an AI Governance Group, a Technology & Policy Expert Committee, an AI Safety Institute — explicitly principle-based and largely voluntary; real legal teeth currently come from the existing DPDP Act and IT Rules, not a purpose-built AI law. The guidelines themselves call for further review on generative-AI liability and accountability. Plus the judicial side already covered — the Supreme Court's July 2026 zero-tolerance ruling is doing more real enforcement than the voluntary guidelines.

Building platforms: under the ₹10,000 crore IndiaAI Mission, three sovereign AI models were unveiled at the Delhi AI Impact Summit — Sarvam AI's Vikram (30B) and a 105B reasoning model, Gnani.ai, and BharatGen. Plus Krutrim (Ola's AI arm, India's first AI unicorn, 22 languages), BharatGPT (already deployed at scale on IRCTC), and Bhashini (government language infrastructure).

Flagged as the sharpest angle: India is racing to build sovereign AI at government scale at the exact moment its own Supreme Court is ruling zero-tolerance on the failure mode those platforms haven't solved either — none of the three sovereign models have a publicly disclosed verification layer of their own.

Sources: Tribune (AI Impact Summit) · Official PIB guidelines PDF · explainx.ai · Business Standard · Rest of World

Q14 — "names of platforms other than those 19"

Added, with sourcing: Nvidia (Nemotron, and the "Nemotron Coalition" open-model alliance whose founding members include India's own Sarvam AI); TII/Falcon (UAE); Moonshot AI (Kimi, China); MiniMax (China); Huawei (Pangu — the only Chinese firm with both a frontier model and its own AI chips); Naver (HyperCLOVA X, Korea); LG AI Research (EXAONE/K-EXAONE, a government-backed Korean sovereign-model project); Samsung (health-specific foundation models). Plus a lighter-confidence list flagged as "not re-verified this session": Inflection AI (Pi), Character.AI, Snowflake (Arctic), Salesforce (xGen/Agentforce), Aleph Alpha, Sakana AI, Preferred Networks (PLaMo), Baichuan, StepFun, SenseTime, iFlytek.

Q15 — "and if any company launch their own ai platform - what measures are taken that it can be used."

Legally required, by region: EU — mandatory pre-market conformity assessment for high-risk systems (real law, real penalties). China — mandatory CAC security assessment + algorithm filing before public reach. US and India — no binding pre-launch requirement; NIST's AI RMF and India's guidelines are both explicitly voluntary. Outside the EU and China, a company can currently launch to the public with no prior approval from anyone.

What companies do on their own regardless of law: red-teaming (adversarial testing, documented with exact prompts and severity); model cards / system cards (published limitations and evaluation results); company-specific frameworks (Anthropic's Responsible Scaling Policy, Google DeepMind's Frontier Safety Framework, OpenAI's Preparedness Framework); staged rollout. Explicitly quoted the researchers' own caveat: "red-teaming practices vary widely, and there are few established standards or best practices" — closer to an honor system than an inspection.

Tied back to Deloitte directly: there was no CAC-style filing or EU-style conformity assessment standing between Deloitte and using AI in that report — the only thing that could have caught it was Deloitte's own internal review, which is exactly the layer that failed.

Sources: VDE (EU AI Act conformity assessment) · FPF (China's Generative AI Measures) · Innodata (AI red teaming) · Towards AI (model/system cards)

Q16 — "what all are actual legal documents - share me links"

Split into binding law (EU AI Act official text on EUR-Lex; China's Interim Measures for Generative AI Services, via China Law Translate and the CAC original; India's DPDP Act, 2023, via the official Gazette and India Code) versus guidance/frameworks that aren't codified statute (NIST AI RMF; India's AI Governance Guidelines PDF via PIB; FINRA's 2026 Annual Regulatory Oversight Report). Flagged explicitly: only the first group is actually law with penalties attached — the second is guidance, and conflating the two is exactly the kind of precision gap that got Deloitte in trouble.

Sources: EU AI Act (EUR-Lex) · China's Generative AI Measures (translation) · CAC original (Chinese) · DPDP Act (Official Gazette) · DPDP Act (India Code) · NIST AI RMF · India AI Governance Guidelines · FINRA 2026 Report (PDF) · FINRA guidance page

Q17 — "is there any other document that is left"

Added three more, since "legal document" specifically was asked: India's IT Rules, 2021 (Intermediary Guidelines and Digital Media Ethics Code), official MeitY-hosted PDF; Mata v. Avianca, Inc. (S.D.N.Y. 2023) — the original, most-cited AI-hallucination sanctions ruling, via CourtListener; Moffatt v. Air Canada, 2024 BCCRT 149 — the actual tribunal decision, via CanLII. Noted honestly that I could not find direct links to the Indian court orders discussed earlier (Supreme Court, Bombay HC, Bengaluru tribunal, Delhi HC, Manipur HC) since Indian judgments aren't reliably indexed by general web search — pointed to the Supreme Court's own judgment portal and Indian Kanoon as the right places to search directly instead of taking an unverified link on faith.

Sources: IT Rules, 2021 (MeitY PDF) · Mata v. Avianca (CourtListener) · Mata v. Avianca (full opinion PDF) · Moffatt v. Air Canada (CanLII)

Q18 — "what all details a company needs to submit to govt to ensure their AI models /platforms are working correctly"

EU AI Act Annex IV — the 9-section technical file (binding 2 Aug 2026, high-risk systems only): general description; design & development details including training/validation/testing data governance; monitoring, functioning & control; performance metrics (accuracy, robustness, cybersecurity); risk management; a lifecycle-changes log; applied standards; a signed EU Declaration of Conformity; a post-market monitoring plan.

China's CAC algorithm filing (within 10 working days of launch): algorithm name, type, intended purpose, technical mechanism, plus a security self-assessment report on a CAC template; only a limited public summary is disclosed (category, entity name, product, purpose, Beian filing number). 988 services filed, 598 app-level registrations completed as of mid-2026.

Everywhere else: nothing required. And the sharper point: neither real regime explicitly requires "prove every citation/fact in AI-generated output is real" — the EU governs the system in aggregate, China's filing is about the algorithm being registered, not auditing every document produced with it. Even where governments do require submissions, none of them ask the one question that would have caught Deloitte's actual failure.

Sources: EU AI Act — Annex IV (official text) · AppInChina (China's AIGC filing, plain-language)

Q19 — "before purchasing any AI tool what all things do companies verify"

Model quality (accuracy for the specific use case, disclosed hallucination rates, offline evaluation support, version-comparison); data handling (what happens to data sent to the tool, whether it trains the vendor's model, access controls, data residency); certifications (SOC 2 as baseline, ISO/IEC 42001 for AI management systems specifically, ISO 27001); contract clauses (ban on training on your data without consent, audit rights, 24–72h incident-notification SLA, no silent model swaps, clear liability/indemnification, a real exit/data-deletion plan); and ongoing monitoring — re-reviewing vendor certifications every 2–3 years or after any serious incident, not just once at signing.

Closed with the gap this doesn't cover: none of this verifies whether any specific document a team produces using the tool is actually true — vendor due diligence and output verification are two separate control layers, and most companies only build the first.

Sources: ACC (Procuring AI) · InfosecTrain (ISO 42001) · Internet Lawyer (10 clauses)

Q20 — "So deloitte purchased this right? Deloitte (Australia, welfare report) Microsoft Azure OpenAI — GPT-4o"

Confirmed: yes, almost certainly through an existing enterprise Microsoft relationship. Ran that against the Q19 checklist directly — SOC 2 ✅, ISO 27001 ✅, data residency controls ✅, no-training-on-data terms ✅ standard, established auditable vendor with SLAs ✅. Deloitte would have passed nearly every item. The vendor was not the problem — Microsoft/Azure OpenAI is one of the most heavily audited, certified platforms in existence. What Deloitte didn't have, and no vendor contract can supply, is a process step internal to Deloitte checking whether the specific citations in that specific report were real before it reached the client. Framed as the project's core insight: "Deloitte didn't fail at picking a trustworthy AI vendor. They failed at the one check no vendor relationship can ever cover: verifying their own output before a client saw it."

Q21 — "so it means deloitte or any company can buy any tool- but they shld also have internal tool that verifies if that tool is giving right output as per their clients bcs vendor might have large problem statement solved but we need to go in depth"

Confirmed and sharpened into one line: a vendor's AI tool is built to solve a broad problem, well, for millions of possible customers. It was never built to know whether it got a specific client's specific context, or a specific deliverable, right. That depth is always the buyer's job, never the vendor's. Explained why structurally: the vendor doesn't know the company's client, industry-specific facts, or what "correct" means in that specific deliverable — only the company itself has that context. So every company using AI needs a second, internal layer between "the AI produced this" and "we're sending this to a client" — one that can't be bought off the shelf, because buying it would just recreate the same problem one level up.

Q22 — "give me answer to all these questions and then i will decide on what i need to go in detail" (India-specific consulting firms, a real hallucination-rate comparison, and what Deloitte has actually changed internally)

1) Any Indian consulting firm caught yet? Not found — but the global pattern got bigger while looking. It's now all four of the Big Four, not three: KPMG's withdrawn Oct 2025 report fabricated specific claims about real, named clients (UBS, NHS Greater Manchester, Swiss Federal Railways, and a false claim that Emirates launched a chatbot called "Sara"); EY pulled a May 2026 loyalty-fraud report citing a McKinsey report that doesn't exist; and PwC was confirmed too — four separate PwC Middle East "thought leadership" reports (2024–2026) with fabricated citations, one assessed as 100% AI-generated once references are excluded, including an entirely invented governance framework called "Citizen Pulse." On India specifically: nothing found linking an Indian firm or engagement — only a mildly ironic Deloitte India press release on the same timeline titled "Indian enterprises lead global peers in at-scale AI adoption."

2) Real hallucination-rate comparison (Vectara HHEM leaderboard): two pictures exist. On the older, easier benchmark, Gemini models lead (Gemini-2.0-Flash at 0.7%). On the harder, Feb 2026 dataset (32K-token documents across legal/medical/financial domains), rankings shuffle and everything gets worse — GPT-5.4-nano 3.1%, Gemini 2.5 Flash-Lite 3.3%, Grok 3 5.8%, Claude Haiku 4.5 9.8%, Claude Opus 4.6 12.2%, Gemini 3 Pro 13.6%. The finding that matters: on the harder test, every current state-of-the-art reasoning model exceeds 10% hallucination regardless of vendor — Vectara's own hypothesis is that reasoning models' extra "thinking" steps introduce drift from the source, so more capable models get worse, not better, on this specific failure mode.

3) What Deloitte's actually changed internally: Deloitte's own position hasn't moved — still "stands by the recommendations" on Canada, still frames it as a citation-verification lapse. The real change came from the client, not Deloitte: Newfoundland & Labrador rewrote its own procurement process within days — mandatory human verification, audit rights over vendor AI use. That's a client protecting itself from Deloitte, not Deloitte fixing itself.

Sources: ThePrint (all four Big Four) · OECD.AI (KPMG) · GPTZero (PwC investigation) · Vectara (leaderboard) · Canadian Accountant (NL procurement update)

Q23 — "Lets get on 3rd problem" (deeper dive on Deloitte's internal response)

Found a brand-new, dated announcement — ControlCatalyst.AI, launched 12 Aug 2026 (days before this conversation), a suite for internal audit/SOX/risk-controls work, built on the same "Deloitte Trustworthy AI™" branding as the pre-Australia "aiStudio" product. Chased down what it actually does: automates control testing and evidence documentation for client engagements — the same pattern as aiStudio, a product sold to clients, not a fix aimed at Deloitte's own report-writing process. Verdict: this is the third AI-governance product shipped since Australia, and the third one aimed at what Deloitte sells rather than the specific internal failure that happened twice. The launch timing — 3 months after Canada, weeks after PwC's scandal broke — reads more like catching up on positioning industry-wide than a purpose-built response to this exact failure.

Sources: Deloitte press release · CPA Practice Advisor · TechEdgeAI

Q24 — "what all tools/websites/apps like ControlCatalyst.AI are present in market"

Three distinct market layers, not one:

Layer A — direct SOX/GRC competitors:

ToolWhat it is
Optro (formerly AuditBoard)Category leader — SOXHUB heritage, 1,585+ G2 reviews at 4.6/5, Fortune 500 + Big Four advisory partnerships
Workiva4,000+ customers, 75% of Fortune 500 — links SOX controls evidence directly to source financial data and SEC filings
Diligent One PlatformAudit management + continuous monitoring, AI analyzing 100% of transactional data, not sampling
KPMG's own "TACO" frameworkAn agentic-AI governance framework reusing SOX's ICFR structure — notable since KPMG is also the firm that got caught fabricating an agentic-AI report

Layer B — the tools that actually caught the Big Four:

ToolWhat it is
GPTZeroThe tool literally used in the KPMG, EY, and PwC investigations. Originally an academic-integrity plagiarism detector, now doing citation-fabrication forensics on Big Four reports. Acquired by Superhuman (Grammarly), June 2026 — $88M valuation, 19M+ users, $30M ARR
Originality.AICompeting AI-content detector; its integrated fact-checker flags fabricated statistics, wrong dates, and unverified attributions. Enterprise API available ($136–179/mo)

Layer C — enterprise LLM output-verification infrastructure (the category closest to a real project):

ToolPositioning
Patronus AIOpen-source "Lynx" hallucination detector, plus domain-specific evaluators (finance, legal, healthcare, copyright)
GalileoReal-time runtime guardrails — sub-200ms inline blocking of unsafe/hallucinated responses before they reach a user
ArizeLLM evaluation platform, hallucination/relevance scoring, LLM-as-judge, strong RAG support
Guardrails AIOpen framework (RAIL) for enforcing structure and validation rules on LLM output

The honest read: Layer C is not empty — it's real, funded infrastructure. What doesn't exist anywhere publicly is the specific product decision to wire that infrastructure into one firm's actual report-delivery workflow. That's a PM gap sitting on solved technology, not a technology gap.

Sources: RiskWatch · Originality.AI (GPTZero review) · Braintrust (hallucination detection tools) · Galileo vs. Patronus

Q25 — "what pm gap missing didnt understand, tell me in detail"

Explained with the actual proof it's real: Deloitte already owned a citation-checker (aiStudio's "Citation Retriever") before Australia, and it still shipped fabricated citations anyway. The technology wasn't the problem. The smoke-detector analogy: owning a detector doesn't stop a fire unless someone's also decided which rooms need one, who checks the battery, what happens when it beeps, and who's accountable if it was dead for months. Six concrete decisions nobody's making at Deloitte: (1) does the check run on every report or only some; (2) where in the workflow does it run; (3) what confidence score is "good enough" to ship without a human; (4) whose job is it to actually check a flagged item; (5) what happens when someone overrides it under deadline pressure; (6) who's accountable if the whole thing gets silently bypassed. None of these are technology questions — they're business-process decisions, which is the actual, portable definition of what a PM does here that a detection vendor never will.

Q26 — "check whether our research work was complete or something left. And then check how government and deloitte are working together" (18 Sep 2026)

Completeness check on this log: Q1–Q25 hold up — every question has a primary source, and the "Where this goes next" list below was already fully resolved or pointing forward into Part 2/3. The real gaps are in the build, not the research: Part 2 is explicitly one audited report with Track C never run, and Part 3 has 4 of 7 planned screens still wireframe-only — both honestly labeled already, both blocked on this environment's network access to a source PDF, not on missing research.

New finding — Canada's accountability gap just got worse, not better. CPA NL (the professional accounting watchdog covering Newfoundland & Labrador) opened an investigation into Deloitte's fabricated citations in the $1.6M healthcare report — and in May 2026 closed it without investigating anything, on a legal-entity technicality: the report was delivered under a contract with "Deloitte Inc.," which isn't a CPA NL-registered firm, while "Deloitte LLP" (the firm that is registered) "was not a party to the contract and had no involvement in the report." Nobody has been held accountable for that report by anyone. This directly extends the project's own thesis from Q7/Q22 — not just that no internal fix is visible, but that the external oversight mechanism that might have forced one has now formally declined to look.

A correction, not an addition — checked and ruled out. A South Africa story surfaced in the same search results (a government AI policy pulled in April 2026 after 6 of 67 citations turned out to be fabricated) and it would have been easy to log as a fifth Big Four incident. Checked directly: it isn't one. The policy was drafted internally by South Africa's own Department of Communications and Digital Technologies under Minister Malatsi — no consulting firm named in any source as author or vendor. Noting this explicitly rather than silently skipping it, since conflating two similar-sounding stories is exactly the kind of unverified-citation error this whole research thread is about.

No evidence found that any government has cut ties with, blacklisted, or restricted new contracts to Deloitte specifically because of these incidents. Newfoundland & Labrador's procurement-process tightening (already logged at Q7) remains the only concrete governmental response on record — and now the one oversight body that could have added a second is off the table.

Sources: The Independent (investigation opened) · Ricochet Media (investigation closed) · Canadian Accountant · ANC Parliamentary Caucus (South Africa statement) · CNBC Africa (South Africa policy pulled)

Where this goes next

Open threads, updated as they resolve:

  • Find out whether any Indian consulting firm has had a Deloitte-style incident yet — checked (Q22): not found; the pattern instead expanded to all four of the Big Four globally.
  • Get a real hallucination-rate comparison across GPT/Gemini/Claude — checked (Q22): Vectara HHEM leaderboard reviewed; on the harder 2026 benchmark, every current frontier model exceeds 10%, regardless of vendor.
  • Track what Deloitte's actually changed internally — checked (Q23): ControlCatalyst.AI (12 Aug 2026) is a third client-facing product, not an internal-deliverable fix — still no public evidence of the specific gap being closed.
  • Re-check whether any oversight body has actually held Deloitte accountable — checked (Q26): CPA NL closed its Newfoundland & Labrador investigation in May 2026 without investigating, on a legal-entity technicality. No government has restricted new contracts to Deloitte over any of this.
  • Design the actual internal verification layer this points toward — see the execution-roadmap exercise, a role-play working through exactly this as a hypothetical Deloitte PM.
  • The research continues directly into the actual hands-on audit and the platform it led to — see Part 2: From Audit to Prototype.

Quick recall

SectionSummary
ProblemStarted as an open question — "what problem is a company like Deloitte actually working on, that a PM could credibly help with?" — and surfaced a real, documented pattern: AI platforms are built to answer broadly, not to verify a specific client's specific facts, and two AI-fabricated Deloitte reports, an EY report pulled for 16 fabricated sources, and a KPMG report with 40 of 45 citations wrong all trace back to the same missing layer.
Research stepsMapped 24 problem statements by who actually owns each one (Q1–Q10) → traced the pattern across 12+ organizations and 6 sectors worldwide, sourced only to primary documents (Q11–Q21) → ruled out a look-alike (South Africa's policy pull — checked and correctly excluded, Q26) → diagnosed the actual gap as 6 unmade business-process decisions, not a technology gap (Q25) → re-checked accountability as late as Sep 2026 and found Canada's oversight body closed its investigation without investigating (Q26).
SolutionNot proposed yet at this stage — this part is diagnosis, not a build. The actual verification-platform architecture comes out of the hands-on audit in Part 2, and the org-adoption plan for it is the role-play execution roadmap.
ConclusionThe gap is real and sourced, not assumed — confirmed across 12+ organizations, not one incident. The honest, portable finding: owning detection technology (Deloitte already had a citation-checker before Australia) doesn't fix anything until someone makes the business-process decisions around it — which is the actual, non-technical job a PM does here.
T35b

The Verification Gap — From Audit to Prototype

View the files on GitHub →

Ran the check myself: a real, unflagged Deloitte report — 100 citations, 49 sourced, 51 not 16-step verification framework → a 12-stage, 3-track platform architecture 8 screens wireframed, designed, and built into a clickable prototype A real landing page, sign-up flow, and honest launch-readiness check

Status: where the research turned into hands-on work — the actual citation audit, the verification framework it produced, and the platform architecture, wireframes, design, and prototype built from it. Picks up exactly where Part 1 left off, at Q26.

Q26 — "ok lets start here" (sourced a real, unflagged Deloitte report and ran the actual check)

Moved from research to hands-on audit. Sourced Deloitte's "Regulatory Outlook for Financial Markets 2024" — a real, public, 56-page Deloitte Centre for Regulatory Strategy report, not one of the already-flagged scandal reports. First check: a discrepancy-detection pass on structural completeness — do the in-text citation numbers and the bibliography actually match. Counted every citation marker across all 56 pages by hand and cross-validated the total by summing page-by-page (32 through page 10, +23 = 55 through page 24, +20 = 75 through page 39, +25 = 100 through page 50 — the sum lands exactly on the highest in-text number, confirming the manual count was accurate). Then opened the Endnotes section: it lists entries 1 through 49 only. Citations 50 through 100 — 51 of the report's 100 in-text citations, just over half — have no bibliography entry anywhere in the document.

Audited document: Deloitte — Financial Markets Regulatory Outlook 2024 (PDF, hosted on deloitte.com)

Q27 — "what is citation first explain in simple terms" / "how many citations each page have"

A citation is just a receipt — a small number in the text that points to exactly where a claim came from, so a reader can go check it themselves instead of taking the author's word for it. Delivered the full per-page count across all 56 pages as the evidence base for the structural check above.

Show more: Q28–Q49 (the verification methodology, the full 51-row unverifiable-citation map, the 16-step framework, the market scan, and the entire platform architecture, wireframes, and prototype)

Q28 — "what all steps should i plan by which i shld verify each thing in the document"

Formalized an 8-step verification methodology, each step mapped to what would need to be built to automate it:

#StepWhat it checks
1Structural completenessIn-text citation count vs. bibliography entry count — cheap, automatable, and the one that caught the real finding
2Existence checkDoes each citation's URL resolve and match a real, findable source
3Content-accuracy checkDoes the source actually say the claim — the hard, AI-model-dependent step
4Named-author verificationCross-check quoted speakers/authors against a real publication database
5Statistical/data-point verificationDoes the quoted number match the source's number
6Self-citation consistencyFlag when the publisher cites itself
7Recency/currency checkHas a cited fact since gone stale or been contradicted
8Sampling/coverage thresholdNot automatable — the PM-owned policy call on how much unverifiable content is acceptable to publish

Q29 — "now check each citation by going the link provided and see if statement is there"

Hit a real environment wall first: WebFetch came back EGRESS_BLOCKED on every domain tested — including a completely neutral sanity-check domain with zero sensitivity, confirming it wasn't a Deloitte-specific block but a full outage. Pivoted transparently to WebSearch as the verification method instead — same rigor, different tool. Result: all 49 present endnotes independently checked out as real — correct organization, correct date, correct author, topically accurate — across sources including DNB, IMF, World Bank, BIS, ESRB, IOSCO, BCBS, EIOPA, FSB, ECB, Bank of England, the European Commission, the Federal Reserve, UK Finance, Reuters, and Deloitte's own site. Two details worth flagging: endnotes #40 and #46 cite the identical "Future of Payments Review" report for two different points (redundant, not wrong), and #49 is a self-citation — Deloitte citing its own report on digital-asset regulation.

Then mapped every one of the 51 uncited citations (50–100) to its exact page and claim — the evidence that turns "roughly half is unsourced" into something concrete and checkable. The five below are the ones that matter most — specific, quotable statistics presented with the exact same confident formatting as the 49 verified ones, with no way for a reader to tell, just by looking at the page, which citations are checkable and which aren't:

#PgThe specific claim it's attached to
6532"£740 billion (around 55%) of all lending to UK businesses" comes via NBFIs
8144"49% of portfolio managers and 69% of stockbrokers had identified no vulnerable consumers"
9649Private credit grew "from less than USD 60 billion in 2002 to over USD 1.3 trillion in H1 2022"
9950Global private-equity dry powder hit "USD 2.69 trillion in December 2023"
10050Deloitte's own "ESG in Private Capital Survey 2023" — 69 respondents, 61 firms, "91% already having ESG policies" — a second, uncredited self-citation
Show all 51 — every citation, every claim, not just the five above (the full page-by-page map)
#PgThe specific claim it's attached to
5023Spain has postponed its MiCAR compliance deadline to December 2025
5123Licence timings "vary between two weeks and two years across the EU27"
5223Bitcoin/Ether daily trading-volume chart (source: CoinMarketCap)
5323Firms should prepare for "heightened scrutiny" on digital-asset licensing
5424Major jurisdictions pushing ahead with their own national AI approaches
5524AI use-case chart (source: Warwick Business School)
5628ECB imposing "periodic penalty payments" on SSM banks
5728UK regulatory-fines chart (source: PRA, FCA)
5829EBA issued revised guidelines on resolvability
5929PRA will review branch-vs-subsidiary thresholds
6029Near-final EU CRD6/CRR3 text published
6129Near-final UK capital policy published
6229Large banks capitalised to absorb "a very significant credit shock"
6329"Evidence" cited from 2023 bank failures
6429Reverse stress tests (RST) recommendation
6532"£740 billion (around 55%) of all lending to UK businesses" comes via NBFIs
6632"Senior supervisors" question effectiveness of indirect NBFI regulation
6733ECB published Oct 2023 review of CCR governance practices
6833PRA identified CCR management shortcomings
6933FCA found poor client-relationship management
7035Reference to "SUK reforms"
7136FSCS branch liability cap of £500m
7237"New rules on MOBI" came into effect end of 2023
7337FCA says wholesale market has "a long way to go" on inclusive culture
7438Supervisors advocate "impact underwriting"
7539Insurers must attest annually to Fundamental Spread/MA sufficiency
7641FCA plans to review the advice/guidance boundary in 2024
7743Info-sharing needed for product governance/value assessments
7843FCA's 2024 review of Duty embeddedness
79–8043FCA highlighted firms not passing on a fair share of revenues
8144"49% of portfolio managers and 69% of stockbrokers had identified no vulnerable consumers"
8245"Significant shortcomings" in the FCA's fund-liquidity review
8345PRA's model risk management principles
8446IOSCO guidance on swing-pricing market-impact modelling
8546FSB recommendation against daily dealing for illiquid-asset funds
86–8747DC/DB pension investment-opportunity measures
8847The Mansion House Compact
8947The LTAF (UK)
9047ELTIF II (EU)
9147New EU loan-origination-fund regime
9247IOSCO warning on rate-driven default/valuation risk
9347Regulators calling for more private-market transparency
9447ESMA's CSA on valuation risks
9548PRA model risk management principles (repeat citation)
9649Private credit grew "from less than USD 60 billion in 2002 to over USD 1.3 trillion in H1 2022"
9749FCA raised concerns re: high-risk/complex investments sold to wealth clients
9850Deloitte's "own analysis" on fee-discount value
9950Global private-equity dry powder hit "USD 2.69 trillion in December 2023"
10050Deloitte's own "ESG in Private Capital Survey 2023" — 69 respondents, 61 firms, "91% already having ESG policies" — a second, uncredited self-citation

Q30 — "so what all checks we did and what we did miss? what shld be the roadmap"

Self-audit against the 8-step framework: fully done — structural completeness, existence checks on all 49. Partial — content-accuracy (topic-level match confirmed, not full sentence-level verification) and self-citation (2 found, not run as a systematic sweep). Not done — named-author verification beyond 2 spot checks, statistical re-verification of every number, and recency (now actually possible in hindsight, since 2024 forward-looking predictions can be checked against what really happened by 2026). Turned that gap analysis into a 7-phase build roadmap — structural scan first (cheapest, catches the most), then existence + topical match, then the hard content-accuracy layer, then statistics, author-verification, self-citation/circularity, recency monitoring, and a governance policy layer last.

Q31 — "what else is missing in 8 step framework"

Pushed past execution gaps into methodology gaps — failure modes the 8 steps don't even have a step for. Found 7 more, restructured into 5 tiers by what kind of work each actually needs:

TierAddsCatches
1 — StructuralCitation-placement accuracyA marker attached to the wrong claim in the same paragraph — a real bug found on page 29, where a duplicated paragraph reused an old citation number
2 — SemanticFraming fidelityA true fact used out of context — a caveat dropped when quoting a source
2 — SemanticDerived-statistic methodologyA number the report calculated itself from raw source data, not quoted verbatim — is the math shown and correct
3 — ProvenanceChain-of-custodyA "source" that's just relaying someone else's number a hop removed — citation laundering
4 — TemporalInternal consistencyThe report contradicting itself across pages
4 — TemporalLibrary-level pattern checkWhether a firm's self-citation rate, or a repeated-but-superseded claim, is trending across its whole publication history — invisible if every audit is per-document
5 — GovernanceMateriality/risk-weightingNothing on its own — it's the triage layer deciding which citations get the deepest scrutiny (headline stat vs. background aside) instead of spending a fixed budget evenly
5 — GovernanceRemediation workflowWhat happens after something's found wrong — retraction, versioning, notifying anyone who already relied on it. Detection without this is a dead end

Q32 — "what all websites exist working on these frameworks"

Fresh market scan against the 5-tier structure — the finding is the shape of the gap, not any single tool:

LayerPlayersWhat they actually cover
StructuralSwanRef, CiteCheck, RefCheck-AIAcademic-paper citations only — 150M+ papers via Crossref/Google Scholar, binary real-or-fake
SemanticPatronus AI (Lynx), Galileo (Luna), Vectara (HHEM-2.3)Live RAG-answer faithfulness, in the moment a chatbot responds — not auditing a finished, already-published PDF
ProvenanceGPTZero, Originality.AIAcademic/AI-content plagiarism forensics — the tool literally used to catch KPMG, EY and PwC, but built for papers, not corporate reports
Legal-specificLexis+ / Protégé, NexLawCase-law and statute citations only
GovernanceWorkiva, Diligent, AuditBoard, Deloitte's own ControlCatalyst.AIInternal SOX/financial controls — not the credibility of a firm's own published thought-leadership

Every specialist tool is built for a different document type, a different moment (live generation vs. after publish), or a different purpose (internal controls vs. public credibility). Nobody stitches structural + semantic + provenance + governance together, aimed at a firm's own published research output — the whitespace this whole thread has been pointing at.

Q33 — "explain me the gap in layman language"

Every existing tool is a specialist checkpoint doing one narrow job — a bouncer checking student IDs (SwanRef), a spell-checker built into a live chatbot (Patronus/Vectara), a plagiarism detector at a university (GPTZero), a lawyer's citation checker (Shepard's/NexLaw), or the internal-audit team checking a company's own books (Workiva/Diligent). Nobody's built the full check-up doctor — the one who takes a firm's own published report and runs it through every layer at once, and specifically tells you where the sourcing simply stops and nobody flagged it. That's exactly what the manual walkthrough of the Deloitte report proved by hand this week.

Q34 — "what all things need to be checked in full report"

Consolidated into a 10-item, end-to-end audit checklist: (1) document-wide structural count-match, (2) per-citation existence/attribution/topic/claim/context/placement, (3) every number and statistic, (4) every chart's data source, (5) every quote and named person, (6) self-referencing, (7) internal consistency across the document, (8) currency — did a forecast actually hold up, (9) the materiality-and-coverage decision of which claims matter most, (10) what happens after something's found wrong. This list is what the platform's architecture is a direct build-spec for.

Q35 — "does it cover — nobody has built the 'full check-up' doctor"

Mapped the 10-item checklist against the market scan, item by item, to confirm the gap is real and not just a hunch: existence/attribution (academic tools only), content-match (live-chat tools only), context/caveat-dropping (nobody, anywhere), citation-placement (nobody), derived-statistics (nobody), chart sourcing (nobody), self-citation as a standing feature (nobody), internal consistency (nobody), currency re-checking after publish (nobody), and materiality/remediation workflow (GRC tools have it, but for internal controls, not public reports). Confirmed: every specialist tool is either the wrong document type, the wrong moment in time, or the wrong purpose — the "full check-up" really doesn't exist yet.

Q36 — "what if each speciality is built so that the system is not heavy and correct output is given"

The instinct is exactly how these systems actually get built in the industry, not a simplification — confirmed against the market research itself (Braintrust, on the current RAG-eval landscape: "there is no single tool that does everything — a production stack layers an eval framework, a faithfulness checker, an observability platform, and an inline guardrail"). Laid out the shape: a cheap parsing layer runs on 100% of citations first; a router decides which specialists a given citation actually needs (a $-figure gets the statistics specialist, a quote gets author-verification, most citations trigger 2–4 of the roster, never all of them); each specialist stays small and narrow — several can be bought/integrated (Patronus, Vectara) instead of built; a lightweight aggregator combines verdicts into one score; governance sits on top as pure rules, no model at all. The real product becomes the router and the governance layer, glued around a handful of licensed specialists plus the handful nobody else has built.

Q37 — "what all specialisations are there"

Full roster, organized by what signal in a citation triggers each one:

TriggerSpecialistWhat it checks
Always runsExistenceDoes the source resolve, is it real
Always runsAttributionRight org, date, author
Always runsContent-matchDoes the source support this specific sentence
Always runsPlacementIs the marker attached to the right claim, not an adjacent one
Always runsSelf-citationDoes the source domain match the publisher's own domain (near-free, pure logic)
$ or % figureStatistics-matchDoes the number match the source
Report's own calculationDerived-calculationIs the math shown and correct
Chart/graph captionChart-dataDoes the visualized data match the source
Named person + quoteAuthor-verificationIs the quote real, right date/venue
Secondary/news sourceChain-of-custodyPrimary source, or laundered through one
Law/regulation referenceLegal-citationDifferent registry entirely (Shepard's-style)
Forecast/deadline languageCurrency/recencyHas this since been superseded or contradicted by events
Caveat in the sourceFraming-fidelityWas the caveat dropped when the report used the fact
Repeated fact in-docInternal-consistencyDo the two instances agree
Matches a past reportCross-report/libraryContradiction or repetition across the firm's own history
Not a checkerMateriality/risk-weightingDecides how deep the other specialists should go, based on where the citation sits (headline vs. footnote)

Q38 — "what should be steps to check and conclude which route to go and which specialisation to be selected"

The router's actual decision sequence, run for every citation: (1) isolate the exact sentence and source attached to the marker; (2) the existence gate — no bibliography entry at all means stop immediately, flag unverifiable, nothing else runs (this single gate alone caught citations 50–100 in the real audit); (3) run the baseline 5 unconditionally; (4) classify the claim by scanning for $/% figures, quotes, forecast language, chart attachment, legal references, secondary sourcing, and attach only the relevant specialists; (5) score materiality separately, in parallel, deciding depth rather than which specialist runs; (6) aggregate every specialist's verdict into one tier — green/amber/red — with a full evidence record; (7) escalate anything uncertain or high-materiality to a human instead of auto-closing; (8) feed the human's decision back to sharpen future classification.

Q39 — "create a complete architecture from document extraction to final output"

Full 12-stage pipeline, grounded in the real Deloitte audit at every stage:

#StageDoes
1ExtractionParses text, footnote markers, chart captions, quotes, and the bibliography as separate structured fields
2Citation Graph BuilderLinks every marker to its exact claim and its bibliography entry, or null — the stage that alone surfaces the structural gap
3Classification & RoutingTags each citation with which specialists it actually needs
4Specialist VerificationRuns only the triggered specialists in parallel, each a small, swappable service
5Materiality ScoringScores prominence — pull-quote/headline vs. supporting detail
6Unified ScorecardCombines everything into one master table — never blends the buckets
7Escalation / Review QueueRoutes anything uncertain to a human, with evidence pre-assembled
8Governance Policy EnginePure rules — a firm's configured coverage threshold decides publish/hold/review
9Output / ExportGenerates the shareable audit report
10Post-Publication MonitoringRe-checks existence and currency on a schedule after the report ships
11Cross-Report LibraryIndexes every audited report — surfaces pattern-level trends a single document can't
12Data storesFive persistent stores backing the pipeline: Citation Graph DB, Source Cache, Review/Audit Log, Policy Config Store, Report History Index

Q40 — "remember our main target is to find AI Hallucination"

An important correction, caught before it became an overclaim: what the manual audit found on the Deloitte report is unverifiable, not hallucinated — those are different claims. Hallucinated means the AI invented something that never existed. Unverifiable means there's nothing to check against; the claim could be entirely true. Re-sorted all 16 specialists against the real target: only 5 — Existence, Content-match, Author-verification, Statistics-match, Chain-of-custody — actually prove fabrication. Everything else (structural completeness, placement, self-citation, framing-fidelity, internal-consistency, currency, materiality/governance) is real report-hygiene value, but it isn't hallucination detection, and the pitch has to stay precise about which is which.

Q41 — "create an architecture for a platform — if required divide it into two things hallucination and unverifiable"

Split the pipeline into two tracks right after the Citation Graph stage, forking on one question — does this citation have a source at all: Track A (has a source) runs the 6 fabrication-proving specialists and can only output Real, Fabricated, or Suspicious. Track B (no source) runs a light 3-step check and can only ever output Unverifiable — it never touches the specialists that would let it claim fabrication, because there's nothing to check against. The two converge into a scorecard with 3 separate, non-blendable counts. Scored against the real report: 49 Verified · 0 Hallucinated · 51 Unverifiable — the honest number, not the inflated "51 hallucinations" headline it would've been easy to overstate.

Q42 — "what shld i make to have complete steps — architecture/wireframe/roadmap"

Recommended three linked deliverables: the architecture (the fork, the 12 stages, the specialist tiers, build-vs-buy calls); wireframes for the 7 screens a real user would need — upload, the scorecard dashboard, a per-citation detail view, the review queue, governance settings, the export report, and a cross-report trend view; and a roadmap sequencing the build so the sharpest claim ships first — Track A only (Existence, Content-match, Author-verify) as the MVP, then Track B and governance, then the remaining specialists, then monitoring, then the cross-report library.

Q43 — "now make a user diagram for complete architecture including internal architectures" (three review-and-fix passes, run against my own critique each time)

Built the diagram, then deliberately audited it against everything discussed above rather than treating a first draft as done. Round 1 found real gaps: Self-citation was missing from the specialist grid entirely, despite being the check that caught Deloitte citing itself twice in the real audit. Framing-fidelity and Internal-consistency — the two most differentiated, least-built checks in the whole market scan — were also missing. The Verdict Resolver could output "Suspicious," but the Scorecard only had 3 buckets, so a Suspicious verdict had nowhere to go. Materiality was described in a text label but never actually drawn as a real signal reaching the router. And the 5 data stores from the written spec were invisible on the diagram entirely.

Fixed all of it in two more passes: added a 4th, dashed-border Pending bucket so Suspicious verdicts can't silently disappear; drew a real materiality-signal line from Stage 1 into both tracks instead of just labeling it; drew all 5 data stores as their own elements with real connectors, including the one — Source Cache — that's genuinely read twice, once at first-check and again by Monitoring, months later. Then one more correction: re-checked whether "AI hallucination" — the stated goal — actually covered fabricated numbers, fabricated authors, and self-made claims with no citation at all. It didn't. Everything in the pipeline was triggered by a citation marker existing in the first place — a bare, uncited assertion stated as flat fact was invisible to the whole system. Added a third track, Track C, branching directly off Stage 1 before the citation graph is even built, scanning all uncited text for real factual claims and labeling them a 5th bucket: Unsupported Assertion — deliberately never confused with Unverifiable, because "cited but the trail ran out" and "never even tried to cite anything" are different failures with different fixes.

Verification Gap Engine architecture diagram: a document flows through extraction and citation-graph building, then forks into three tracks. Track B (amber, no source) runs a 3-step check ending in Unverifiable. Track A (rust, has a source) runs a materiality-aware classifier dispatching 12 parallel specialists into a verdict resolver producing Real, Fabricated, or Suspicious. Track C (violet, branching off extraction directly) scans uncited body text for unsupported factual claims. All three converge into a 5-bucket scorecard — Verified, Hallucinated, Unverifiable, Pending, Unsupported — feeding materiality scoring, a dashed-border human-run escalation stage, governance policy, and output, before splitting into monitoring and a cross-report library, with five labeled data stores and a trend-data feedback loop back to the top.

The current architecture: three tracks (Hallucination-checked / Unverifiable / Never-cited), 12 pipeline stages, 12 specialists, and all 5 data stores — nothing left as a label-only fix. Download the full diagram as a PDF →

Q44 — "at what step ai hallucination is checked" / "are fabrication of numbers, authors, and self-made claims all covered"

Precise answer, not a hand-wave: eligibility is decided at the fork (only Track A citations can ever be tested for fabrication); the actual checking happens across the 12-specialist grid; but the verdict itself — the single point allowed to output "Fabricated" — is the Verdict Resolver, one step after the grid. Nowhere else in the pipeline may produce that label. On coverage: fabricated numbers (Statistics-match, Derived-calc), fabricated authors/quotes (Author-verify), and fabricated sources (Existence) were all genuinely covered — but self-made claims with no citation at all were not, until Track C closed that specific hole.

Q45 — "create wireframe for each screen"

Built deliberately low-fidelity — gray boxes, one shared sidebar shell so all screens read as one product, and every label pulled from the real Deloitte audit rather than lorem ipsum: 49 Verified · 0 Hallucinated · 51 Unverifiable · 0 Pending · — Unsupported throughout.

Wireframe of the Scorecard screen: five stat tiles for Verified, Hallucinated, Unverifiable, Pending, and Unsupported, a proportional stacked bar showing 49% verified and 51% unverifiable, a gate-status badge reading Needs Review, and a citation table with status chips and materiality ratings.

The Scorecard — the headline view, gate status and the 51% gap visible before reading a single row.

Q46 — "now check wireframe and compare with User diagram — does it cover everything?"

Ran a real gap review, not a self-congratulation pass. Found: 8 of 12 pipeline stages had screen coverage, but Stages 1–3 (Extraction, Citation Graph, the Fork itself) had no visible processing state at all. Bigger problem — because the real audit came back with 0 Hallucinated and 0 Pending, every wireframe example only ever demonstrated the boring outcomes. No screen showed what it actually looks like when the system catches a fabrication, what a human sees mid-review with confirm/override buttons, or what Track C's detail view even looks like (it needs a genuinely different layout — no source panel makes sense when nothing was ever cited). Also flagged: only 5 of 12 specialists were ever shown running in any one example, and the Review Queue had no record of past reviewer decisions, only Governance's separate policy log.

Q47 — "add all missing details"

Closed every gap from the review: added a new Processing screen showing Stages 1–4 actually running as a live checklist. Expanded Citation Detail from one example to four full verdict states — Verified (real), Fabricated (illustrative, since the real audit found none, marked clearly as such), Pending under review (with the Confirm Verified / Confirm Fabricated / Request more evidence buttons that were missing entirely), and Unsupported — Track C's own layout, with no source panel and a different, 2-item specialist set. Gave Review Queue its own decision log, separate from Governance's. Added a cache timestamp to citation examples, giving the Source Cache store — invisible until now — an actual UI presence.

Q48 — "add on all specialists in form of drop down or something in roster"

Rather than force all 12 specialists into every example (unrealistic — a typical citation only triggers 2–5), added one real, working reference: a native <details> dropdown listing the complete 12-specialist roster with what triggers each one and what it checks, linked directly from every Track A example ("5 of 12 shown — view full roster →"). It's a genuinely functional disclosure, not a static image pretending to be one — the artifact is live HTML, so it actually expands and collapses.

Wireframe of the Citation Detail screen showing all four verdict states stacked: Verified (Klaas Knot speech, matched source), Fabricated (a fake Cambridge Centre for Financial Ethics survey, with specialists failing existence and attribution checks, and confirm/overturn buttons), Pending under review (a partial-match claim with reviewer decision buttons), and Unsupported Assertion under Track C (no source panel, just a claim with no citation ever attached). Below all four, an expanded dropdown lists the full 12-specialist roster with trigger conditions and what each checks.

Citation Detail's four verdict states, plus the full 12-specialist roster reference below them. Download all 8 screens as a PDF →

Q49 — "now compare this wireframe with actual tools present in world currently" / "so our is good to be launched in marketing through Deloitte"

Benchmarked each screen against real products instead of reasoning from memory. Citation Detail's split-panel layout is close to Turnitin's Similarity Report (matched-text-left, source-right, click to filter) — a mature UX pattern borrowed from plagiarism detection, not invented. Review Queue matches Optro's (formerly AuditBoard) own description of its dashboards: "purpose-built... automatically updated in real-time... quickly see what is in their queue." Governance's threshold-alerting concept is confirmed by Workiva's GRC platform. But the Scorecard's 5-way bucket split has no real precedent — GPTZero, SwanRef, and Vectara's leaderboard all collapse to one score or rate, never distinguishing "checked and fine" from "never checked." And Monitoring's per-firm trend concept has no precedent either — Vectara's leaderboard trends rates across different models, not one publisher's own report history. The finding: this product stitches together two tool categories that have never been combined — citation-checkers' simple in/out interaction, and GRC platforms' workflow/governance machinery.

On whether it's launch-ready: no, and worth being precise about why. What's real — the problem (found by hand, on an actual report), the market gap (confirmed against real tools, not assumed), and a complete, internally consistent architecture. What's missing before an actual pitch — zero working product code, a sample size of one audited report, Track C never actually run, no cost or data-handling story, and no adoption path inside a real firm. There's a real difference between this being the strongest answer in an interview room and this being ready to send to Deloitte's leadership — right now it's clearly the former, not yet the latter.

Where this goes next

Open threads, updated as they resolve:

  • Run a real citation-fabrication check on an actual public Deloitte report myself, using the same method that caught KPMG/EY/PwC — done (Q26–Q29): 100 in-text citations, 49 bibliography entries, all 49 confirmed real, 51 unverifiable — with the exact claim behind every one of the 51 mapped out.
  • Turn the 8-step framework and the checklist into an actual product architecture — done (Q36–Q43): a 12-stage, 3-track platform architecture (Hallucination-checked / Unverifiable / Never-cited), full specialist roster, and a complete system diagram.
  • Wireframe the screens this architecture implies — done (Q45–Q48): 8 screens, all four verdict states, the full 12-specialist roster, benchmarked against real tools (Q49) — a sample size of one report and zero working code, so not launch-ready, and said so plainly.
  • Build Track C's actual scanner and run it against the same Deloitte report — the one bucket on the scorecard that's still a "—", not a number.
  • Prototype the MVP slice (Existence + Content-match + Author-verify only) against a second, different unchecked report — still on a sample size of one until that happens.
  • Both of the above are still open — this environment's network access to the source PDF is currently blocked, so neither could be run for real yet. Meanwhile the build side kept moving — the data-handling question, the actual design and prototype, a landing page, and a full screen-by-screen completeness pass: see Part 3: Data, Design & the Full Screen Set.

Quick recall

SectionSummary
ProblemPart 1 diagnosed the gap; this part asks whether it's really there — sourced a real, unflagged, public Deloitte report and ran the citation check by hand instead of trusting the pattern found in others' incidents.
Research stepsAudited the report structurally — 100 in-text citations vs. 49 bibliography entries (Q26) → built an 8-step verification method, then expanded it to 16 failure modes across 5 tiers (Q28, Q31) → scanned the market and found no tool stitches structural+semantic+provenance+governance together (Q32–Q35) → corrected scope from "hallucination" to a precise 3-way split after catching the risk of overclaiming (Q40–Q41) → designed and audited a 12-stage, 3-track architecture through 3 self-review passes (Q39, Q43) → wireframed 8 screens and benchmarked them against real products (Q45–Q49).
SolutionThe Verification Gap Engine's 12-stage, 3-track pipeline (Hallucination-checked / Unverifiable / Never-cited), a 12-specialist roster triggered selectively per citation, and 8 wireframed screens — all grounded in the real audit's actual numbers, not placeholder data.
Conclusion49 Verified · 0 Hallucinated · 51 Unverifiable — the honest result, not the inflated "51 hallucinations" headline it would've been easy to claim. The architecture is real and benchmarked against actual competing tools, but it's a sample size of one report with zero working code — the strongest answer in an interview room, not yet something to pitch to a real client.

Which feature is used at which step

FeatureUsed at
Citation Graph Builder (Stage 2)Right after extraction — links every marker to its claim and bibliography entry, or null; the single stage that alone surfaced the real 49/51 split
Track A / B forkRight after the citation graph — separates "has a source, testable for fabrication" from "no source, stays Unverifiable," so the system can never overclaim hallucination where the honest label is unverifiable
12-specialist roster + routerClassification & Routing (Stage 3) through Specialist Verification (Stage 4) — only the specialists a given citation type actually needs ever fire
Materiality scoringStage 5 — decides how deep the specialists go (headline stat vs. background aside), not which ones run
Escalation / Review QueueStage 7 — routes anything uncertain to a human, with evidence pre-assembled, instead of auto-closing it
Governance Policy EngineStage 8, before Output — pure rules deciding publish/hold/review against a firm's configured coverage threshold
Post-Publication Monitoring + Cross-Report LibraryStages 10–11 — re-checks currency/existence on a schedule after a report ships, and surfaces patterns across a firm's whole publication history
T35c

The Verification Gap — Data, Design & the Full Screen Set

View the files on GitHub →

Closed the confidentiality gap first — Patronus Lynx and Vectara both confirmed self-hostable, so a draft report never has to leave the firm's network A full plain-language data map: what's uploaded, what stays in-house, what actually calls out Built in the right order: wireframe → static design → interactive prototype → landing page A completeness pass across every screen — navigation, buttons, sections, and the gaps still open

Status: where the architecture and wireframes from Part 2 got the two things they didn't have yet — a real answer to "what happens to the data," and screens built past gray boxes. Picks up at Q50.

Q50 — "No data-handling story. This tool would ingest confidential draft reports and call out to third-party APIs (Patronus, Vectara) — a real firm will ask about that before anything else, and there's no answer yet"

A real, self-flagged gap, not a prompted one — every screen up to this point assumed data access without ever saying where the data goes. Checked whether the two AI specialists the architecture already leans on can actually run privately: Patronus Lynx is fully open-source (weights on HuggingFace, runnable via vLLM or Ollama inside a firm's own infrastructure), and Vectara offers on-prem/VPC/air-gapped private deployment. Both confirmed by search, not assumed. That single fact is what unlocks a real answer: the specialists that need to read the confidential draft text can be run inside the firm's own network instead of a public multi-tenant API.

Show more: Q51–Q56 (the full data-flow map and 3rd-party touchpoints, the MVP path — design, prototype, landing page — and the full screen-by-screen completeness audit)

Q51 — "i want to know in detail about data... where we need to input, where we will get output, from where we will fetch, where third party api involved, where api to be created etc as a layman person" / "where do we need 3rd party apis"

Walked the whole pipeline stage by stage, in plain language, and landed on only four points where anything ever needs to leave the firm's own systems — everything else (upload, extraction, the citation graph, the scorecard, the export) stays entirely in-house:

TouchpointWhat goes outWhy it has to
Existence checkOnly the citation's public URL/metadataHas to ask the open web whether that source actually resolves — nothing confidential in the request
Content-match & Framing-fidelityThe draft's own sentence, to be compared against the source's sentenceThe one place actual draft text leaves the parsing layer — routed to a self-hosted Patronus Lynx or Vectara instance, never a public shared endpoint
Author-verificationOnly the quoted name, date, and venueCross-checked against a public speech/quote or news record — not the surrounding confidential paragraph
Legal-citationOnly the law/regulation citation stringA completely separate registry (Shepard's-style) — a different specialty API from the other three

The layman version of the answer: everything about the report you're auditing stays on your own servers by default. The only time anything goes outside is to ask a public question ("does this URL exist," "did this person really say this," "is this citation format real") — and the one touchpoint that does need to see your actual sentences runs on a copy of the model you control, not someone else's server.

Q52 — "what should i do to make mvp design prototype using the wireframe" / "make complete prompt from wireframe in detail" / "make prototype" → corrected: "first make design using wireframe than prototypr"

Answered the roadmap question the same way this whole thread had already earned the right to answer it — from the earlier wireframe pass, not from scratch: Wireframe (gray, structural — already done) → Design (static, real visual identity) → Prototype (interactive) → ship. Wrote a full build prompt translating the wireframes' 8 screens, their layout decisions, and the real Deloitte numbers into a build spec — then jumped straight to building the interactive prototype, skipping the middle stage. Caught directly: "first make design using wireframe than prototypr" — a fair, plain correction. Went back and built the missing Design stage before continuing, in the right order this time.

Q53 — the Design stage: static, full color, five screens

Built on the wireframes' exact structure, but with a real visual identity for the first time — a deep-teal accent (#146370), true green/rust status colors instead of gray labels, and every number still pulled from the real audit. Five screens: Upload, Processing, Scorecard, Citation Detail — Verified, and Citation Detail — Fabricated (illustrative only, since the real audit found zero fabrications — labeled as such on the screen itself, the same discipline kept since the wireframes).

High-fidelity design of the Scorecard screen in full color: a browser-style window shows the audited report's title, page and citation counts, and a Needs Review badge, above three stat tiles reading 49 Verified, 1 Fabricated, and 51 Unverifiable, a green-to-rust proportional bar, and a two-row citation table with status pills.

The Scorecard in real color for the first time. The "1 Fabricated" tile is the illustrative example — the real audit's actual count is 0.

Q54 — the Prototype: the same four core screens, actually clickable

Same visual system as the Design stage, this time wired up with real state — Start audit on Upload triggers a live Processing animation that steps through the specialists actually running, which lands on the Scorecard, which links into Citation Detail's Confirm fabricated / Overturn — mark verified / Add note buttons. A visible trace of the earlier data-handling answer made it into the build itself: the Upload screen carries a real "Data sensitivity" field alongside report type, not just a file picker — a direct, checkable link between Q50's finding and what actually got built.

Q55 — "Create homepage style or first sign up page"

A screen that hadn't existed anywhere before this — every earlier build started at Upload, with nothing in front of it. Built as a genuine front door: a hero making the actual finding the whole pitch ("100 citations. We could only check 49."), a footnoted stat citing the audit itself, three proof-point numbers (4 of 4 Big Four firms with a fabricated-citation incident since 2024, 0 existing tools that audit a firm's own published reports end-to-end, 2 failure modes that look identical to a reader but need different fixes), a "how it works" walkthrough, and a working client-side sign-up form — work email, firm name, and report type, gated behind the same "Data sensitivity" thinking from Q50.

Landing page for the Verification Gap Engine: a hero reading '100 citations. We could only check 49.' with Request early access and See how the audit worked buttons, a scorecard preview beneath it, three proof-point stats (4 of 4, 0, 2), and a How It Works section with four steps — Upload, Check, Score, Review.

The front door the product never had until this pass — every number on it traces back to the real audit, not placeholder copy.

Q56 — the completeness pass: "what need to be on the first screen" / "how many screens" / "what each page should have like navigation bar" / "buttons" / "sections on each page" / "add up gap pages also when if something goes wrong"

A rapid, screen-by-screen audit across everything built so far, checked against the actual files rather than answered from memory:

QuestionWhat the audit found
Screen countLanding (new) + the 4-screen MVP slice (Upload, Processing, Scorecard, Citation Detail) built to design/prototype fidelity — of the 7 screens Q42 originally planned, Review Queue, Governance, Export, and Cross-report trend are still wireframe-only, not yet designed or prototyped
Navigation barA real gap, not a style choice — neither the Design stage nor the Prototype has a persistent sidebar or top nav; each screen exists as a standalone view. Flagged as unbuilt product chrome, not a deliberate minimalist decision
Buttons per pageUpload: Start audit. Citation Detail: Confirm fabricated, Overturn — mark verified, Add note. Landing: Request early access (hero + form) and See how the audit worked — every button traced to an actual element in the built files, not a hypothetical list
Sections per pageLanding breaks into 4 real sections: hero, proof-points, "how it works," and the sign-up form — each with its own job, none of them decorative
Gap/error statesIdentified, not yet built: what a screen looks like when a document fails to parse, when zero citations are found, when every specialist times out, and when a sign-up submits with no available seats. None of these exist as screens yet — a named, open gap rather than a silent one

Where this goes next

Open threads, updated as they resolve:

  • Answer the data-handling question a real firm asks first — done (Q50–Q51): a 4-touchpoint map, with the only content-reading specialist confirmed to be self-hostable.
  • Build the Design and Prototype stages, in the right order — done (Q52–Q54): 5 static screens, then the same 4 wired up interactively, corrected mid-build to insert the design stage that was skipped the first time.
  • Build the front door — a landing/sign-up page — done (Q55): a real hero, proof points, and a working sign-up form, all numbers traced to the real audit.
  • Build the 3 still-wireframe-only screens (Review Queue, Governance, Export, Cross-report trend) to the same design/prototype fidelity as the MVP slice.
  • Build the missing gap/error states named in Q56 — none of the four identified failure screens exist yet.
  • Track C's scanner and the second-report MVP test (still open from Part 2) remain blocked on this environment's network access to a source document.

Quick recall

SectionSummary
ProblemPart 2's architecture and wireframes still had two unanswered questions a real firm asks first: what happens to the confidential draft data, and does anything exist past gray-box wireframes.
Research stepsChecked whether the two AI specialists the architecture already needed could run privately — confirmed Patronus Lynx is open-source and Vectara offers on-prem/VPC deployment (Q50) → walked the entire pipeline stage by stage and found only 4 points where anything ever needs to leave the firm's network (Q51) → built Design (5 static, full-color screens) after self-correcting a skipped step (Q52–Q53) → built an interactive Prototype wired with real state (Q54) → built a landing/sign-up page that hadn't existed before (Q55) → ran a completeness audit against the architecture and named every gap found (Q56).
SolutionA 4-touchpoint data-handling map (everything else stays in-house), 5 static design screens, a clickable prototype of the 4-screen MVP slice, and a real landing page with a working sign-up form — every number on it traced to the actual Deloitte audit, not placeholder copy.
ConclusionThe confidentiality question — the one most likely to kill the pitch on contact — now has a real, checked answer. But the self-audit (Q56) is just as important as the build: 3 of 7 planned screens are still wireframe-only, there's no persistent navigation, and 4 named failure/error states don't exist yet — named directly rather than smoothed over.
T36

Deloitte PM Role-Play — A 5-Year Execution Plan

5 phases, ~2.5–4 years, mapped team-by-team and decision-by-decision Split into what's doable as an outsider vs. only possible once hired 4 concrete plans: this week, everything outsider-doable, the full insider plan, and the gap between them Explicitly a rehearsal exercise — useful whether or not this becomes a real application

What this is: after building the research in The Verification Gap, I used it as a prompt to practice strategic PM thinking — "if I actually owned this problem at Deloitte, what's the real plan." This is explicitly a role-play/rehearsal exercise, not a claim of real Deloitte involvement or insider access. Kept verbatim for the same reason as the research log: it's more honest, and more useful as prep, than a cleaned-up summary.

Q1 — "so what should i do exactly right now? Second scenario - think I am working as PM in Deloitte working on this problem- what all steps should i plan for next two to five years in detail. Which teams shld be involved. What knowledge gap to be fulfilled? what reports to be taken out from past data and how to utilize them for current job"

Laid out five phases: Phase 0 — Diagnostic (usage audit, tool inventory, post-mortem interviews, an internal exposure scan); Phase 1 — Pilot (policy, threshold, wiring a detection tool into one practice's real workflow); Phase 2 — Scale (expand practices, certification training, a leadership dashboard, folding it into performance metrics); Phase 3 — Embed & Enforce (firm-wide mandatory gate, contract language, partner workflow); Phase 4 — Industry leadership (feedback loop, a public transparency report, evaluating productization).

Knowledge gaps flagged: Deloitte's real delivery workflow end to end, RAG/LLM evaluation literacy, regulatory literacy (EU AI Act, India's guidelines), change management, and metrics design that survives external audit.

Past-data reports to pull: prior peer-review/QA error rates as a baseline, the Australia/Canada incident timelines root-caused precisely, aiStudio's actual usage logs (was Citation Retriever ever invoked, by whom), report volume/value by practice to prioritize the pilot, client complaint/escalation history, and external benchmark data (KPMG/EY/PwC findings) as internal-urgency ammunition.

Q2 — "role-play exercise (practice thinking like a Deloitte PM, useful prep either way. Here's the 2–5 year plan I'd run if I were the PM who owned this at Deloitte,"

Confirmed as the framing going forward, and offered four ways to keep going: deepen one phase, rehearse explaining it aloud, stress-test it against a skeptical stakeholder, or leave it as reference.

Show more: Q3–Q14 (what's possible as an outsider, teams, decision rights, the full sequential plan, the four final plans with timelines, and the team-by-team roadmap)

Q3 — "check what all things are currently possible since i am not part of deloitte and what all will be possible post joining only in all phases"

Phase 0 — Diagnostic

ActivityStatus
Internal GenAI usage audit🔒 Employees-only — needs internal survey/access data
Tool inventory (aiStudio, ControlCatalyst.AI)✅ Already done — found through public press releases/PDFs
Post-mortem interviews with Australia/Canada teams🔒 Impossible from outside — only public reporting exists
Internal exposure scan (run a citation check on sample reports)✅ Fully possible right now, on real public reports — the same technique that caught KPMG/EY/PwC

Phase 1 — Pilot

ActivityStatus
Write the actual policy (disclosure, threshold, escalation)✅ Fully writable now — a PM design artifact, not something requiring employment
Wire a real detection tool into Deloitte's actual pipeline🔒 Requires internal system access and engineering resources
Prototype what that integration would look like (UI, workflow, audit trail)✅ Fully buildable — the OpsPilot pattern, re-aimed at document verification
Define success metrics (adoption %, flag rate, override rate)✅ Fully definable as a planning exercise

Phase 2 — Scale

ActivityStatus
Expand to more practice areas🔒 Requires organizational authority
Design the certification/training curriculum✅ The content design doesn't require employment, only the rollout does
Build a leadership dashboard mockup✅ Fully buildable as a prototype
Fold metrics into performance reviews🔒 Requires HR system access and organizational authority

Phase 3 — Embed & Enforce

ActivityStatus
Firm-wide mandatory gate🔒 Requires real authority over delivery workflow
Legal/contract language for clients🔒 Real version needs actual Legal and real contracts — but ✅ sample clause language is draftable as a demonstration
Partner sign-off workflow integration🔒 Requires access to real internal systems

Phase 4 — Industry leadership

ActivityStatus
Feed real override data back into the model🔒 Requires real production data that doesn't exist outside the company
Publish an actual company transparency report🔒 Requires authority to speak for Deloitte
Draft what that transparency report would say✅ A mock version is writable as a portfolio artifact
Track regulatory shifts (EU AI Act, India's guidelines)✅ Already doing this — this whole research thread is exactly that work

The standout ✅ across all five phases: running a real citation-fabrication check on an actual public Deloitte report. Almost everything in Phase 0–1 skews ✅; Phase 2–4 skew increasingly 🔒 as it requires real organizational authority — a healthy signal about where "PM work" ends and "PM job" begins, not a gap in effort.

Q4 — "which all teams shld actually be involved in these phases be it designers, developers, complaince, data etc"

Phase 0 — Diagnostic

TeamRole in this phase
Quality & Risk Management (QRM)Sponsors the diagnostic — owns the mandate to ask "how bad is this really"
AI Center of ExcellenceProvides the internal usage data — who's using Azure OpenAI/Copilot, how
Data/AnalyticsBuilds the actual usage-audit query across internal systems
Data Science/ML engineersRun the citation-fabrication scan on sample reports
Information SecurityClears any tool used to scan client documents — a real data-handling decision
Delivery leads (Australia/Canada engagements)The actual post-mortem interviews — without them, the diagnosis is blind
Legal / General CounselNeeds visibility from day one — the diagnostic could surface existing liability exposure

Phase 1 — Pilot

TeamRole in this phase
Product/UX DesignerDesigns the review interface — what a flagged citation looks like, how someone approves/overrides
Backend/Platform EngineersWire the detection tool into the real document pipeline
Data EngineersBuild the audit-trail data model — who checked what, when, what the outcome was
AI/ML EngineersTune or select the detection model, calibrate the confidence threshold
QRMOwns the policy document — what "mandatory" means, what happens on override
LegalReviews the escalation policy for liability implications
Practice lead (pilot practice)Sponsors the pilot inside their team, absorbs the "this slows us down" pushback
Delivery teams in the pilotThe actual users — their friction reports are the real signal
ProcurementNegotiates and contracts an external tool, if licensing one

Phase 2 — Scale

TeamRole in this phase
Learning & Development (L&D)Builds and delivers the certification training — real instructional design
Data/AnalyticsBuilds the leadership dashboard — flag rate, override rate, trends by practice
Product/UX DesignerDesigns that dashboard so leadership can actually act on it
HRFolds override-rate/compliance metrics into performance review criteria
Practice leads (expanded set)Each new practice needs its own sponsor doing the same internal selling job
Internal CommunicationsRolling out a mandatory process to a much larger population needs real change-comms

Phase 3 — Embed & Enforce

TeamRole in this phase
Legal / ContractsWrites the actual client-facing contract language disclosing the process
Backend EngineersBuild the hard technical gate — a report can't be marked "ready to send" without the check
Partners (client-facing leadership)Formal sign-off authority — this phase doesn't happen without partner-level buy-in
QRMEnforces the mandate — "recommended" becomes "required"
Sales / Business DevelopmentStarts using the process as a competitive point against EY/KPMG/PwC
Compliance / Internal AuditIndependently checks enforcement is actually happening, not just declared

Phase 4 — Industry leadership

TeamRole in this phase
Data ScienceBuilds the feedback loop — using override data to improve detection accuracy
Corporate Communications / PRDrafts and manages the public transparency report
LegalReviews everything before it's public — real legal risk in a report about a past failure
Product/Strategy teamEvaluates whether to productize the internal tool and sell it externally
Regulatory Affairs / Policy teamTracks EU AI Act enforcement, India's guidelines, FINRA-style guidance
FinanceOwns the budget case if this becomes an external product

Noted pattern: Phase 0–1 skews technical/data/design; Phase 2–3 shifts to organizational roles (HR, Legal, partners) as it moves from pilot to mandate; Phase 4 shifts again toward strategic/external roles as it goes from internal fix to market position.

Q5 — "how PM work in each phase along with them"

Phase 0 — Diagnostic: the PM isn't building yet, just turning scattered signals into one decision-ready picture. With Data/Analytics: define the actual question (not "how many people use AI" but which practices, weighted by risk), then get it pulled. With Data Science/ML engineers: scope what "done" looks like for the exposure scan, then translate their technical output into a risk narrative leadership can act on. With delivery leads: run the post-mortem interviews personally — structured discovery, not delegated. With Legal/InfoSec: loop in early to flag landmines before they become blockers.

Phase 1 — Pilot: with the Designer, write the requirements, critique the design against a reviewer's real workflow under deadline pressure. With Backend Engineers, write the integration spec and negotiate feasibility, cutting scope yourself when needed. With Data Engineers, define exactly what the audit trail needs to capture so it's usable evidence later. With AI/ML engineers — the big one — don't set the confidence threshold alone or hand it fully to engineering either; understand the tradeoff curve together and own the number and its consequence. With QRM/Legal, write the first policy draft yourself rather than waiting for Legal to author it. With the practice lead and delivery teams, run the pilot rollout personally, collecting friction weekly. With Procurement, do the actual vendor bake-off yourself.

Phase 2 — Scale: with L&D, define the learning objectives, not the training content itself. With Data/Design on the dashboard, define what decision a QRM leader makes differently because of it, before anyone builds anything. With HR, make the case with data — you influence this lever, you don't own it. With each new practice lead, adapt the pitch to their specific risk profile rather than reusing one deck. With Internal Comms, write the core narrative; they own the channels and cadence.

Phase 3 — Embed & Enforce: with Legal/Contracts, supply the plain-language description of what the process does; they turn it into binding language. With Backend Engineers, define the hard-gate requirement and work through real edge cases (emergency overrides) together. With Partners — the hardest conversation in the whole plan — translate this into their language: risk avoided, not process added, usually one-on-one, not a group memo. With Sales/BD, supply the real flag-rate and override-rate trend data as ammunition. With Compliance/Internal Audit, welcome being audited without getting defensive.

Phase 4 — Industry leadership: with Data Science, define what "improving the model" should optimize for, based on real complaints collected over the previous phases. With Corporate Comms/PR, supply the honest data and actively guard against over-claiming — you know the real limitations better than anyone. With Strategy leadership, make the actual build-vs-sell recommendation, backed by years of real data, not a guess.

The pattern that repeats every phase: the PM sits between two people who each see half the problem, and makes the call neither owns alone — that's the actual, portable answer to "what does a PM even do here that the specialists don't already do."

Q6 — "who is decision maker in each phase"

PhaseReal decision-makerPM's actual authority
0Head of QRM (whether to investigate); GC (how findings are documented)None over go/no-go — leverage is the memo's quality
1Chief Risk Officer (threshold); Practice Managing Partner (opt-in); QRM+Legal (policy)Owns the pilot's design, not the permission to run it
2Each Practice's own Managing Partner; Chief People Officer (HR metrics)Repeats the sell per practice; supplies data to HR, doesn't own the lever
3Executive Committee / CEO (firm-wide mandate — above QRM's own authority)Builds the case; decision genuinely leaves your usual sponsor's authority
4CEO + GC + Comms (public report); Chief Strategy Officer (productization)Supplies the honest data; doesn't decide what becomes public

The pattern: your decision authority stays roughly constant in scope while the stakes around you keep rising, so your relative power shrinks every phase even as the work gets no less rigorous — the real skill on display is building recommendations good enough that people with more authority keep saying yes to them.

Q7 — "once everything is approved in these reasearches - what should be next steps"

Step 1 — decide what this research is actually for. If it's becoming a portfolio project, the buildable slice is Phase 0–1, not the full 5-year roadmap. If it's interview/application prep, the deliverable is a tight, spoken 90-second version, not the full document. If both (the likely right call), the portfolio project becomes the interview prep — walking in having built the thing, not just described it.

Step 2 — do the one real, non-hypothetical thing first. Run the citation-fabrication check on an actual public Deloitte report. Everything else in this research is analysis of other people's findings; this would be a finding of your own — the highest-leverage action still sitting unstarted.

Step 3 — turn the diagnostic memo into something real. The Phase 0 content already exists (thesis, sourced pattern, the aiStudio irony, the KPMG/EY/PwC pattern) — package it as the actual one-page artifact a real PM would produce, written as if going to a QRM sponsor tomorrow.

Step 4 — scope the buildable prototype. Once the memo exists, decide exactly what Phase 1 looks like as a working demo — the reviewer screen, the confidence threshold, the audit trail — same pattern as OpsPilot, aimed at document/citation verification instead of a case queue.

Step 5 — decide on the real-world application angle. Real, live Deloitte India Product Manager roles exist (Bengaluru/Hyderabad/Pune) — worth deciding whether this body of work is also meant to feed an actual application, since that changes some of the framing.

Recommended sequencing: Step 2 first (fast, genuinely novel, de-risks everything after it), then Step 3 (mostly writing, not new research), then pause and decide Steps 4–5 together once the memo and the audit findings can be seen side by side.

Q8 — "i will decide this later- first tell me thinking i am PM of Deloitte and working on these phases"

Walked through it as a lived narrative rather than another table:

Phase 0 — Diagnostic. You've just been handed this mandate after the Canada incident made leadership nervous. Your first move isn't a meeting — it's a message to the AI Center of Excellence asking for raw usage numbers, and a separate, quieter message to two people who were on the Australia and Canada engagements, asking if they'd talk to you off the record first, before anything official. Those conversations are uncomfortable. Nobody wants to relive it. You're not there to assign blame — you keep saying that, and you mean it — but you still walk out with a clear picture: the citation-check tool existed, someone knew about it, and it just wasn't part of the actual checklist anyone followed under deadline pressure. That single sentence becomes the spine of your diagnostic memo. Meanwhile Data Science runs the citation-fabrication scan on a sample of 60 recent reports. The number comes back worse than expected. You land on: "14 of 60 sampled reports contain at least one citation we could not independently verify. At current volume, that's a material, recurring exposure, not an isolated incident." That one paragraph is what gets you Phase 1.

Phase 1 — Pilot. You pick government/public-sector engagements — same domain as both scandals, easiest case to make. The practice's Managing Partner is skeptical; his exact words are something like "we already have QA, why do we need another gate." You don't argue. You show him the memo, the 14-of-60 number, and one sentence: "This gate would have caught both of the incidents that already happened." He signs off, but tells you plainly he'll pull out if it slows delivery by more than a day. Now you're in the room with an ML engineer arguing about a threshold. She wants 85% — fewer false flags, less reviewer fatigue. You push for 95%, because you've just spent three months reading two post-mortems and you know exactly what a missed citation costs. You settle on tiered thresholds by report type, which is more work to build but is the right call, and you're the one who has to justify it later if it's wrong either way. By the end of the pilot, adoption is real but not universal — about 70% of eligible reports go through the gate. You report that number honestly instead of rounding up.

Phase 2 — Scale. This phase is less building, more repeating yourself to different audiences. You give the same pitch to four more practice leads, and each time you adjust it — Tax cares about a different kind of citation risk than Consulting does, so you actually go learn enough about each practice's report types to make the pitch land, rather than reusing one slide deck everywhere. The dashboard ships. Within a month you notice one practice has a flag-override rate three times everyone else's. You don't go in accusatory — you go find out why, and it turns out their reviewers genuinely don't have time allocated for this work, so the check exists but gets rubber-stamped. That's a staffing problem disguised as a compliance problem, and it's not something you can fix with a threshold. You escalate it as a resourcing gap, not a rule-breaking problem.

Phase 3 — Embed & Enforce. This is the phase you spend more time with partners than with any technical team. Making the gate mandatory firm-wide means asking the Executive Committee to change how every partner signs off on every deliverable — and you don't get that meeting easily. It takes a champion, someone senior enough to get you the room, and weeks of making sure the business case is airtight beforehand: adoption trend, override trend going down, zero incidents since rollout, and — quietly — the fact that two more competitors have now had the identical failure publicly. The meeting itself is short. Someone asks "what does this cost us in delivery time." You have the real number ready, because you tracked it honestly the whole time instead of hoping nobody would ask. It's small enough that the room moves on. Legal takes the plain-language description you wrote and turns it into three paragraphs of contract language you barely recognize, but it says what it needs to say.

Phase 4 — Industry leadership. By now this isn't your pet project anymore — it's just how the firm works. You spend more time on the feedback loop, tightening the model against years of real override data, than on convincing anyone of anything. Comms wants to publish a transparency report and asks you to write the technical section. You catch yourself pulling one sentence back — a draft that says "we've eliminated this risk" — because you know that's not true, and you'd rather it say "we've built a system that catches this before it reaches a client, and we keep improving it," which is honest and still strong. Someone in Strategy asks if this should become a product. You say yes, but only after showing them what it actually costs to run well — because you've watched enough well-intentioned tools get sold before anyone checked whether they were profitable to support.

Q9 — "dont tell year wise- i just need answer like if this problem statement in phase 0 is solved and approved then next step will be coversing with operations team, designer, developer etc according to the issue we solving" (refined over two follow-ups — "table," then "phase wise and each problem wise" — into this final form)

PhaseProblem just solvedNext: talk toAbout what
0Diagnostic confirmed by QRMDelivery/Practice teams + AI COEAgree scope — which reports, which practice, for the pilot
1Scope agreedData Science + DesignerDetection approach; what a flagged citation looks like to a reviewer
1Detection + design existBackend EngineersFeasibility — where this sits in the real pipeline
1Buildable, confirmedQRM + LegalWrite the real policy — mandatory scope, threshold, override rule
1Policy draftedPractice's Managing Partner + ProcurementSign-off to run it; contract the tool
2Pilot proven with real dataAdditional Practice LeadsExpand, using real pilot numbers as proof
2Expansion agreedL&DBuild the certification training
2Training liveData/Analytics + DesignerBuild the leadership dashboard
2Dashboard reveals ignored flagsHRFold compliance into performance metrics
3Enough evidence existsExecutive Committee sponsorPush for a firm-wide mandatory gate
3Mandate approvedLegal/ContractsWrite it into client agreements
3Contracts updatedBackend Engineers + PartnersBuild the literal hard stop; change sign-off workflow
4Hard gate live and stableData ScienceFeed override data back to improve accuracy
4Feedback loop shows improvementComms + Legal + Strategy/FinancePublic transparency report; productization decision

Q10 — "now make four plans- one that in this week as a PM outside deloitte - I should do, second include all things in the phases as discussed can be done even if not part of deloitte. Third plan as a PM of Deloitte to complete all phases. FOurth- tell me what all things already covered by non PM of Deloitte and left for the time I become PM of deloitte."

Plan 1 — this week, as a PM outside Deloitte:

WhenAction
Day 1–2Actually run a citation-fabrication check on 2–3 real, public Deloitte "thought leadership" reports (same method GPTZero used on KPMG/EY/PwC); document what's found with evidence
Day 2–3Write the Phase 0 diagnostic memo — one page, synthesizing the aiStudio irony, the all-four-Big-Four pattern, and the citation-check findings into one decision-ready document
Day 3–4Draft the actual Phase 1 policy — disclosure rule, threshold framework, escalation path — as a real written artifact
Day 4–5Sketch the reviewer-screen concept — what a flagged citation looks like, what actions a reviewer can take
Day 5Shortlist 2–3 real, currently open Deloitte India PM roles (Bengaluru/Hyderabad/Pune) this work would actually speak to
Day 6–7Consolidate everything into one portfolio-ready outline, ready to act on

Plan 2 — everything across all phases doable without being a Deloitte employee:

PhaseWhat's fully doable as an outsider
0Tool inventory (done) · regulatory tracking (ongoing) · a real citation-fabrication scan on public reports · the diagnostic memo itself
1The written policy · a working prototype of the reviewer screen + audit trail · a defined success-metrics framework
2A training curriculum outline · a dashboard mockup
3Sample contract clause language, written as an illustrative exercise
4A mock transparency report draft · continued regulatory/industry tracking

Plan 3 — the full plan to complete all phases as an actual Deloitte PM:

PhaseWhat actually gets completed
0Real internal usage-audit data pulled · real post-mortem interviews with the Australia/Canada teams · diagnostic memo validated against real numbers · QRM sponsorship secured
1Threshold set with real risk-tolerance input from the CRO · tool wired into the actual production pipeline · a real pilot with real adoption/flag-rate data · policy formally signed off by QRM + Legal
2Expansion into additional practices, each sold individually · training actually delivered · dashboard connected to live data · performance-review metrics changed with real HR access
3Executive Committee approval for a firm-wide mandatory gate · real contract language live in client agreements · a literal hard stop built into the delivery system · partner workflows actually changed
4Real override data feeding a real improvement loop · an actual public transparency report published under Deloitte's name · a real go/no-go decision on productizing, backed by real cost data

Plan 4 — what's already covered outside Deloitte vs. what's genuinely left for after joining:

PhaseAlready covered as an outsiderOnly possible once actually the PM
0Tool inventory, regulatory tracking, a real citation-audit, the diagnostic memo draftReal internal usage numbers · real post-mortem interviews · actual QRM sponsorship
1The written policy, a working prototype, a metrics frameworkReal threshold sign-off from an actual CRO · real pipeline integration · a real pilot with real people and deadlines
2Training curriculum outline, dashboard mockupActually training real staff · real HR system changes · real multi-practice adoption data
3Sample/illustrative contract languageReal Executive Committee approval · real binding contracts · a real hard-enforced gate · real partner behavior change
4A mock transparency reportA real published transparency report · real productization decision with real financials

The honest bottom line: Plan 2 covers the thinking and design layer of every phase — genuinely real, genuinely buildable. Plan 4's right column is where it stops being design and starts requiring actual organizational authority, actual people, and actual data that only exists inside the building.

Q11 — "now tell me genuine tentative timeline for plan 2 shld be kept"

ItemTentative duration
Tool inventory / regulatory trackingDone / continuous
Real citation-fabrication scan (2–3 reports, done properly)~3 weeks
Diagnostic memo~1.5 weeks
Written policy + metrics framework~2 weeks
Working prototype (reviewer screen + audit trail)~4–5 weeks
Training curriculum outline + dashboard mockup (parallel)~2 weeks each
Sample contract language + mock transparency report<1 week / ~1 week

Sequential: ~16–17 weeks (~4 months). With realistic parallelization: ~10–12 weeks (~2.5–3 months). Flagged two items not worth compressing further regardless of time pressure: the citation scan and the prototype — both lose their entire value if rushed.

Q12 — "plan 3" (same treatment, for the full insider plan)

PhaseRealistic durationReal bottleneck
0 — Diagnostic3–4 monthsSecuring sponsorship; scheduling sensitive post-mortem interviews
1 — Pilot6–9 monthsLegal review cycles; real production-pipeline integration; running long enough for real data
2 — Scale8–12 monthsHR's annual review cycle — stuck on a calendar date regardless of effort
3 — Embed & Enforce9–14 monthsExecutive Committee's own meeting cadence; contract renewal cycles; partner behavior takes months to actually stick
4 — Industry leadership12–18 monthsNeeds a real year of accumulated data before the feedback loop is trustworthy

Sequential total: ~3–4.5 years. With realistic overlap: ~2.5–3.5 years. Plan 2 (you, alone): ~3 months, gated by your hours. Plan 3 (as an employee): ~2.5–4 years, gated by other people's calendars and committees — roughly a 10x gap, and it's not about effort, it's the real difference between building something and getting a large organization to adopt it.

Q13 — "make roadmap for each team including all phases"

Flipped the axis — one row per team, showing its complete arc across all five phases:

TeamPhase 0Phase 1Phase 2Phase 3Phase 4
QRMSecures sponsorship, oversees diagnosticOwns policy draft, ongoing oversightContinues oversight as scope expandsBuilds the Executive Committee business case—
Legal / General CounselReviews diagnostic memoReviews the policy, multiple rounds—Rolls verification into client contractsReviews the transparency report
AI Center of ExcellenceProvides usage data and tool context————
Information SecurityClears the scanning tool————
Data / AnalyticsBuilds the usage-audit query—Builds the leadership dashboard with Design——
Data Science / ML EngineersRuns the citation-fabrication scanCalibrates the confidence threshold——Owns the feedback loop — 12+ months of real data
Delivery Teams / Engagement LeadsSit for post-mortem interviewsActually run the pilot———
Product / UX Designer—Designs the reviewer screenDesigns the dashboard with Data——
Backend / Platform Engineering—Real pipeline integration — longest item in Phase 1—Builds the hard enforcement gate—
Data Engineers—Builds the audit-trail data model———
Procurement—Negotiates vendor contract if licensing———
Practice Leads / Managing Partners—Pilot practice sign-offSell to each additional practice, staggered——
L&D——Designs and delivers certification training——
Internal Communications——Manages rollout comms per wave——
HR——Changes performance-review criteria — the real bottleneck——
Executive Committee———Agenda slot + decision—
Partners (broader, client-facing)———Actual behavior change — 24+ weeks of reinforcement—
Sales / Business Development———Folds proof points into pitches—
Compliance / Internal Audit———First independent audit—
Corporate Comms / PR————Drafts the public transparency report
Strategy / Product Leadership————Evaluates productization
Finance————Builds the ROI case
Regulatory AffairsStarts tracking from day oneContinuousContinuousContinuousFeeds into the productization call

What this view makes obvious that the phase-view didn't: Regulatory Affairs is the only team with zero gaps, active from Phase 0 straight through Phase 4 — literally the function I've been personally performing this whole research thread. QRM and Legal are the backbone, touching four of five phases each. Data Science has the strangest arc — heavy in Phase 0, silent through Phases 1–3, then back for all of Phase 4, a real staffing risk if that team scatters to other projects in between. Six teams (AI COE, InfoSec, Data Engineers, Procurement, Sales, Finance) appear in exactly one phase each — the easiest stakeholders to manage precisely because their ask is bounded.

Q14 — "now plan 4" (the same genuine-timeline treatment, applied to Plan 4 — the integrated real-world timeline)

Stage 1 — now, as an outsider (already covered by Plan 2): ~3 months, starting today. Tool inventory, regulatory tracking, the citation-audit, the diagnostic memo, the written policy, the working prototype, the training outline, the dashboard mockup, sample contract language, the mock transparency report — all done or in motion regardless of employment status.

Stage 2 — the honest gap in between: unknown duration. Applying, interviewing, getting an offer, onboarding. Genuinely not mine or yours to control, so no fake number gets attached to it.

Stage 3 — once actually hired: the leftover (Plan 4) items, with real timing. Everything below only starts counting from Day 1 as an employed PM:

PhaseLeft for after joiningWhen it happens post-hire
0Real internal usage-audit numbers · real post-mortem interviews · actual QRM sponsorship securedMonths 1–4
1Real threshold sign-off from the actual CRO · real pipeline integration · a real pilot with real deadlines and real peopleMonths 4–13
2Real training delivery to real staff · real HR system changes · real multi-practice adoption dataMonths 13–25
3Real Executive Committee approval · real binding contracts · a real enforced hard gate · real partner behavior changeMonths 25–39
4A real published transparency report · a real productization decision with real financialsMonths 39–57

The one number worth keeping: the outsider work doesn't get thrown away once hired — it compresses Stage 3's start. Walking in with an already-written policy draft, an already-built prototype, and an already-run citation audit means Phase 0–1 likely move faster than the table above suggests, because it's revising real work instead of starting from a blank page.

Quick recall

SectionSummary
ProblemThe Verification Gap research diagnoses a real gap but doesn't say how it would actually get adopted inside a company. This rehearsal exercise answers: "if I actually owned this problem at Deloitte, what's the real 2–5 year plan" — explicitly a role-play, not a claim of real Deloitte involvement.
Research stepsLaid out 5 phases (Diagnostic → Pilot → Scale → Embed & Enforce → Industry leadership) with teams and knowledge gaps (Q1) → split every activity into outsider-doable vs. employee-only, phase by phase (Q3–Q4) → mapped who actually makes each decision vs. what a PM only influences (Q6) → walked it as a lived narrative, not just tables (Q8) → built 4 concrete plans (this week / everything outsider-doable / the full insider plan / the gap between them) with realistic timelines (Q10–Q12) → flipped the view to one row per team across all 5 phases (Q13) → merged outsider and insider work into one integrated timeline (Q14).
Solution4 concrete, sequenced plans — a 1-week starter plan, everything buildable without being a Deloitte employee (~3 months), the full insider execution plan (~2.5–4 years), and an honest map of exactly where outsider work stops and real organizational authority has to take over.
ConclusionPlan 2 (alone, ~3 months) and Plan 3 (as an employee, ~2.5–4 years) differ by roughly 10x — not from less effort, but because Plan 3 is gated by other people's calendars, committees, and budget cycles. The one number worth keeping: outsider work isn't wasted — it compresses the insider timeline's start, since Phase 0–1 become revision work instead of a blank page.

Which plan item is used at which step

DeliverableUsed at
Diagnostic memoPhase 0 — the one document that has to exist before anyone will sponsor a pilot; built outsider-doable (Plan 2), then validated against real numbers once hired (Plan 3)
Written policy + metrics frameworkPhase 1 — defines what "mandatory" means and what the confidence threshold is, before any engineering work starts
Working prototype (reviewer screen + audit trail)Phase 1 — the artifact a Practice Managing Partner actually signs off on, not a slide deck
Training curriculum + dashboard mockupPhase 2 — rolled out once the pilot has real adoption/flag-rate data to justify expansion
Sample contract languagePhase 3 — the illustrative version Legal turns into real binding text once the firm-wide gate is approved
Mock transparency reportPhase 4 — the draft Comms would adapt once there's a year of real override data to report honestly
T43

Travel Problem Radar — Travel Problems Turned Into Product Ideas

Searches the web for travel problems, sorts them into 4 categories, flags what is new, and attaches a product opportunity to each Public demo replays a saved 8-item sample: no AI calls, no key, no cost Live page is read-only: a server job sweeps every 5 hours, and visitors never trigger an AI call 68 server tests and 19 repo checks, all offline, run on every push Not yet measured: whether what the AI finds is accurate. No real sweep has been scored

Tools used: React (one component, three ways to run), Claude with its web-search tool, Vercel Functions, Cron and Blob storage, GitHub Actions (the 5-hour schedule), GitHub Pages, esbuild. Optional Slack and email (Resend) delivery of each digest. Product owner; built with AI (Claude).

Why This Project

The best product ideas in travel start as someone's bad day: a refund that never came, a visa fee that quietly quadrupled, a booking that vanished after payment. Those signals are scattered across news, government notices, outage trackers and review sites, and no PM has time to read all of it every day. The radar is a standing feed that finds the problems, sorts them and suggests the "so what" before the morning starts.

Key Decisions and Tradeoffs

  • Four category calls instead of one big prompt. Gave up a single ranked list. Got parallel calls, and one failed category no longer sinks the other three.
  • Visitors never trigger an AI call. The live page reads a sweep saved by a server job, so cost does not grow with traffic and the API key stays on the server.
  • The public demo replays a saved sample. Anyone can open it with no key and nothing that can break; the trade-off is that the sample is from July 2026.
  • "New" means the headline has not been seen before. A simple rule that can be tested; the same event under a new headline will show as New again.

What Went Wrong, and What Is Still Unproven

The first production sweep saved items from only 1 of 4 categories. The cause was an output token limit (the model's thinking counts toward it). It is fixed, and a local run returned all four categories. I have not yet confirmed it with a scheduled production run. The bigger open risk is that no real sweep has been scored for accuracy, so the page tells readers to check each source link. A draft scoring rubric is in the repo; it has not been run.

Quick recall

SectionSummary
ProblemTravel problems are scattered across news, government notices, outage trackers and review sites, so a PM finds the ones worth solving late or not at all
SolutionA sweep of four categories with web search, a product opportunity on every item, a demo that replays a sample, and a live page refreshed by a server job
ResultWorking demo and live page; 68 server tests and 19 repo checks pass offline; a token-limit failure found in the first production sweep and fixed
Not yet measuredAccuracy of the AI's findings, and whether anyone besides me finds the opportunities useful

Metrics & Analytics

// KPI design, dashboards, and the metric tree behind them
T40

Merchant Payment Operations & Dispute Hub

Metrics & Analytics

View the files on GitHub →

17 real artifacts across 7 stages, discovery to post-launch 47-item backlog across 9 epics (252 MVP points); 24 requirements traced to 62 test cases + 9 compliance controls 3-scenario business case: base NPV $149k, conservative ($839k), optimistic $1.34M Simulation project — fictional merchants, figures and teams; not an actual client system

Tools used: Excel (live-formula workbooks), Word, PowerPoint, HTML (wireframes + interactive SQL lab), SQLite. Built with AI assistance for drafting, the same way I'd use it on real client work.

The Problem (Simulated)

Merchants handle card disputes by email and spreadsheets: deadlines tracked by hand, evidence emailed to dispute operations and re-keyed, nobody able to see where a dispute actually stands. In the simulated baseline: 78% of merchant responses arrive on time, and 9% of closed disputes are lost only because the deadline passed. Dispute operations spends 42 minutes handling each dispute, with 100% of evidence arriving by email — including card numbers sitting in attachments, a PCI DSS exposure. Finance and Operations calculate win rate differently: 40% vs. 62.5% on the same disputes.

The Solution

One portal where merchants see every dispute and its deadline, get reminders, upload evidence with card numbers auto-redacted, and understand how disputes affect their payouts. Behind it: processor integration, a prioritized ops work queue, and one agreed set of dispute numbers. Beyond the MVP, the roadmap differentiates through prevention and explainable AI — pre-dispute alerts, dispute-ratio health, an AI evidence copilot with human review, friendly-fraud evidence automation, and readiness for purchases made by AI shopping agents.

How I Approached It — 7 Stages

StageWhat I did
1. DiscoverBenchmarked American Express, Adyen, Stripe and Razorpay; logged findings, gaps and decisions with sources and status.
2. DefineWrote the PRD; built a 47-item backlog across 9 epics (252 MVP points); traced 24 requirements to 62 test cases and 9 compliance controls; defined 21 KPIs with outcome definitions.
3. DesignCreated 13 annotated wireframes: 8 MVP screens and 5 concept screens, with rules, edge states and open questions.
4. Plan & governOutcome-based roadmap (30 initiatives), RAID log, change control with impact assessments, stakeholder map, RACI and governance calendar.
5. Justify & launchThree-scenario business case, pilot-led go-to-market with KPI gates, launch readiness, hypercare, benefits tracking and BAU transition.
6. Communicate17-slide executive review ending in five explicit decisions.
7. MeasureSQL practice database with KPI, reconciliation and data-quality queries, including deliberately planted data problems.

Decisions Worth Discussing

  • Resolving a stakeholder conflict without escalating or giving in. The Fraud Operations Lead wanted full fraud case management in the MVP. I proposed a read-only fraud case reference in the MVP with a Phase 2 commitment (CR-01, Option C).
  • Fixing a metric before reporting it. Finance and Ops disagreed on win rate. The proposed definition reports Won, Partially won and Lost separately, with value recovery as the North Star (CR-05).
  • Tracing one risk end to end. Stale deadlines (R-03) → webhook assumption (A-02) → sandbox issue (I-01) → blocked story (US-019) → mapping defect (DEF-03) → failing requirement (BR-06) → latency KPI (KPI-12) → incident playbook (PB-02).
  • Showing a business case that can lose money. Base case NPV is $149k with a 2.3-year payback, but the conservative case is ($839k). That points leadership at the two assumptions that matter: merchant adoption and whether released ops capacity is cashable.
  • Exiting hypercare on evidence, not a date. Twelve exit criteria, including 10 business days without a Sev 1 or Sev 2 incident.
  • Differentiating in a crowded market. Online dispute portals are standard, including at American Express, so the roadmap competes on prevention and explainable AI rather than the portal itself.

Business Case (Base Scenario, Illustrative)

MeasureValue
One-time investment$612k
3-year NPV$149k (conservative ($839k), optimistic $1.34M)
3-year ROI26%
Payback after go-live2.3 years
Share of benefit from released ops capacity~85%
Merchant value at Year 3 (excluded from NPV)~$771k a year

Known Gaps & Next Steps

  • The Post-Launch Plan's iteration roadmap doesn't yet include the six beyond-MVP initiatives (RM-25 to RM-30) added to the Product Roadmap.
  • Concept screens S09–S13 are directional; each needs discovery, compliance review and a business case before it's real.
  • Next artifacts: API specification with a Postman collection, analytics tracking plan, and a BI dashboard mockup.
T5

Flight Delay Prediction Dashboard

View the files on GitHub →

8,000 flights analyzed Self-contained HTML/JS dashboard CVD-validated palette Full dark-mode support

Tools used: Python (pandas, gamma-distribution modeling), HTML/CSS/JS.

Why Nobody Acts on This Today

An OTA or airline can't act on delay risk it hasn't aggregated. A single flight-status page tells one traveller their flight is late; nothing rolls that up into the pattern a PM actually needs: which routes are structurally unreliable, which airlines run tighter operations, and when in the calendar or the day risk spikes. This project builds that rollup, answering the three questions a travel-tech PM would actually be asked before shipping something like the "delay-risk badge" recommended in the Flight Booking Teardown (T1): which routes delay most, how airlines differ, and how season and time of day shape delay risk.

My Approach

The honest constraint first: this was built in an environment that can't reach DGCA, US BTS, or Kaggle to pull a real historical dataset. Rather than present scraped-looking numbers that weren't actually real, I built a modeled dataset instead and said so directly on the dashboard: 8,000 flights generated from parameters I chose and documented: fictional airline names (so no delay number is ever attributed to a real, identifiable carrier), real Indian city-pair routes (airport congestion at Delhi/Mumbai/ Bengaluru and effects like North Indian winter fog are public aviation-industry knowledge, not a claim about any company), and additive delay effects for route congestion, season, and time-of-day. Delay minutes are drawn from a gamma distribution, not a normal one: real delays are right-skewed with a hard floor at zero, and gamma is bounded at zero and long-tailed by construction, which a normal distribution isn't. The first pass came out at a 48.9% on-time rate (implausibly low for any real airline), so I re-tuned the distribution parameters until the fleet-wide numbers matched a plausible real-world range, while keeping the underlying patterns (winter fog, monsoon, evening cascade) intact. Full calculation, every parameter, and the reasoning behind each one are documented in the dashboard's own Methodology section, summarized here in full.

Show more: full calculation methodology, parameter tables (every number, and why)

Methodology: the calculation, step by step

For every one of the 8,000 simulated flights:

  • Draw a base delay from Gamma(shape=1.3, scale=3.2): mean ≈ 4.2 minutes, right-skewed.
  • Multiply by that flight's airline reliability index (a single fixed number per airline, see table below), the only airline-level effect in the model, kept deliberately simple so differences between the fictional carriers stay modest rather than dramatic.
  • Multiply by a time-of-day multiplier (0.90 Early Morning → 1.55 Night) to model how delays cascade later into an operating day.
  • Add a fixed route congestion term (see table below), scaled by 0.35, a stand-in for how much busier Delhi/Mumbai/Bengaluru are than smaller airports.
  • If the route is monsoon-sensitive and the month falls in Jun–Sep, add an extra Gamma(1.6, 5.0) term (mean ≈ 8 min).
  • If the route is fog-sensitive and the month is Dec/Jan, add an extra Gamma(1.8, 6.0) term (mean ≈ 10.8 min), amplified 1.3× for Early Morning departures, since fog is worst at dawn.
  • Cancellation is decided separately, as its own probability: a 0.6% base rate, +1.8pp if monsoon-exposed that month, +2.4pp if fog-exposed that month. If cancelled, delay is zeroed; a cancellation is a different outcome from "a very late flight," not the same thing at the extreme end.
  • A flight counts as delayed if its final delay exceeds 15 minutes (the standard on-time threshold) and it wasn't cancelled.
Airline (fictional)TypeReliability index
BlueSky AirFull-service0.75 (best)
IndiFlyLCC1.00
Skyline NationalFull-service1.15
Metro ExpressLCC1.25
Coastal WingsRegional/turboprop1.35 (worst)
RouteCongestion term (min)Monsoon-sensitiveFog-sensitive
DEL-BOM9—Yes
DEL-BLR7—Yes
DEL-GOI7YesYes
DEL-HYD7—Yes
DEL-CCU6—Yes
DEL-AMD6—Yes
DEL-PNQ6—Yes
BOM-BLR6——
BOM-COK5Yes—
COK-BOM5Yes—
BLR-HYD4——
MAA-CCU4——
BLR-COK4Yes—
HYD-MAA4——

Why these specific numbers: not derived from any real source, chosen and then iterated. The first draft produced an implausible 48.9% on-time rate; I scaled the base gamma parameters and congestion multiplier down until the fleet-wide rate landed at 84.4%, inside a plausible real-world range, while preserving the relative shape of every effect (which season is worst, which route is worst, night vs. morning). What this deliberately doesn't model: real distributional parameters fit from a published dataset (none was reachable from this environment), or flight-to-flight correlation (a delayed inbound aircraft delaying its own next outbound leg). A real airline operations model would need that, and it's a materially harder problem than this exercise set out to solve.

The Analysis

With the dataset built, I grouped it four ways (by route, by airline, by season, and by time of day), computing average delay, the share of flights delayed over 15 minutes, and cancellation rate for each. A single "worst route" or "worst season" number hides the risk that actually matters operationally, which is where two dimensions compound, so I cross-tabbed the top 8 routes against all 5 seasons into a heatmap instead of stopping at one-dimensional rankings.

Key Findings

  • 84.4% on-time fleet-wide, 9.2min average delay, 1.1% cancellation rate.
  • DEL-GOI is the single worst route (12.6min avg, 34% delayed >15min): the only route in the model that's both winter-fog- and monsoon-sensitive, so it takes a hit twice a year instead of once.
  • Winter is the worst season by a wide margin (11.9min avg vs. 7.5min in Spring/Post-Monsoon): every Delhi-origin route in the heatmap spikes in December–January.
  • Delays compound through the day: Night departures average 11.0min vs. 8.3min in the Early Morning, consistent with how real airline operations cascade delays leg-to-leg.
  • Airline-level spread is real but secondary: the gap between the most and least reliable modeled carrier (7.6–10.7min) is smaller than the gap between Winter and Spring (7.5–11.9min). Route and season explain more than carrier choice does.

Recommendations

What a PM would actually do with this, if it were live data instead of modeled data:

  • Surface delay risk during booking, not after. Feed a route + season + time-of-day risk score into the delay-risk badge recommended in the Flight Booking Teardown, so a traveller sees it before they book, not after they're already delayed.
  • Pad schedule buffers seasonally, not year-round. A flat on-time SLA misses that Delhi-origin routes need a materially wider buffer specifically in December–January, not every month.
  • Treat Night departures as an upstream ops problem. Since delay compounds through the day in this model, padding the last flight of the day treats a symptom; the real fix is earlier-leg reliability.
  • Re-run this against real DGCA/BTS data before shipping anything downstream of it. This dashboard's job was to prove the analysis and the product judgment, not to replace a real data pipeline.

Where the Real Risk Sits

The chart itself is rarely a delay dashboard's real product risk. The risk is presenting modeled or incomplete data as if it were ground truth. The discipline that matters here is the same one from the AI Trip Planner's guardrails: say what's real, say what's modeled, and never let a polished visualization imply more certainty than the underlying data supports.

Flight Delay Prediction Dashboard: KPI tiles (8,000 flights, 84.4% on-time, 9.2min avg delay) and a bar chart ranking all 15 routes by average delay, DEL-GOI worst at 12.6 minutes

Click the screenshot to open the live, interactive dashboard (also available in dark mode).

Quick recall

SectionSummary
ProblemA single flight-status page tells one traveller their flight is late, but nothing rolls that up into the pattern a PM actually needs — which routes are structurally unreliable, which airlines run tighter operations, and when risk spikes — the exact rollup needed before shipping a delay-risk badge like the one recommended in the MakeMyTrip teardown.
Research stepsBuilt 8,000 flights from documented parameters since no real DGCA/BTS/Kaggle dataset was reachable → chose a gamma distribution over a normal one because real delays are right-skewed with a floor at zero → the first pass came out an implausible 48.9% on-time, so re-tuned parameters until the fleet-wide numbers matched a plausible real-world range while keeping the underlying patterns intact → grouped the data 4 ways (route, airline, season, time of day) and cross-tabbed routes against seasons into a heatmap instead of stopping at one-dimensional rankings.
SolutionA self-contained, CVD-validated HTML/JS dashboard surfacing on-time rate, average delay, and cancellation rate by route/airline/season/time-of-day, with every modeling choice documented in its own Methodology section rather than presented as scraped real data.
ConclusionDEL-GOI is the worst route specifically because it's the only one exposed to both winter fog and monsoon risk — a pattern invisible in a one-dimensional ranking. The real product risk was never the chart; it's presenting modeled data as if it were ground truth, which is why every claim on the dashboard is labeled as modeled, not measured.
T2

Airline Operations Dashboard

View the files on GitHub →

6,000 flights analyzed Single fictional airline (Aravali Air) Self-contained HTML/JS dashboard Full dark-mode support Also built in Power BI: star schema + DAX

Tools used: Python (pandas), HTML/CSS/JS, Power BI (Power Query, DAX).

A Different Question Than the Delay Dashboard

The sibling Flight Delay Prediction Dashboard answers an OTA's question — which routes and airlines should get flagged as delay-risky — across five carriers. This one answers the question an airline's own ops team asks about itself: is my operation actually healthy, hub by hub and month by month, and what downstream cost does unreliability actually carry? One carrier, tracked in depth on on-time performance, load factor, revenue per flight, cancellation rate, complaints, and baggage handling, is the right shape for that question. This is also the SQL/Power BI discipline I used for PCMS reporting at Bajaj Finance, applied to a domain I hadn't worked in before — deliberately, as ETL/data-product practice.

My Approach

Same honest constraint as the sibling project: no real airline ops data is public or reachable from this environment, so this is a modeled dataset, disclosed as such on the dashboard itself. 6,000 simulated flights for one fictional carrier ("Aravali Air") across 5 real Indian hubs, with on-time performance, cancellation, load factor, revenue, complaints, and baggage mishandling each modeled with a documented, deliberate rule, not arbitrary noise: for example, complaints are weighted to rise sharply when a flight was delayed and further still when it was cancelled — a real, well-documented airline-ops pattern, and the whole reason this dataset exists rather than being five independent random columns. Full calculation, every parameter, and the reasoning behind each one are documented in the dashboard's own Methodology section, summarized here in full.

Show more: full calculation methodology, the data-quality fix I made (every number, and why)

Methodology: the calculation, step by step

For every one of the 6,000 simulated flights, in order:

  • On-time performance: starts from a base probability of 90%, reduced by that hub's congestion index (Delhi and Kolkata run busiest), then reduced further by −22pp if the hub is fog-sensitive and the month is Dec/Jan, or −14pp if monsoon-sensitive and the month is Jun–Sep.
  • Cancellation: decided separately as its own probability (base 0.6%, +hub congestion, +weather exposure). A cancelled flight is always counted as not on-time, never the reverse.
  • Load factor: drawn from a normal distribution around a base of 79%, shifted up in the Oct–Nov festive season and down in the Jan–Feb low season and at monsoon-exposed hubs during monsoon months.
  • Revenue: ticket revenue = passengers × a per-hub base fare × a seasonal fare multiplier; ancillary revenue (bags, seats, meals) added per passenger on top. Both zero for a cancelled flight.
  • Complaints: drawn from a weighted distribution that shifts hard toward more complaints when the flight was delayed, and harder still when it was cancelled — the one deliberate correlation this generator exists to demonstrate.
  • Baggage mishandling: one independent per-bag draw, rate rising with hub congestion and on any day operations went irregular (delayed or cancelled).

A data-quality decision worth naming: the dataset tracks two different passenger counts — passengers_expected (a demand estimate, computed the same way regardless of outcome) and passengers_boarded (zero for a cancelled flight, by definition). Load factor and revenue are correctly built on boarded passengers, since those only exist for flights that actually flew. My first pass used boarded passengers as the denominator for the complaints-rate metric too, which made cancelled flights show zero complaints per 1,000 passengers: the exact opposite of reality, purely because a cancelled flight boards nobody. I caught this by sanity-checking the output before building the chart, not after — the reliability-vs-complaints chart on the dashboard would otherwise have shown cancellations as the "safest" outcome for a customer, which is backwards. Complaints are rated against expected passengers instead: the people actually affected by the flight, whether it flew or not. This is the same kind of judgment call as excluding cancelled flights before averaging delay minutes in the sibling dashboard — a metric's denominator has to match what the metric is actually claiming to measure, and getting it wrong doesn't throw an error, it just silently produces a plausible-looking wrong number.

Why these specific numbers: not derived from any real source, chosen and then checked against public ballpark ranges for plausibility (domestic OTP typically 70–88%, load factor typically 75–85%, baggage mishandling typically 4–8 per 1,000 bags industry-wide). The fleet-wide numbers landed at 77.9% on-time, 78.9% load factor, and 5.4 bags mishandled per 1,000 — all inside those ranges. What this deliberately doesn't model: real distributional parameters fit from a published dataset (none was reachable from this environment), or knock-on delay (one late inbound aircraft delaying that same tail number's next departure) — a real ops model would need that, and it's a materially harder problem than this exercise scopes to.

Key Findings

  • 77.9% on-time fleet-wide, 78.9% average load factor, 1.2% cancellation rate, ₹6,38,860 average revenue per flight.
  • Bengaluru is the most reliable hub (83.3% on-time), Mumbai the least (74.6%): the two hubs with no modeled weather exposure sit at the top, the two with monsoon exposure sit at the bottom.
  • Delhi and Kolkata both collapse in Winter (63.1% and 66.9% on-time) from their own fog exposure; Mumbai and Kolkata separately collapse in Monsoon (66.0% and 67.3%) — Kolkata is the only hub that takes a hit twice a year instead of once, the same "double exposure" pattern that made DEL-GOI the worst route in the sibling delay dashboard.
  • Complaints track reliability, not price or amenities: 0.29 complaints per 1,000 passengers when on-time, 2.05 when delayed, 6.10 when cancelled — a cancelled flight generates roughly 21× the complaint volume of an on-time one, per passenger affected.
  • Revenue doesn't track on-time performance: Delhi has the worst-adjacent on-time rate but the highest revenue per flight (₹6,86,112), because fare levels are set by hub demand, not by operational reliability. Reliability and revenue are two separate levers, not one.

Recommendations

What a PM would actually do with this, if it were live ops data instead of modeled data:

  • Fund reliability improvements by the complaint cost they prevent, not just the delay minutes they save. The 21× complaint multiplier on cancellations makes a stronger budget case than "average delay went down 2 minutes" ever will.
  • Give Kolkata a dedicated irregular-ops playbook, not a shared one. It's the only hub exposed to both major seasonal risks in this model; a generic playbook sized for single-exposure hubs will under-resource it twice a year.
  • Don't let a high-revenue hub's numbers hide its reliability problem. Delhi's revenue leadership could mask its on-time performance from a purely commercial dashboard — reliability and revenue need to stay on the same page, not two separate reports someone has to cross-reference manually.
  • Re-run this against real ops data before shipping anything downstream of it. This dashboard's job was to prove the ETL pipeline and the product judgment behind each transform decision, not to replace a real data source.

Where the Real Risk Sits

The same discipline as the sibling dashboard applies here: the risk isn't a wrong chart, it's a metric whose denominator quietly doesn't measure what its label claims — exactly what the passengers_expected/passengers_boarded fix above was catching. Say what's modeled, say what's real, and check a metric's denominator before trusting what it shows, not after.

Also Built as a Power BI Report

The same analysis, built a third way: a Power BI report using Power Query (M) for the transform step and DAX for every measure, instead of pandas. A star schema with one fact table (Fact_Flights) and two dimension tables (Dim_Hub, and a Dim_Date table built with a DAX CALENDAR() function rather than imported) — the tool-native way an ETL/BI Product Owner's team would actually structure this, distinct from and complementary to the Python version above, not a replacement for it.

Built specifically for practice, not because the Python version needed a second build. Worth naming: the DAX measures landed on numbers matching the Python version almost exactly — 83.25% on-time for Bengaluru in Power BI vs. 83.3% in Python, 76.34% vs. 76.3% for Delhi — two completely independent computations agreeing. The Complaints per 1000 Pax DAX measure also carries forward the same passengers_expected denominator fix from the Python build, rather than re-discovering the same bug a second time.

Power BI Overview page: KPI cards for flights operated, on-time %, load factor, and cancellation %, a bar chart of on-time % by hub, a chronological month trend line, and a hub slicer Power BI Hub Deep-Dive page: a table of all 5 measures by hub, and a hub-by-season matrix with conditional-formatting heatmap on on-time %, Delhi Winter darkest at 63.09% Power BI Reliability and Complaints page: a column chart of complaints per 1,000 passengers by reliability bucket, cancelled flights roughly 21x the on-time rate

Power BI report screenshots — Overview, Hub Deep-Dive, and Reliability & Complaints pages. Download the .pbix file → Build guide and source data: projects/airline-operations-dashboard/power-bi/ in the repo.

Airline Operations Dashboard: KPI tiles (6,000 flights, 77.9% on-time, 78.9% load factor) and a bar chart ranking all 5 hubs by on-time performance, Bengaluru best at 83.3%, Mumbai worst at 74.6%

Click the screenshot to open the live, interactive dashboard (also available in dark mode).

Quick recall

SectionSummary
ProblemAn airline's own ops team needs to know if its operation is actually healthy — hub by hub, season by season — and what unreliability costs downstream in complaints, not just in delay minutes. No real airline ops data was reachable from this environment, so the gap had to be closed with a disclosed, rule-based model instead of pretending to real data.
Research stepsModeled 6,000 flights for one carrier across 5 real hubs, each metric built on a documented rule (congestion, fog/monsoon exposure, seasonal fare multipliers) → sanity-checked the output before charting it, which is what caught the passengers_expected/passengers_boarded bug → checked the fleet-wide numbers against public plausibility ranges → rebuilt the same analysis a second, independent way in Power BI to cross-check the first.
SolutionA self-contained HTML/JS dashboard (dark-mode supported) plus a Power BI star-schema report, both surfacing on-time %, load factor, revenue, cancellations, complaints and baggage handling together on one page instead of across separate commercial/ops reports.
ConclusionThe two independent builds landed on matching numbers (83.25% vs. 83.3% Bengaluru on-time), which is the real proof point — not the specific numbers themselves, since the data is modeled. The job this dashboard actually proves is the ETL pipeline and the judgment behind each transform decision; it explicitly should not be trusted as real ops data, and says so on the dashboard itself.
T8

Booking Funnel Analysis

5-stage funnel model Root-cause diagnosis, not just a chart 3 A/B test hypotheses

Frameworks used: 5-stage funnel model, A/B test hypothesis design — modeled analysis, no build.

What a Funnel Chart Leaves Out

A funnel chart only shows where users leave. The actual PM work is pairing each drop-off with a testable behavioral reason instead of recommending "reduce friction" everywhere. This case study models a realistic OTA flight-booking funnel end to end and does that pairing for the stage that leaks hardest.

The Funnel

Search → Results → Details (fare review) → Traveller Info → Payment → Confirmation. The biggest drop-off by far is Results → Details, a modeled ~62% loss: the fare-review price reveal is the single largest leak in the funnel. Users arrive price-anchored from the results page and bounce once taxes, fees, and add-ons inflate the total.

Funnel stageModeled conversionPrimary friction
Search → Results~92%Slow results load on heavily filtered searches
Results → Details (fare review)~38%Price at search ≠ price at fare review: biggest leak
Details → Traveller Info~61%Add-on upsell screens before core info capture
Traveller Info → Payment~74%Repeat manual entry for return-trip co-travellers
Payment → Confirmation~88%OTP/gateway timeouts on mobile networks

Root-Cause Reasoning

Three behavioral mechanisms explain most of the Results → Details leak:

  • Anchoring bias: once a user sees a low headline fare, any upward revision reads as deception, regardless of accuracy.
  • Add-on defaults: pre-selected insurance/seat fees compound the price-shock instead of being clearly separable.
  • No fare-lock: an indecisive user has no reason to stay on the page vs. reopening a new tab to compare.

Recommendations

  • Show all-in price (base + tax + fees) at the results stage, even at the cost of a slightly less eye-catching headline number.
  • Make add-ons opt-in with one clear toggle, not a pre-ticked line item.
  • Add a 10-minute fare-lock CTA on the details page for indecisive users.
  • Autofill/remember traveller info for repeat and return-trip bookings.
  • Auto-retry payment with the saved method before falling back to full re-entry on a gateway timeout.

Test Plan

  • All-in pricing: showing all-in fare at results increases Results→Details conversion by removing price-shock, possibly at a small cost to search-engine click-through.
  • Opt-in add-ons: switching seat/insurance from pre-selected to opt-in increases Details→Traveller Info conversion, even if average order value dips slightly.
  • Fare-lock: a visible "hold this fare 10 min" CTA increases session-to-booking conversion, measurable via reduced multi-tab bounce in session replay.

Success Metrics

Stage-to-stage conversion (especially Results→Details), overall search-to-confirmation conversion, average time-in-funnel, and fare-review abandonment rate segmented by add-on opt-in vs. opt-out cohort.

What This Generalizes To

A funnel chart only shows where users leave. The RICE-worthy work is pairing each drop-off with a testable behavioral reason (anchoring, default bias, form fatigue), not recommending "reduce friction" everywhere.

Quick recall

SectionSummary
ProblemA funnel chart only shows where users leave — the actual PM work is pairing each drop-off with a testable behavioral reason instead of a generic "reduce friction" recommendation everywhere.
Research stepsModeled the 5-stage OTA flight-booking funnel end to end → identified Results→Details as the biggest leak by far (~38% conversion) → diagnosed 3 specific behavioral mechanisms behind it (anchoring bias, add-on defaults, no fare-lock) rather than stopping at "price shock."
Solution5 targeted fixes (all-in pricing, opt-in add-ons, a fare-lock CTA, autofill for repeat travellers, payment auto-retry) paired with 3 specific A/B test hypotheses, each naming what metric to watch and what trade-off to expect.
ConclusionThe generalizable lesson, not just the specific fix: a funnel's real diagnostic value comes from naming the behavioral mechanism behind each drop-off, since that's what separates a testable hypothesis from a guess.
T20

Cross-industry example

The Analysis

Swiggy's order/delivery experience, with a proposed North Star metric: orders delivered on time, per active user. AARRR read on the same funnel: Acquisition — app installs and first-order conversion from restaurant/city expansion. Activation: a new user's first on-time delivery, since a late or broken first order is the single highest-leverage churn moment. Retention — repeat order rate and days-between-orders, the actual denominator the North Star metric normalizes against. Revenue: average order value plus delivery and platform fees, weighed against the subsidy cost of promotions that inflate raw order counts without improving the on-time number. Referral: organic share, which correlates far more with delivery reliability than with any discount campaign.

Metric Tree

Orders placed (acquisition, inflatable with discounts) → on-time delivery (what users actually value) → normalized per active user (removes growth-inflation noise). Leading indicator: delivery partner availability. Lagging indicators: NPS, repeat order rate. Guardrail metric: restaurant cancellation rate.

Hypotheses & Experiments

Hypothesis: on-time delivery is a stronger repeat-order predictor than price, meaning a rupee spent improving delivery-partner availability in a lagging zone outperforms the same rupee spent on a discount in that zone. Test: in a matched pair of pin codes, run a delivery-partner-availability improvement in one and an equivalent-cost discount campaign in the other; compare 30-day repeat-order rate alongside order volume, since volume alone would favor the discount and hide the actual retention effect. Expected impact: if the hypothesis holds, the availability intervention should show a smaller immediate order-volume bump but a larger 30-day repeat-rate lift — the pattern that would justify shifting budget from blanket discounting toward delivery-reliability infrastructure in underperforming zones.

Quick recall

SectionSummary
ProblemOrders placed is an inflatable, discount-driven number that doesn't actually measure what users value — a North Star metric and a growth model need to separate real reliability from promotion-driven volume.
Research stepsProposed "on-time deliveries per active user" as the North Star → ran the full funnel through AARRR, flagging activation (first on-time delivery) as the single highest-leverage churn moment → built a metric tree separating the inflatable top-line number from the normalized, actually-meaningful one → designed a matched-pair pin-code test to isolate reliability's effect on retention from a discount's effect on raw volume.
SolutionA North Star metric tree (orders placed → on-time delivery → normalized per active user) with named leading (delivery-partner availability), lagging (NPS, repeat order rate), and guardrail (restaurant cancellation rate) indicators.
ConclusionThe hypothesis worth testing: on-time delivery predicts repeat orders more strongly than price does — meaning a rupee spent on delivery-partner availability in an underperforming zone should outperform the same rupee spent on a blanket discount, measured by 30-day repeat rate, not volume alone.
T30

SQL Analytics — SaaS Subscriptions (Basic)

View the files on GitHub →

900 customers · 3 plans · 24 months 8 queries: SELECT, JOIN, GROUP BY, CASE Real local SQL Server install, not a browser sandbox Fictional company (Aptura), documented churn model

Tools used: SQL Server 2022 Express (T-SQL), SSMS (data import), VS Code with the SQL Server (mssql) extension (query writing and execution) — installed and run locally, not a hosted playground.

Why This Exists

SQL is listed as a skill across this site, but until this project the only place it was actually demonstrated was the PCMS case study — real Bajaj Finance work, which stays exactly as delivered and untouched, permanently a written account rather than something a visitor can independently verify. This project exists to prove the same skill a second way: a fresh, fictional scenario, built and queried end to end on my own machine, not something pre-generated for me to just display.

My Approach

Built a synthetic B2B SaaS company ("Aptura") from scratch: a Python generator producing 900 customers, 3 pricing plans (Starter $29 / Growth $99 / Scale $299), and 24 months of subscription history, with churn modeled on two well-documented SaaS patterns rather than independent randomness — cheaper plans churn more (weaker product lock-in) and Paid-Search-acquired customers churn more than Referral-acquired ones (referred customers arrive with higher intent). Then installed SQL Server 2022 Express and SSMS on my own Windows machine, connected VS Code's SQL Server (mssql) extension to it, created the database, imported the data, and ran all 8 queries below myself.

Real Setup, Real Errors

None of this went in cleanly on the first try, which is the actual point of doing it hands-on rather than reading a guide:

  • "Cannot open database … login failed" the first time VS Code connected to a database name that didn't exist yet — fixed by connecting to the default master database first, then running CREATE DATABASE aptura_saas directly from a query window before reconnecting.
  • "There is already an object named 'subscriptions'" from SQL Server's Import Flat File wizard, which always tries to create a brand-new table — it collided with tables I'd already created manually. Fixed by dropping the empty manually-created tables and letting the wizard build them fresh from the CSVs instead.

Key Findings

  • $63,298 current MRR across 662 active subscriptions (238 cancelled, 26.4% cumulative churn).
  • Starter churns at 34.0%, more than 3× Scale's 10.1% (Growth sits in between at 20.8%) — the cheapest plan is also the leakiest.
  • Paid Search churns at 31.6% vs. Referral's 21.5%, with Organic (26.9%) and Outbound Sales (23.7%) in between — acquisition channel predicts retention almost as strongly as plan tier does.
  • Scale carries the fewest active subscriptions (98) but the highest MRR contribution per plan (₹29,302 modeled) — plan mix matters as much as headcount when reading a subscription business.

What This Means

The churn numbers point to a retention problem that is concentrated in specific parts of Aptura's business rather than being a company-wide issue. Starter customers are churning at 34% versus just 10% on Scale, which suggests Starter customers may not be reaching enough value early enough, or that the plan is attracting customers with weaker product fit. Similarly, Paid Search has 31.6% churn versus 21.5% from Referral, indicating that acquisition source and customer intent may be affecting retention. As PM, I'd first segment these cohorts further by activation, use case, company size, onboarding completion, and time-to-value to identify the real driver rather than immediately changing pricing. I'd then test a stronger Starter onboarding and activation journey, clearer value communication, and targeted in-product nudges, while reviewing Paid Search messaging and landing-page qualification so we attract customers whose expectations match the product.

T-SQL query: SELECT COUNT(*) AS total_customers FROM customers; result: 900 T-SQL query grouping subscriptions by status; result: 662 active, 238 cancelled T-SQL query joining subscriptions to plans for current MRR; result: 63298 T-SQL query for MRR by plan; result: Scale 29302/98 subs, Growth 24948/252 subs, Starter 9048/312 subs T-SQL query for new customers by signup month using FORMAT(); result showing a growth ramp from 14 in Jan 2024 to 33 by Jul 2024 T-SQL query for customers by acquisition channel; result: Paid Search 266, Organic 253, Referral 191, Outbound Sales 190 T-SQL query for churn rate by plan using CASE WHEN; result: Starter 34.0%, Growth 20.8%, Scale 10.1% T-SQL query for churn rate by acquisition channel via a 3-table join; result: Paid Search 31.6%, Organic 26.9%, Outbound Sales 23.7%, Referral 21.5%

All 8 screenshots are my own SQL Server / VS Code session — query and result grid together, unedited.

Quick recall

SectionSummary
ProblemSQL is listed as a skill across the site, but the only place it was actually demonstrated was the PCMS case study — real Bajaj Finance work that stays untouched as a written account, with nothing a visitor can independently verify.
Research stepsBuilt a synthetic B2B SaaS dataset (900 customers, 3 plans, 24 months) with churn modeled on two documented SaaS patterns, not random noise → installed SQL Server 2022 and SSMS on an actual local machine, not a browser sandbox → hit and fixed 2 real setup errors (a login failure from connecting before the database existed, an Import Flat File naming collision) → ran all 8 queries personally and verified the result grids.
Solution8 T-SQL queries (SELECT, JOIN, GROUP BY, CASE, aggregates) surfacing current MRR, churn by plan, and churn by acquisition channel — each with its own unedited query-and-result screenshot.
ConclusionStarter plan churns at over 3× Scale's rate, and acquisition channel predicts retention almost as strongly as plan tier does — the real PM move is segmenting further by activation and time-to-value before touching pricing, not reacting to the headline churn number alone.
T31

SQL Analytics — SaaS Subscriptions (Advanced)

View the files on GitHub →

4 queries: CTEs, LAG, SUM() OVER, RANK() PARTITION BY Monthly cohort retention analysis Real T-SQL dialect bug hit and fixed Same Aptura dataset as the Basic case study

Tools used: SQL Server 2022 Express (T-SQL), VS Code with the SQL Server (mssql) extension.

Why Advanced, Not Just Basic

Aggregates and joins prove I can retrieve data; this half proves I can shape it the way a PM actually needs to — cohort retention curves, month-over-month growth, and running totals, the exact SQL patterns behind most subscription-business dashboards. Same Aptura dataset as the Basic case study (T30), four new query patterns instead of eight simpler ones.

A Real T-SQL Dialect Bug, and the Fix

These 4 queries were first written and verified against SQLite for a browser-based version of this project, then ported to T-SQL for SQL Server (FORMAT() instead of strftime(), DATEDIFF() instead of julianday()). Three of the four ported cleanly on the first run. The running-cumulative-signups query didn't:

Msg 8729, Level 16: ORDER BY list of RANGE window frame has total size of 8000 bytes. Largest size supported is 900 bytes.

The cause: FORMAT() returns nvarchar(4000) regardless of how short the actual string is, and SQL Server's default window-frame type (RANGE) caps its ORDER BY column at 900 bytes for that reason. The fix was one clause, not a rewrite — swapping the implicit RANGE frame for an explicit ROWS UNBOUNDED PRECEDING frame (the semantically correct frame for a running total anyway):

SUM(new_customers) OVER (ORDER BY month ROWS UNBOUNDED PRECEDING) AS cumulative_customers

Key Findings

  • 3- and 6-month cohort retention bounces between roughly 62% and 96% month to month, because monthly cohorts are small (14–33 signups each) — reading that as expected noise rather than over-reacting to any single month's number is itself part of the skill.
  • Month-over-month new-MRR growth swings widely for the same reason (e.g. −40.5% in March, +183.6% in May) — a small-N, growing company's month-over-month metrics are volatile by construction, not necessarily a signal.
  • Cumulative signups climb cleanly (14 → 37 → 55 → 71 → 94 across the first five months) once the running-total fix above was applied, confirming the window function now computes correctly.
  • Ranking active customers by tenure within each plan surfaces exactly who's been on a plan longest — useful for e.g. identifying who to interview for retention research, or who to invite to a beta first.

What This Means, Applied

The cohort-retention query above is what turns the Basic case study's churn-by-plan and churn-by-channel diagnosis into something actionable rather than a one-time snapshot: I'd track 30/60/90-day retention, activation rate, and churn by cohort to validate whether interventions — a stronger Starter onboarding journey, tighter Paid Search targeting — actually improve retention and lifetime value, instead of just assuming they worked.

T-SQL CTE query for monthly cohort retention; result showing 3-month and 6-month retention percentages per signup-month cohort T-SQL query using LAG() OVER for month-over-month new-MRR growth; result showing new MRR per month and percent change vs. the prior month T-SQL query using SUM() OVER with an explicit ROWS UNBOUNDED PRECEDING frame for running cumulative signups; result climbing 14, 37, 55, 71, 94 T-SQL query using RANK() OVER PARTITION BY to rank active customers by tenure within each plan; result showing the longest-tenured Growth-plan customers ranked 1 through 7

All 4 screenshots are my own SQL Server / VS Code session — query and result grid together, unedited, including the frame-fix retry on the cumulative-signups query.

Quick recall

SectionSummary
ProblemThe Basic case study proves I can retrieve data (SELECT, JOIN, GROUP BY) but not shape it the way a PM actually needs to — cohort retention curves, month-over-month growth, running totals — the SQL patterns behind most real subscription-business dashboards.
Research stepsPorted 4 queries from SQLite (a browser-based prototype) to T-SQL for SQL Server → 3 of 4 ported cleanly, but the running-cumulative-signups query hit a real dialect error (RANGE window frame's 900-byte ORDER BY cap, triggered by FORMAT()'s nvarchar(4000) return type) → diagnosed the root cause rather than guessing, and fixed it with one clause: an explicit ROWS UNBOUNDED PRECEDING frame, which is also the semantically correct frame for a running total anyway.
Solution4 T-SQL queries using CTEs, LAG(), SUM() OVER, and RANK() PARTITION BY — monthly cohort retention, month-over-month MRR growth, cumulative signups, and tenure ranking within each plan — on the same Aptura dataset as the Basic study.
ConclusionSmall-N monthly cohorts (14–33 signups each) produce genuinely volatile retention and growth numbers month to month — reading that as expected noise rather than over-reacting to any single month is itself part of the skill. This is what turns the Basic study's churn diagnosis into something trackable over time, instead of a one-time snapshot.
T32

Amplitude Analytics — Strideway (Basic)

View the files on GitHub →

3,000 users · 52,082 events, real Amplitude account 10-event tracking plan built via the Amplitude API Fictional habit app (Strideway), documented behavior model 5 segmentation charts, real screenshots

Tools used: Amplitude (tracking plan, Event Segmentation), Python + Amplitude's HTTP API V2 for data ingestion.

Why This Exists

Amplitude/Mixpanel are listed among the tools worth having, alongside SQL — but with no prior hands-on use. This project is the real thing, not a simulation: a genuine Amplitude organization (free Starter plan), a tracking plan built for real through Amplitude's own API, and real usage data sent through Amplitude's actual ingestion pipeline for a fresh scenario — "Strideway," a fictional fitness/habit-tracking app.

My Approach

First, a real tracking plan: 10 events (Sign Up, Complete Onboarding, Log Workout, Hit Streak Milestone, View Paywall, Start Free Trial, Upgrade to Premium, Cancel Subscription, Open App, Invite Friend), 7 event properties, and 5 user properties, created directly in the live Amplitude project — this is the part of the job that happens before any engineer writes a tracking call. Then a documented synthetic dataset: 3,000 users and 52,082 events, with every behavioral difference modeled on a deliberate, real pattern rather than random noise — Referral- and Organic-acquired users complete onboarding and retain far better than Influencer-driven signups (lower-intent curiosity clicks, a real growth pattern), and hitting a 7-day workout streak sharply raises the odds of seeing — and converting on — the premium paywall shortly after (a real behavioral-commitment upsell pattern). That data was sent into the actual Amplitude project through Amplitude's own HTTP API, run from a Python script with my own API key — real ingestion, not fabricated chart data.

Key Findings

  • Daily signups climbed from single digits in May to 50-65/day by late July across all 4 channels, with Organic consistently the largest share — the deliberate growth ramp built into the dataset, confirmed in real Amplitude charts, not just the generator script.
  • Onboarding completion clearly splits by channel: Organic and Referral-driven signups convert to a completed profile far more consistently than Paid Social or Influencer signups — visible directly in the segmentation chart, matching the model's Referral 88%/Organic 81% vs. Influencer 55% completion rates.
  • Cardio and Strength are Strideway's most-logged workout types, HIIT and Yoga meaningfully behind — a simple distribution chart, still a real product-mix read.
  • iOS modestly outpaces Android (55/45 split, matching the model) — a simple platform check, useful before reading too much into any other channel's numbers.
Amplitude Segmentation chart: Daily Unique Users for Sign Up Event, May 1 to Aug 13, grouped by signup_channel — Organic (blue) consistently highest, all 4 channels trending upward Amplitude Segmentation chart: Daily Unique Users Logging Workouts by Workout Type — Cardio and Strength leading, HIIT and Yoga lower, all trending upward with overall growth Amplitude Segmentation chart: Daily Unique Users Completing Onboarding by Signup Channel — Organic (blue) clearly highest, Referral/Paid Social/Influencer lower and closer together Amplitude Segmentation chart, full browser view: Daily Unique Sign Up Users by Platform — iOS (blue) modestly higher than Android (green), both trending upward

All 4 screenshots are real charts from my own live Amplitude project, built through the actual Amplitude UI against the ingested dataset — the platform chart left with the full browser window visible rather than cropped to just the chart.

Beyond the Guide: Amplitude's Own Default Dashboard

Past the specific charts in the practice guide, I also opened Amplitude's auto-generated "User Activity" notebook — a template it builds automatically once real data starts flowing in, not something I set up. It confirmed the data pipeline end to end: 243 daily active users, new-signup and returning-user trends, and a visible retention decay curve, all matching the pattern the dataset was built around. It also surfaced something worth understanding rather than dismissing: the device/country/platform breakdown panels all show "Invalid chart definition." That's not the pipeline breaking — those panels reference Amplitude's own auto-captured device/geo properties, normally filled in by a real mobile/web SDK. Since this dataset was sent server-side through the HTTP API rather than a live app SDK, those specific built-in properties were never populated — expected, not a bug, and a real distinction between "data sent by a script" and "data sent by an instrumented app" worth knowing.

Amplitude's auto-generated User Activity dashboard: 243 daily active users, new-user and returning-user trend charts, a retention decay curve, and device/country/platform breakdown panels showing 'Invalid chart definition' since this data was sent server-side without SDK-captured device properties

Amplitude's own auto-generated dashboard, exported as PDF directly from the tool — included as-is, "Invalid chart definition" panels and all, rather than cropped down to only the clean parts.

Quick recall

SectionSummary
ProblemAmplitude is listed as a skill worth having, but with no prior hands-on use demonstrated anywhere on the site — the gap this project closes with a real account and real data, not a simulation.
Research stepsBuilt a real 10-event tracking plan (7 event properties, 5 user properties) directly in a live Amplitude project, the part of the job that happens before an engineer writes a tracking call → generated a documented synthetic dataset (3,000 users, 52,082 events) with every behavioral difference modeled on a real growth pattern → sent it through Amplitude's own HTTP API with a personal key, real ingestion rather than fabricated chart data → opened Amplitude's own auto-generated dashboard as a second, independent check and investigated rather than dismissed the "Invalid chart definition" panels it surfaced.
Solution5 real segmentation charts (signup by channel, workout type distribution, onboarding completion by channel, platform split) plus Amplitude's own auto-generated User Activity dashboard, all built against live ingested data.
ConclusionThe "Invalid chart definition" panels weren't a bug — they're the expected result of sending data server-side via API instead of through a live app SDK, which never populates Amplitude's auto-captured device/geo properties. Understanding that distinction, not just getting clean charts, is the real finding.
T33

Amplitude Analytics — Strideway (Advanced)

View the files on GitHub →

Full 6-step monetization funnel Retention curve segmented by channel Real conversion-window bug found and fixed Hit a real Starter-plan limit on cohorts

Tools used: Amplitude (Funnel analysis, Retention analysis, Cohorts), the Amplitude Chart API (used to diagnose the bug below).

Why Advanced, Not Just Basic

Segmentation charts prove I can read a single metric; this half proves I can trace a user journey end to end — funnel conversion, retention over time, and cohort definition, the exact Amplitude patterns behind most growth/monetization dashboards. Same Strideway dataset as the Basic case study (T32).

A Real Amplitude Bug, Found and Fixed

The first funnel I built — Sign Up → Complete Onboarding → Log Workout → View Paywall → Start Free Trial → Upgrade to Premium — came back showing only 19% conversion from onboarding to a first logged workout. That looked wrong: nearly every onboarded user in this dataset eventually logs a workout. I checked directly against the raw dataset and confirmed it — 92.5% of onboarded users eventually log at least one workout, nowhere near 19%.

The cause, found by pulling the chart's real definition through Amplitude's own API rather than guessing: Amplitude funnels default to a 1-day conversion window between each step (conversionSeconds: 86400). Since many users in this dataset don't log their first workout until well after onboarding — they open the app sporadically over weeks, not daily — a 1-day window was silently dropping every conversion that happened even slightly later than that, while still reporting a clean-looking, plausible percentage. Widening the window to 90 days fixed it exactly: Log Workout jumped to 68.97% (2,069 of 2,236), matching the raw dataset precisely.

Key Findings

  • Full funnel, corrected: Sign Up 3,000 → Complete Onboarding 74.53% → Log Workout 68.97% → View Paywall 11.97% → Start Free Trial 5.9% → Upgrade to Premium 1.63% (49 users) — a realistic, multi-week monetization path, not a same-day one.
  • Retention by channel is the clearest chart in the whole project: at Day 30, Referral-acquired users retain around 46%, Organic around 36%, Paid Social around 22%, and Influencer around 9% — a roughly 5x spread between the best and worst channel, purely from acquisition source.
  • Cohort creation hit a real plan limit: dynamic (auto-recalculating) cohorts require a paid Amplitude plan; the Starter tier blocked saving one. The definition itself — "users who performed Upgrade to Premium, count ≥ 1" — was correct; only persisting it as a live cohort was gated. A genuine, worth-knowing constraint of the free tier, not a mistake.

What This Means, Applied

The retention chart changes the decision quite a bit: I wouldn't automatically scale Influencer just because it brings users in. I'd treat Referral as the stronger quality channel if its retained-user curve is consistently better. As PM, I'd shift more acquisition effort toward Referral, while keeping Influencer as a controlled acquisition experiment. For Influencer, I'd dig into which creators, audiences, and landing experiences are producing retained users rather than judging the channel only on acquisition volume. I'd also set up a cohort comparison for Influencer vs. Referral — same signup period, same retention window, ideally normalized by plan — to make sure the difference isn't caused by audience mix. The goal wouldn't be "kill Influencer"; it would be optimize for retained customers, not just acquired customers. If Referral continues to show stronger retention, I'd increase investment there and use Influencer selectively where creators can demonstrate comparable downstream retention.

Before / After: the funnel-window fix

Amplitude funnel chart before the fix: Sign Up 100%, Complete Onboarding 74.53%, Log Workout only 19% (570 users), View Paywall 0.3%, Start Free Trial 0.13%, Upgrade to Premium 0% — the 1-day conversion window silently dropping real conversions Amplitude funnel chart after widening the conversion window to 90 days: Sign Up 100%, Complete Onboarding 74.53%, Log Workout 68.97% (2,069 users), View Paywall 11.97%, Start Free Trial 5.9%, Upgrade to Premium 1.63% — matching the raw dataset exactly Amplitude retention chart: Daily retention from Sign Up to Open App by signup channel over 30 days — Referral (purple) highest around 46% by Day 30, Organic (blue) around 36%, Paid Social (green) around 22%, Influencer (teal) lowest around 9% Amplitude retention chart builder, full browser view: Starting event Sign Up, return event Open App, Measured as 'Return On or After' — the actual configuration panel behind the retention-by-channel chart, not just its output Amplitude New Cohort screen: users who performed Upgrade to Premium, count >= 1 time, during Last 90 days — a correct definition that could not be saved on the free Starter plan (dynamic cohorts are a paid feature)

All 5 screenshots are real captures from my own live Amplitude project — including the broken funnel and the retention chart's builder panel, left in on purpose to show the actual process rather than only clean, cropped results.

Quick recall

SectionSummary
ProblemThe Basic case study proves I can read a single metric; it doesn't prove I can trace a full user journey — funnel conversion, retention over time, cohort definition — the Amplitude patterns behind most real growth/monetization dashboards.
Research stepsBuilt the 6-step funnel → the first result (19% onboarding→workout conversion) looked implausible against the raw dataset (92.5% actually convert) → pulled the chart's real definition through Amplitude's own Chart API instead of guessing → found the default 1-day conversion window was silently dropping late conversions → widened it to 90 days and confirmed the fix matched the raw data exactly (68.97%) → built the retention-by-channel chart and attempted to save a cohort, which hit a real Starter-plan limit.
SolutionThe corrected funnel, a retention-by-channel chart, and a verified cohort definition — plus the "What This Means, Applied" recommendation above: shift acquisition investment toward Referral, use Influencer as a controlled experiment, and normalize any channel comparison by plan before trusting it.
ConclusionThe real finding isn't the funnel percentages themselves — it's that a tool's default setting (a 1-day conversion window) can silently produce a clean, plausible, and completely wrong number, and the only way to catch that is checking a surprising result against the raw data instead of trusting the chart.
T37

Partner Ops — SQL & API Testing Practice

Metrics & Analytics

View the files on GitHub →

18 partners, 90 payouts, 160 activity logs — 5 issues seeded on purpose, found by query 13 SQL queries, basic → advanced, verified live on SQL Server 10-request Postman collection + mock server, incl. a chained POST→PATCH workflow Found and fixed 2 real bugs along the way, not just a clean first pass

Tools used: SQL Server 2022 Express (T-SQL), VS Code with the SQL Server (mssql) extension, Postman (collection + a self-hosted mock server).

Why This Project

A self-directed practice project, not client work — built to turn "I know SQL and APIs conceptually" into something demonstrable, in the same operational world I already know from owning PCMS at Bajaj Finserv: partners, e-agreements, payouts, and activity logs. Five real data issues (duplicate payouts, payments made after an agreement had expired, agreements stuck unsigned for 90+ days, activity logged after an account was suspended) were seeded into the raw data on purpose, not pre-labeled — the exercise was writing the SQL and API checks that actually find them.

Two Real Bugs, Found and Fixed

Nothing here was a clean first pass, which is arguably the more useful part to show. A regional-average SQL query came back with the right leading region but the wrong number — tracing it back showed the payouts data was correct and the problem was 10 of 18 partners carrying the wrong region tag in a dimension table, found with a JOIN against a known-correct reference set and fixed with one set-based UPDATE ... FROM ... JOIN. Separately, a Postman mock server was silently returning the wrong saved example for two requests that shared a path (/payouts) but differed only by query string — two conditions in the generated server code both matched on path alone, so the first one always fired and the second was dead code. The fix was adding a check on the actual query parameter the two examples were supposed to be distinguished by.

Key Findings

  • 3 duplicate payout pairs surfaced with a single GROUP BY ... HAVING COUNT(*) > 1 — the classic signature of a non-idempotent retry or a batch job re-run, not a data-entry typo.
  • 4 payouts moved after their governing agreement had already expired, across 2 partners — the sharpest finding in the dataset, surfaced by a 3-table JOIN comparing payout date against agreement expiry.
  • 2 payout requests logged after the partner's account was suspended, found by joining the activity log to account status — the same "cross-reference what someone did against what their account should allow" pattern, just on behavioral data instead of transactional data.
  • The Postman side mirrors the SQL side exactly — every example response in the API collection was pulled directly from the same database, so testing "fetch a partner's activity logs" via API returns the identical numbers already verified in SQL.

What This Means, Applied

The methodology matters more than any single query: start with a structural check (does every payout have a valid, currently-active agreement behind it), then a duplication check, then an activity-consistency check, then consolidate everything into one queryable report instead of three separate spreadsheets. That's a real discipline, not a one-off script — the same instinct behind the research above, applied to payments and partner operations instead.

Walkthrough — Input → Output → Understanding

Six of the real query/request pairs from this project, each with what was sent, what actually came back, and what it means — the same way I'd walk an interviewer through it verbally rather than just showing a result.

1. Duplicate payouts — Input

SELECT partner_id, amount, payout_date, COUNT(*) AS cnt FROM payouts GROUP BY partner_id, amount, payout_date HAVING COUNT(*) > 1;

Output

partner_idamountpayout_datecnt
150935.882026-03-262
162587.812025-03-262
113526.912026-06-232

Understanding — 3 pairs, same partner/amount/date, each appearing exactly twice. That's the signature of a retry that wasn't idempotent or a batch job that re-ran — not a typo. A running total or reconciliation report built on this data without deduping first would silently overstate what that partner was actually owed.

2. Payouts after agreement expiry — Input

SELECT p.payout_id, pt.partner_name, p.payout_date, e.expiry_date FROM payouts p JOIN partners pt ON pt.partner_id = p.partner_id JOIN e_agreements e ON e.partner_id = p.partner_id WHERE e.status = 'expired' AND p.payout_date > e.expiry_date AND p.status = 'completed';

Output

payout_idpartner_namepayout_dateexpiry_date
177Bluepeak Finserv2026-04-192026-02-15
178Bluepeak Finserv2026-04-142026-02-15
179Ganges Retail Finance2026-03-192026-02-15
180Ganges Retail Finance2026-04-042026-02-15

Understanding — 4 payouts, 2 partners, all moved 1–2 months after the governing agreement had already lapsed. This is the sharpest finding in the dataset: money moved on a contract that wasn't valid anymore. The 3-table JOIN is what makes it visible — payouts alone has no idea an agreement even exists.

3. Activity after suspension — Input

SELECT pt.partner_name, a.activity_ts FROM activity_logs a JOIN partners pt ON pt.partner_id = a.partner_id WHERE a.activity_type = 'payout_request' AND pt.status = 'suspended';

Output

partner_nameactivity_ts
Lakeside Capital2026-08-15 10:22:00
Everest Capital Partners2026-08-15 10:22:00

Understanding — 2 partners requested a payout after their account had already been suspended. This is the same instinct as finding #2, applied to behavior instead of money: cross-reference what someone did against what their account status should have allowed.

4. Fetch a partner's aggregate summary via API — Input

GET {{baseUrl}}/partners/1/summary

Output — 200 OK

{ "partner_id": 1, "partner_name": "Aravali Traders", "status": "active", "total_payouts": 6, "total_amount_paid": 254101.2, "agreement_status": "signed", "flagged_duplicate_payouts": 6 }

Understanding — one API call returning the same numbers that took several SQL queries to assemble (6 payouts, ₹254,101.20 total, 6 duplicate-flagged rows — matching finding #1 exactly, since all 3 duplicate pairs happen to belong to partner 1). This is the API equivalent of a JOIN + aggregate: the server does the assembling so the client doesn't have to make 4 separate calls.

5. Create a discrepancy record — Input

POST {{baseUrl}}/discrepancies Body: { "partnerId": 1, "issueType": "manual_review", "note": "Flagged during interview practice run" }

Output — 201 Created

{ "discrepancy_id": 999, "partnerId": 1, "issueType": "manual_review", "note": "Flagged during interview practice run", "detected_date": "2026-09-09", "resolved": false }

Understanding — 201, not 200: this request created something new, and the ID it returned (999) gets saved automatically into a collection variable by a test script, ready for the next request to reuse — no copy-pasting an ID by hand.

6. Resolve it — the chained request — Input

PATCH {{baseUrl}}/discrepancies/{{createdDiscrepancyId}} Body: { "resolved": true }

Output — 200 OK

{ "discrepancy_id": 999, "resolved": true }

Understanding — {{createdDiscrepancyId}} in the URL was never typed by hand — it's the exact ID request #5 saved a moment earlier. That's a real multi-step workflow (create → confirm → resolve), the same shape as create-an-order → pay-for-it → ship-it in any real transactional API, not three disconnected calls that happen to run one after another.

Quick recall

SectionSummary
ProblemTurn "I know SQL and APIs conceptually" into something demonstrable, in the same operational world already known from owning PCMS at Bajaj Finserv — partners, e-agreements, payouts, activity logs.
Research stepsSeeded 5 real data issues into raw data on purpose, unlabeled (duplicate payouts, post-expiry payments, stuck-unsigned agreements, activity after suspension) → wrote 13 SQL queries from basic to advanced to find them, not pre-knowing which query would surface which issue → built a 10-request Postman collection mirroring the same database, including a chained POST→PATCH workflow → found and fixed 2 real bugs along the way: a wrong region tag in a dimension table (traced via JOIN against a known-correct reference), and a Postman mock server silently returning the wrong example for two requests sharing a path.
Solution13 verified SQL queries plus a 10-request Postman collection against a self-hosted mock server, with every API example response traceable back to the same database the SQL queries already verified.
ConclusionThe methodology matters more than any single query: structural check → duplication check → activity-consistency check → consolidate into one queryable report instead of three separate spreadsheets. The sharpest single finding: 4 payouts moved after their governing agreement had already expired — invisible without a 3-table JOIN.
T39

Fintech API Landscape — Testing Reference Guide

Metrics & Analytics

View the files on GitHub →

9 categories, 24 real-world API examples a BFSI company like Bajaj Finserv actually integrates Every example: realistic endpoint, sample request/response, and the test scenarios that matter Written after Partner Ops and NSDL KYC — a map of the wider terrain those two projects sit inside A reference document, not a built system — honestly labeled as such

Why This Exists

Partner Ops and NSDL KYC are two categories tested hands-on. This is the map of the other seven a compliance-flavored PM role would also need to speak to — not built or tested, a reference compiled to know what exists, what each one is for, and what a good test suite checks, before ever touching one. The honest boundary matters: knowing the landscape isn't the same as having built in it, and this document says so plainly rather than blurring the two.

The 9 Categories

CategoryCountReal examples
Identity & KYC4NSDL PAN verification, Aadhaar eKYC, DigiLocker, CKYC lookup
E-agreement / e-sign2Aadhaar eSign, e-stamp & e-sign
Credit & risk3Credit bureau pull, risk-scoring engine, bank statement analysis
Payments & payouts4UPI, NEFT/RTGS/IMPS, payment gateway, e-NACH
Core banking / ledger1Finacle integration — real Bajaj Finserv experience
Fraud & AML screening4PEP list, sanctions list, device fingerprinting, velocity checks
Regulatory reporting3RBI/SEBI feeds, CKYC upload, STR filing
Notifications / webhooks2SMS/email/WhatsApp gateway, internal webhooks
Account aggregator1RBI's AA framework (Setu, Finvu)

Two Examples, in Full

UPI Payment — POST /v1/payments/upi

Insufficient balance returns 200, not 400 — {"status": "FAILED", "reason": "INSUFFICIENT_FUNDS"} — because the request was valid, the transaction just didn't succeed. That distinction — request-validity vs. outcome — is one of the most commonly missed in payment API testing.

PEP (Politically Exposed Person) Screening — POST /v1/screening/pep-check

Three-way result, not two: clean (auto-clear), confirmed match (auto-block), or a fuzzy partial match — {"match": "POSSIBLE", "requiresManualReview": true} — that routes to a human instead of an automatic decision either way. Real fraud screening has that middle state; testing it means checking for three outcomes, not two.

The Pattern Across All 24

Every one of the 24 reduces to the same 7 questions:

  1. Does it work with valid input?
  2. Does it correctly reject bad or missing input?
  3. Does it enforce who's allowed to ask?
  4. Does it separate "the request was fine but the outcome wasn't" from "the request itself was broken"?
  5. Does it handle duplicates/retries without double-processing?
  6. What happens when a downstream/third-party system is slow, down, or lying?
  7. Is there a compliance-specific rule layered on top (consent, masking, a regulatory deadline) that a generic test wouldn't catch without knowing the domain?

That seventh question is the actual differentiator for a compliance-flavored PM role — anyone can check "does it return 200." Knowing which extra domain rule applies to which API is what this document was really testing fluency on.

Quick recall

SectionSummary
ProblemPartner Ops and NSDL KYC tested 2 of 9 real fintech API categories hands-on; a compliance-flavored PM role needs fluency across the other 7 too — identity, payments, credit/risk, fraud/AML, core banking, regulatory reporting, account aggregator.
Research stepsMapped all 9 categories against 24 real-world API examples a BFSI company like Bajaj Finserv actually integrates → worked 2 examples in full detail (UPI payment, PEP screening) to show the request-validity-vs-outcome distinction and the 3-way fraud-match result → reduced all 24 examples to the same 7 underlying test questions, isolating the one (domain-specific compliance rules) that actually differentiates a compliance PM from a generic API tester.
SolutionA 24-example reference guide (Markdown + .docx), organized by category, each with a realistic endpoint, sample request/response, and the specific test scenarios that matter — honestly labeled as a reference document, not a built or tested system.
ConclusionAnyone can check "does it return 200" — the real skill is knowing which extra domain rule (a consent requirement, a masking rule, a regulatory deadline) applies to which specific API, which is what this document was built to test fluency on.

Feature Reverse Engineering

// dissecting a single shipped feature
T18

Cross-industry examples

Framework used: Feature teardown (user flow, edge cases, failure/abuse scenarios, trade-offs) — analysis of a shipped third-party feature, not something I built.

The Feature

Duolingo's Streak: the daily-use counter that resets on a missed day. The user problem: learners forget, feel guilty, then quit. The business problem: D30 retention collapses without an intervention loop. Streak exists because loss aversion outperforms achievement motivation as a retention lever.

User Flow & Workflow

Complete at least one lesson before the user's local day rolls over → the streak counter increments and the flame icon updates → as the day-end window approaches with no lesson yet done, a reminder notification fires. Under the hood: streak state is per-user and evaluated against local midnight, not server time, the single detail that makes "what counts as today" a genuinely nontrivial edge case (see below). "Streak Freeze" can be pre-equipped as an item to auto-preserve the streak through one missed day; "Streak Repair" lets a user restore a broken streak after the fact, monetized alongside the free-currency ("gems") economy and the paid subscription tier.

Assumptions & Edge Cases

The design bets that loss aversion (not wanting to lose an accumulated number) beats pure achievement-seeking as a daily-return driver, and that a single short lesson is low enough friction to sustain the habit. Real edge cases this creates: timezone travel shifts the local-midnight boundary mid-streak; offline lesson completions have to sync before that boundary or the streak silently breaks; very long-running streaks (some users publicly report 1,000+ days) turn loss aversion into extreme lock-in, which is a retention asset but also a support-ticket and goodwill liability the moment a bug breaks one.

Failure & Abuse Scenarios

The most-discussed failure mode: users protect the streak with the fastest, easiest lesson available rather than one that reflects real learning progress: the metric gets gamed against the product's actual goal. The anxiety side can also overcorrect: a user who fears an inevitable eventual miss disengages entirely rather than risk it, which is the opposite of the intended effect. Streak Freeze mitigates the anxiety but introduces its own abuse case: it can become a crutch that quietly reduces real daily engagement while the visible number keeps climbing.

Trade-offs & Redesign

The "streak freeze" mechanic is the interesting abuse case here: it normalizes breaks while still keeping users inside the ecosystem. But the feature creates user anxiety alongside motivation; it's not a free win. A redesign worth considering: surface a visible "grace period" upfront to reduce that anxiety while preserving the retention hook.

Launch, Adoption & Business Impact

Streak is one of Duolingo's oldest gamification mechanics, broadly rolled out well before the company's more recent AI-driven feature set, then iterated on over years. Streak Freeze and Streak Repair came later as monetization and retention levers, and "Friend Streaks" (a shared streak between two mutual users) is a natural, later extension of the same loss-aversion mechanic applied socially. Duolingo has publicly described streaks as one of its strongest engagement and retention correlators in shareholder communications and its own product writing (fill in a specific cited figure if one is worth quoting directly, and a source link), directionally credible given how central the mechanic remains to the product, even without an internally-verified number to cite here.

Quick recall

SectionSummary
ProblemLearners forget, feel guilty, then quit — D30 retention collapses without an intervention loop, and Streak exists because loss aversion outperforms achievement motivation as a retention lever.
Research stepsTraced the user flow down to the one nontrivial implementation detail (streak evaluated against local midnight, not server time) → surfaced the real edge cases it creates (timezone travel, offline sync races) → named the central abuse case (users protect the streak with the easiest lesson, gaming the metric against the product's actual learning goal) → identified that Streak Freeze solves the anxiety problem but introduces its own crutch-abuse case.
SolutionA redesign idea: surface a visible "grace period" upfront to reduce streak anxiety while preserving the retention hook, rather than only offering Streak Freeze as a paid/equipped recovery mechanic after the fact.
ConclusionThe feature creates user anxiety alongside motivation — it's not a free win. The interesting design tension is that Streak Freeze normalizes breaks while still keeping users inside the ecosystem, which is a retention win dressed as a convenience feature.

The Feature

ChatGPT's Memory: persistent, cross-conversation memory. The user problem: people repeat context in every new conversation; the assistant "forgets" preferences, ongoing projects, and prior corrections. The business problem: a session-to-session experience that feels disposable without persistence is a real retention and differentiation lever against competitors without it. Memory exists because personalization compounds engagement: the more the assistant "knows" a user, the higher the switching cost to a competitor.

Design Decisions

What gets remembered vs. forgotten: explicit user statements vs. inferred preferences. The guardrails: user-visible memory management (view/edit/delete), opt-out control, no silent retention of sensitive data.

The Hard Problem

There's no ground-truth "correct" memory; success has to be measured by correction rate and re-engagement, not accuracy. That creates a real trade-off between personalization and privacy: more useful over time, but a "creepiness" risk if memory surfaces unprompted. The sharpest failure mode is stale memory: the model over-indexing on outdated info (e.g. remembering a job the user has since left), which needs a decay/update strategy layered on top of plain accumulation.

Redesign Idea

Confidence-weighted memory with visible "last updated" timestamps, so users can trust and correct what's remembered in-line.

The Broader Pattern Here

The general pattern for any AI feature with implicit state: the PM's real job is designing the correction and visibility mechanism, not the memory mechanism itself.

Quick recall

SectionSummary
ProblemPeople repeat context in every new conversation; a session-to-session experience that feels disposable without persistence is a real retention and differentiation gap against competitors that have it.
Research stepsSeparated what gets remembered (explicit statements vs. inferred preferences) from the guardrails needed around it (user-visible management, opt-out, no silent retention of sensitive data) → identified that there's no ground-truth "correct" memory, so success has to be measured by correction rate and re-engagement, not accuracy → named stale memory (over-indexing on outdated info) as the sharpest failure mode, requiring a decay/update strategy on top of plain accumulation.
SolutionConfidence-weighted memory with visible "last updated" timestamps, so users can trust and correct what's remembered in-line instead of memory being an opaque black box.
ConclusionThe general pattern for any AI feature with implicit state: the PM's real job is designing the correction and visibility mechanism, not the memory mechanism itself — the same lesson that recurs across the AI copilot case studies elsewhere on this site.

Product Strategy

// business model, moat, and competitive positioning
T7

Travel Loyalty Program Redesign

Framework used: Lifecycle/retention design, KPI tree — scope doc, not yet built.

Scope

Redesign a loyalty program for an OTA or airline: the same lifecycle-design problem every points program eventually hits: it drives initial sign-up, then stops driving behavior once a user stalls mid-tier.

Current Pain Points

Redemption friction: points that are easy to earn but hard to actually spend (blackout dates, limited reward inventory) train users to stop valuing them, which defeats the program's purpose. Tier stagnation: a user who reaches a mid-tier and realizes the next tier is unreachable within a normal booking cadence disengages entirely rather than working toward it. Opaque earning rules: when users can't predict what a booking will actually earn them, the program stops functioning as a behavioral lever at the moment of booking, which is the only moment that matters.

Redesign: Earning, Redemption & Gamification

Earning: shift part of the accrual from pure spend-based points to behavior-based points (completing a review, booking during a shoulder season the business wants to fill, or booking further in advance) so the program can be tuned toward business-value behavior instead of raw transaction volume. Redemption: guarantee a minimum real-value redemption path (even if less attractive than a rare premium reward) so points always feel spendable, closing the "earn but can't use" trust gap. Gamification: a visible streak for consecutive-quarter bookings (the same loss-aversion mechanic as Duolingo's Streak, see Feature Reverse Engineering above, applied to a much lower-frequency behavior); milestone tiers with a clearly visible "X bookings to next tier" progress indicator instead of an opaque point threshold; occasional surprise-and-delight unlocks (an unexpected small perk) to reward loyalty outside the predictable earn/redeem loop, which research on variable reward schedules suggests sustains engagement better than a purely predictable one.

KPIs

Repeat booking rate within the program (the core retention signal), tier upgrade rate (whether the redesigned progress visibility actually moves people, separate from whether points simply accrue), redemption rate (whether the "always spendable" fix actually closes the trust gap), and, as a guardrail, cost-per-loyalty-booking, since a redesign that drives repeat bookings purely by giving away margin is a loss dressed up as a win.

What This Draws On

Retention/lifecycle design: the loyalty counterpart to the funnel-conversion work in the Booking Funnel Analysis above.

Quick recall

SectionSummary
ProblemEvery points program eventually hits the same lifecycle problem: it drives initial sign-up, then stops driving behavior once a user stalls mid-tier — driven by redemption friction, tier stagnation, and opaque earning rules.
Research stepsNamed 3 specific pain points rather than a vague "engagement is low" diagnosis → redesigned earning to shift part of accrual toward behavior-based points the business actually wants (reviews, shoulder-season bookings) → guaranteed a minimum real-value redemption path to close the "earn but can't use" trust gap → borrowed the loss-aversion mechanic from the Duolingo Streak teardown, applied to a much lower-frequency travel-booking cadence.
SolutionRedesigned earning (behavior-based accrual), redemption (guaranteed minimum spendable value), and gamification (visible progress-to-next-tier, a consecutive-quarter streak, occasional surprise-and-delight unlocks).
Conclusion4 KPIs, each tied to a specific redesign bet — repeat booking rate (retention), tier upgrade rate (does visibility actually move people), redemption rate (did the trust gap close), and cost-per-loyalty-booking as the guardrail, since a redesign that buys repeat bookings purely by giving away margin is a loss dressed up as a win.
T17

Airline/Hotel Distribution System (GDS)

Framework used: Industry/business-model study — not yet built.

The System

Travel Distribution Systems: Amadeus IT Group, Sabre Corporation. The B2B plumbing that makes every other system on this list possible: the GDS is what an OTA's search actually queries behind the scenes. What PMs build here is B2B infrastructure for travel agencies and OTAs, not a consumer product; API reliability, partner SLAs, and integration timelines are the actual PM discipline, not UX polish.

Key Concepts

  • GDS as B2B infrastructure: the customer is a travel agency or OTA, not a traveller; a completely different PM discipline than every consumer-facing system above.
  • NDC (New Distribution Capability): the airline industry's real, current push to bypass traditional GDS with direct API access; an active disruption to this exact business model, worth naming by name.
  • Inventory synchronization latency: a seat sold on one channel must be un-sold everywhere else within seconds, or the airline oversells.
  • Switching costs & lock-in: why airlines and agencies rarely change GDS providers; the actual moat this multi-decade business runs on.

Open Questions

Why is NDC a genuine threat to Amadeus/Sabre's business model, and how would a PM there respond? What does a GDS integration project actually look like from an OTA's engineering side (ties to T1's "technical execution" comfort)?

Where This Skill Set Applies

B2B/infrastructure product management, API-as-product thinking, understanding a legacy moat well enough to know what threatens it.

T21

Cross-industry example

Framework used: SWOT, competitive-moat/flywheel analysis — analysis of a real company, not something I built.

The Business

Zepto is an Indian quick-commerce company (founded 2021, Mumbai) promising grocery and daily-essentials delivery in roughly 10 minutes, built on a network of hyperlocal "dark stores": small, delivery-only micro-fulfillment centers rather than customer-facing retail. Industry: quick commerce / India, one of the most heavily VC-funded and fiercely contested categories in Indian consumer tech. fill in: precise TAM/SAM/SOM figures, with a cited source; a defensible estimate needs real market-sizing data, not a guess. Business model: revenue from delivery fees plus retail margin on grocery/FMCG items, at high order frequency and low basket size per order, the opposite unit-economics shape of a traditional e-commerce order. Growth strategy: category expansion beyond core grocery into adjacent high-margin categories (electronics, beauty, pharmacy) once the delivery network exists.

Competitors: Blinkit (Zomato-owned), Swiggy Instamart, BigBasket's quick-commerce arm (Tata-owned), and newer entrants like Flipkart Minutes and Amazon Now, all racing on the same dark-store-density playbook, which makes this one of the clearest multi-player moat questions in Indian consumer tech right now.

The Moat

The competitive advantage is precision over scale: dark store density concentrated in top-10 pin codes rather than broad coverage. The network effect: orders → better placement/inventory decisions → faster delivery → more orders (flywheel). The moat itself is dark store density in high-density pin codes. The risk is thin margins.

SWOT

  • Strengths: dense, defensible urban delivery infrastructure; fast, habit-forming delivery times that raise the switching cost once a user's routine forms around it.
  • Weaknesses: structurally thin per-order margins at low basket sizes; heavy dependence on continued VC funding to sustain the dark-store buildout ahead of profitability.
  • Opportunities: category expansion into higher-margin verticals; private-label/in-house brands, which is where several competitors in this category have already started moving to fix the margin problem.
  • Threats: at least three other well-funded players running the identical playbook in the same cities; regulatory attention on deep discounting and gig-worker labor practices in the quick-commerce sector; a plausible consolidation wave that could compress the number of viable players.

Strategic Recommendations

Don't compete on core delivery-speed parity once it's table stakes across all major players; compete on basket economics instead. Push private-label penetration in high-frequency categories (staples, household) to improve gross margin per order without relying on delivery-fee increases that would hurt the retention the density moat already bought. Treat premium/high-margin SKU categories (electronics, beauty) as the actual profitability lever, not the core grocery business.

Outlook

5-year prediction: the profitability path runs through premium SKUs, not core delivery margin.

Quick recall

SectionSummary
ProblemQuick commerce is one of the most heavily VC-funded, fiercely contested categories in Indian consumer tech, with at least 4 well-funded players running the identical dark-store-density playbook — what, if anything, actually differentiates Zepto's position.
Research stepsNamed the real competitive advantage as precision over scale — dark-store density concentrated in top-10 pin codes rather than broad coverage → traced the flywheel (orders → better placement/inventory decisions → faster delivery → more orders) → ran a full SWOT naming the structural weakness (thin per-order margins at low basket size) alongside the real strength (habit-forming delivery times raising switching cost).
SolutionDon't compete on delivery-speed parity once it's table stakes — compete on basket economics instead: push private-label penetration in high-frequency categories, and treat premium/high-margin SKUs (electronics, beauty) as the actual profitability lever, not core grocery.
Conclusion5-year outlook: the profitability path runs through premium SKUs, not core delivery margin — the dark-store density moat buys retention, but margin has to be earned somewhere the delivery-speed race doesn't touch.

PM Writing

// PRDs, decision memos, and structured product docs
T10

Travel PRD Portfolio

Framework used: 7-field PRD structure — scoped, 0 of 6 PRDs written yet.

Scope

Short PRDs, one per feature, each reusing the Problem → User Stories → Business Goals → Requirements → Acceptance Criteria → Success Metrics → Launch Plan structure from the PM Writing case study (T22) below. Depth on any one of these isn't the point here; the AI Trip Planner PRD (T4) already proves that. The point is range: can the same structure hold up across six genuinely different feature shapes without turning into a template exercise.

Why These Six

Each was picked for a different reason a PM actually has to reason through, not because the list needed padding. Price alerts: pure retention mechanic, no new inventory required, the easiest one to justify to an engineering team on effort alone. Smart refunds: the trust problem: refund policy already gets fought over in the payments work above (T16); this is the user-facing half of that same fight. Waitlist for sold-out flights: a demand-signal play as much as a feature: it tells the business what routes to add capacity on, beyond simply telling engineering what to build. Multi-city planner: explicitly the complexity the AI Trip Planner (T4) scoped out of its v1; this is where that deferred requirement would actually get written up. Carbon footprint calculator: the one feature here that's regulation-adjacent rather than revenue-adjacent, worth including precisely because it forces a different kind of requirement (disclosure accuracy) than the other five. AI travel assistant: the hardest one to scope honestly, since "AI assistant" is where PRDs most often turn into vague feature wishlists instead of real requirements.

Progress: 0 of 6 actually written; this page tracks real completion, not intent; a PRD only gets checked off once it exists

What This Portfolio Proves

PRD writing at range and speed: breadth across a feature roadmap rather than depth on one project.

T22

Cross-industry example

Framework used: 7-field PRD structure, at the template/meta level.

The Template

A PRD structured as seven fields (problem statement, user stories, business goals, requirements, acceptance criteria, success metrics, launch plan), the same structure reused across the Travel PRD Portfolio (T10) and the AI Trip Planner PRD (T4). Seven fields, not twelve: every extra section is a section a stakeholder skims past, and a skimmed section is worse than a missing one because it creates the illusion the question got answered.

Positioning

"A PRD reads like a spec doc. It works like a decision memo in disguise." The tell is which section gets read first: nobody opens a PRD to find out how a button works; they open it to find out whether the thing is worth building at all, which is a Goals and Requirements question, not an Acceptance Criteria one. Write the doc in that order and most of the actual disagreement in a review happens on page one, where it's cheap to argue about, instead of surfacing three weeks into build.

Where This Template Breaks

It's built for a feature with a defined shape: something that has a clear "done." It strains on genuinely open-ended, discovery-stage work, where forcing Acceptance Criteria this early just produces confident-sounding fiction. Swapping in a different template doesn't fix that; the honest fix is admitting the doc is a hypothesis memo at that stage, not a requirements doc, and labeling it that way instead of dressing it up as more certain than it is.

fill in: executive updates, stakeholder memos, product vision docs, roadmap summaries, release notes, and decision proposals as additional writing samples, once written

Quick recall

SectionSummary
ProblemA PRD reads like a spec doc but has to work like a decision memo in disguise — most templates bury the "is this worth building" question under implementation detail, so disagreement surfaces three weeks into build instead of on page one where it's cheap to argue about.
Research stepsCut the structure to 7 fields deliberately, not 12, on the logic that every extra section is one a stakeholder skims past, and a skimmed section is worse than a missing one → ordered the fields so Goals and Requirements come before Acceptance Criteria, matching the order a reader actually needs answers in → stress-tested the template against open-ended, discovery-stage work and found where it genuinely breaks rather than claiming universal fit.
SolutionThe 7-field structure (problem statement, user stories, business goals, requirements, acceptance criteria, success metrics, launch plan), reused as-is across the Travel PRD Portfolio and the AI Trip Planner PRD.
ConclusionThe template strains on genuinely open-ended work, where forcing Acceptance Criteria too early just produces confident-sounding fiction. The honest fix isn't swapping templates — it's labeling the doc a hypothesis memo at that stage instead of dressing it up as more certain than it is.

Technical Execution

// API integration, SQL, and backlog discipline
T9

Visa Application Tracker

Tools used: none yet — scope doc only, not built.

Scope

Visas are the one travel step nobody enjoys and everybody procrastinates on. This tracks application status across embassies and visa-processing services, nudges the user before a document actually goes bad rather than after, estimates realistic processing time by visa type and country, and stores the underlying documents somewhere more durable than a phone's downloads folder.

Core Flows

Status tracking pulls from whatever an embassy, VFS, or BLS portal actually exposes, scraping where there's no real API, because most consulates don't have one, which is an honest constraint worth naming up front rather than assuming away. Reminders run on two independent clocks at once: one counting down to the application deadline, another counting down to passport or document expiry, the second one being the clock people actually forget about, since it's not the one they were watching. The document vault is the unglamorous but load-bearing piece: encrypted storage for passport scans, photos, bank statements, and invitation letters, each with its own expiry flag so a document going stale in storage doesn't turn into a problem at the airport counter.

Compliance Considerations

Secure document storage directly reuses the eKYC/RBI digital-lending compliance discipline from the fintech toolkit: data sensitivity is the hard part here, ahead of the UX itself.

What This Required

Regulated, documentation-heavy product design, and notification design that has to reason about two independent deadlines instead of one. Outcome: not built yet; stays labeled that way until it is

Quick recall

SectionSummary
ProblemVisas are the one travel step nobody enjoys and everybody procrastinates on — status is scattered across embassy/VFS/BLS portals, documents go stale in a phone's downloads folder, and the deadline people actually forget about (document expiry) isn't the one they're watching (the application deadline).
Research stepsNamed the honest constraint up front — most consulates have no real API, so status tracking has to scrape whatever a portal actually exposes, not assume a clean integration away → designed reminders around two independent clocks instead of one, specifically because the less-watched clock (document expiry) is the one that actually causes problems → traced the compliance angle back to the same eKYC/RBI digital-lending discipline already used in the fintech toolkit, since document sensitivity is the hard part here, ahead of the UX.
SolutionA tracker spanning status pulls, dual-clock reminders (deadline + document expiry), and an encrypted document vault where every stored document carries its own expiry flag.
ConclusionNot built yet, and labeled that way deliberately rather than implied otherwise — the real design insight (two independent deadlines, not one) is the part worth carrying into a build, whenever it happens.
T16

Travel Payments & Refunds System

Tools used: none yet — industry study guide, not built.

The System

Travel Payments Systems: Stripe, Adyen. The payment infrastructure underneath every other system here: a single itinerary can touch 4+ currencies and vendors at once. What PMs build here is multi-currency, multi-method payment handling and refund/dispute resolution at travel-specific scale and complexity, not a generic checkout flow.

Key Concepts

  • Split payments & multi-vendor settlement: one trip booking might pay an airline, a hotel, and a local tour operator; who gets paid when, and who eats a refund when one leg cancels?
  • Chargeback & dispute handling: travel has uniquely high dispute rates (cancelled flights, no-show hotels): a payments PM's core recurring headache.
  • Currency conversion & FX risk: who absorbs the exchange-rate movement between the booking date and the travel date, months later?
  • Refund policy as a product surface, not a policy doc: partial refunds, cash vs. credit, voucher expiry are UX decisions with real financial modeling behind each one.

Open Questions

How would you design a refund policy for a 3-vendor itinerary where only one vendor cancels? What's the actual dispute-resolution flow when a traveller claims non-delivery of a paid-for service?

What This Calls On

Payments infrastructure design, multi-party settlement logic, dispute/refund policy treated as product design rather than left as a support-team script.

T27

Cross-industry examples

Tools used: SQL, Postman, JIRA, Azure DevOps, Power BI, Salesforce — real work at Bajaj Finserv, not a portfolio exercise.

The Project

PCMS (Payout Calculation Management System) was built 0→1 at Bajaj Finserv, integrating with Finacle, Bajaj Finserv's core banking system. Partner payout calculations previously ran through a manual Excel process; connecting PCMS to Finacle let payouts calculate directly off live account data instead.

My Role

I owned the integration requirements end-to-end: scoping what data needed to flow, in what direction, and on what trigger, working directly with engineering. Integration type: fill in: REST / SOAP / batch file, whatever it actually was. I used Postman to test and validate API requests/responses during integration work.

SQL & Data Work

I drafted and validated KPI queries (SELECT, JOINs, GROUP BY / aggregations) before they went to stakeholders, the same pattern used for Power BI and Salesforce reporting. Sample query: fill in: a real query you've written, e.g. a payout reconciliation check.

Backlog & Sequencing

I ran the build in Jira and Azure DevOps: epics broken into stories, prioritized by what was actually blocking partner onboarding or a revenue number that sprint, ahead of a formal reach/impact score. fill in: a real example of re-sequencing this backlog under stakeholder pressure.

Outcome

FD booking turnaround dropped from 7 days to 1; the integration also removed a recurring manual reconciliation step.

What Transfers From This

Comfort scoping a technical integration directly with engineering, beyond writing a PRD and handing it off. The same rigor applies whether the "backlog" is a PRD's stories or a data-migration checklist.

The Project

fill in: what you actually built: an Airtable tracker, a Zapier/Make automation, a Webflow/Bubble prototype, or a Retool internal tool, using fill in: which of Zapier, Make, Airtable, Webflow, Bubble, or Retool you actually used.

The Problem

fill in: the manual process or gap this replaced. No-code was the right call because fill in: why this was faster to build without waiting on engineering time.

My Role & Outcome

fill in: did you build it solo, or scope it and hand off to someone else. Outcome: fill in: time saved, adoption, or what it replaced.

Reusable Insight

fill in

User Psychology

// behavioral analysis of a real user flow
T19

Cross-industry example

The Observation

A fintech app's KYC onboarding flow, watched on first open with no instructions. Emotional trigger: fill in. The friction point: a 7-step KYC flow where users abandoned at step 4 of 7. Motivation: fill in.

Behavioral Signals

The habit loop is broken — daily check-ins are expected by the product, but zero streak/return mechanism exists. The cognitive bias at play is anchoring — complexity on the first screen set user expectations for the whole flow. One user said "finally" on reaching the dashboard, read as relief, not delight. fill in: behavioral patterns, product implications, business implications

Lessons

User empathy is the most analytical thing a PM can do — it isn't a "soft" skill.

Career Lessons

// what four years across three companies actually taught
T23

Cross-industry example

Three Companies, Three Kinds of PM

BFSI (Bajaj Finance) → Telecom/enterprise (Ericsson) → Startup. Role and responsibilities at each: fill in.

Leadership Lessons — Ericsson

Enterprise PMs operate with global stakeholders and slow decision cycles; the PM has to "be the constant."

Product Lessons — the Startup

Speed is the product; ship, learn, adapt. PM scope is roughly 10× wider than in a large org.

Execution Lessons — Bajaj Finance

A data-first culture cut Fixed Deposit booking turnaround from 7 days to 1 and drove a 30% increase in sales-team operational efficiency through the FD Partner Onboarding App — real, quantified outcomes, also in the Impact Dashboard below.

Team & People Dynamics

Stakeholder conflicts, customer complaints, why specific projects succeeded or failed, people dynamics, and personal mistakes along the way: fill in.

Key Learnings

Each environment shaped a different kind of PM — the combination of BFSI rigor, enterprise stakeholder management, and startup speed is the actual differentiator.

Decision Journal

// real decisions, real trade-offs, real outcomes
T24

Cross-industry example

The Decision

fill in: a specific decision made, and what was happening at the time. Goal, constraints, and assumptions going in: fill in.

Weighing the Options

Available data, options considered, and their pros/cons: fill in.

The Call

Final decision, expected outcome, and known risks at the time: fill in.

What Actually Happened

Actual outcome and what would be done differently in hindsight: fill in.

Lessons Learned

Most PM decisions are "two-way doors" (reversible): decide fast, ship, learn, iterate. Most PMs mistakenly treat them like "one-way doors," which is why products move slowly.

Google PM Interview

// interview-format practice, STAR method and beyond
T25

Cross-industry example

The Question

"Tell me about a time you influenced without authority" — a leadership question, alongside product design, execution, estimation, and strategy questions worked through the same way: fill in.

The Framework

STAR — Situation → Task → Action (data, relationships, shared goals) → Result. Assumptions made going into the answer: fill in.

The Answer

fill in: your own real example; this is the story an interviewer will actually probe, worth writing in full.

Feedback & Improvements

Feedback received, common mistakes to avoid, and what to improve next round: fill in.

Reusable Insight

This question tests whether you can lead without a title, read stakeholder dynamics, and drive alignment without forcing it.

Personal Growth

// self-directed practice, tracked honestly
T26

Cross-industry example

The Challenge

A self-directed 6-month tracking exercise across weekly wins, failures, and feedback: fill in.

What Grew

Skills improved, skills still lacking, books and articles read, and PM concepts learned along the way: fill in.

Building Toward Something

Networking efforts, interview prep status, and monthly/quarterly goals against a longer career vision: fill in. Next action: launching the "PMgirlz" community — "no PM should figure this out alone."

6-Month Tallies

fill in: real counts per category, once case studies are actually published.

Biggest Insight

"Journals build judgment, not just knowledge. Judgment is the compound interest of a PM career."

← Back to the homepage case studies