Challenge
A global confectionery brand ran a high-volume consumer promotion requiring participants to submit purchase receipts as proof of qualifying purchases. The campaign attracted tens of thousands of submissions within weeks of launch, spanning receipts from retailers across multiple countries, currencies, and receipt formats — printed, handwritten, and photographed under varying lighting conditions. Manual verification was ruled out at that volume, but accuracy was non-negotiable: incorrectly accepting ineligible entries risked promotional budget overrun, while incorrectly rejecting valid submissions created consumer complaints and brand damage. Existing simple OCR solutions failed on low-quality images and non-standardized layouts at an unacceptable rate.
Approach
We built a two-stage validation pipeline using an ensemble architecture that combines dedicated OCR with a large language model reconciliation pass. Submitted receipt images are first processed by AWS Textract, which performs structured field extraction — retailer name, date, line items, totals, and any promotional product identifiers. The raw extraction output is then passed to Google's Gemini model, which acts as an intelligent reconciliation and validation layer: cross-checking extracted fields for internal consistency, inferring partially obscured values from context, mapping retailer and product names against the eligible product catalog, and verifying that the purchase date falls within the campaign window. Entries that pass both layers are marked as validated; entries where the two models disagree or confidence falls below threshold are flagged for targeted human review rather than silently accepted or rejected. The pipeline processes each submission asynchronously and returns a structured validation result with field-level confidence scores.
Outcome
The pipeline processed over 50,000 receipt submissions across the campaign window with an end-to-end validation accuracy of 94%, compared to an estimated 71% accuracy from single-model OCR tested in pre-deployment benchmarking. Processing time per submission averaged under 8 seconds, enabling near-real-time participant feedback. The targeted human review queue captured roughly 4% of total submissions — edge cases that genuinely required judgment — allowing a small review team to focus exclusively on ambiguous entries rather than reviewing everything. Campaign budget exposure from incorrectly validated entries was reduced by an estimated 80% compared to projections under a simpler automated approach.