Retail Wallet Platform — Architecture & Capacity Specification

Section 1Scale assessment

1.2 Traffic directions

• API in — Embargo app / backend → Partner API
• Provider events in — Wallester, Stripe, Plaid → events-ingest
• Provider calls out — platform → Wallester, Stripe, Plaid
• Webhooks out — platform → Embargo endpoint
• DB writes — rows written in PostgreSQL

Section 2Architecture

• Region and network
  • AWS eu-west-2 (London), three availability zones
  • Private subnets; only the load balancer is internet-facing
• Managed services only — ECS Fargate, SNS FIFO, SQS FIFO, RDS PostgreSQL, ElastiCache Redis, ALB, WAF, KMS, Secrets Manager, CloudWatch
  • No EC2 hosts to patch or log into
• ECS on Fargate
  • Health checks, rolling deploys, autoscaling, multi-zone placement — no cluster to operate
  • VPC-native task networking: no overlay network, no extra proxy hop between load balancer and service
• CI/CD and infrastructure as code
  • AWS CodePipeline and CodeBuild test, build and deploy container images to ECS Fargate
  • Infrastructure is managed with AWS CDK
• One codebase, six roles — Go 1.26 (version policy: latest stable minus one); roles scale independently (§2.1)
• Funds system of record: Wallester — card accounts, authorisation balances, customer money
• Platform PostgreSQL
  • Transaction ledger (mirror of processor events), rewards ledger, wallet ↔ card mapping, merchant catalogue, reporting
  • Wallet balances are a projection of Wallester events and the source of every balance read — no call to Wallester on the read path (§6)
  • Reconciled nightly against Wallester statements and Stripe payouts
  • Dedicated instance per partner programme — Embargo shares no database with another partner
• Redis — cache and rate limiting (partner rate limits, partner idempotency keys for 24 h, hot balances), never the source of truth; not used for event deduplication or job coordination
  • Losing it costs latency, not correctness: reads fall back to PostgreSQL, and rate limiting falls back to per-task limits (§6)
• Not in the card-authorisation path — authorisation and spending controls execute at the processor; the platform receives the resulting events
• Device side — the Embargo app embeds the Retail Wallet Mobile SDK
  • Card data → Stripe, bank authorisation → Plaid, wallet tokens → Google / Apple, straight from the device — never via the platform or the partner backend. Card data is collected directly by the payment provider, which reduces PCI scope; the applicable PCI obligations are confirmed for the integration
  • The app talks only to the Embargo backend; the Partner API is server-to-server

2.2 Compute and storage sizing

Section 3Data and durability

• Residency
  • Platform data is hosted in AWS London (eu-west-2), with backups in Ireland (eu-west-1)
  • Third-party processing locations are documented separately — Provider dependencies
• Redundancy
  • RDS Multi-AZ with synchronous standby — RPO 0 for committed database transactions in a zone failure
  • Redis, SNS and SQS Multi-AZ; services in three zones
• Backups
  • Continuous point-in-time recovery, 35-day window
  • Daily snapshots kept 12 months; cross-region copy within 15 min
  • Quarterly timed restore drill
  • Every provider event is kept in the SNS FIFO archive for 90 days and can be replayed for a chosen time range (§6)
• Recovery targets
  • Regional recovery targets are RPO ≤ 5 minutes and RTO ≤ 60 minutes, subject to validation through an end-to-end recovery drill before launch
  • Zone failure: expected automatic recovery in 1–2 minutes with no data loss (synchronous standby); the time may be longer and is measured in failover tests
  • Card authorisation and customer funds sit with the issuer and follow its recovery objectives: RTO 1–4 hours depending on component, database RPO about 5 minutes (Wallester, 14 September 2026)
• Encryption
  • AES-256 at rest (KMS, customer-managed keys): database, queues, event archive, cache, backups, logs
  • TLS 1.2+ in transit, including service-to-service
• Retention
  • Financial records 7 years
  • Provider event archive 90 days; event, reward and payout keys are kept at least as long, so a replay can never apply an event twice
  • Personal data per the DPA with Embargo; deletion on request with documented exceptions
• Integrity
  • Append-only ledger; balances derived, never overwritten
  • Nightly three-way reconciliation — platform ledger vs Wallester statements vs Stripe payouts; any difference pages on-call

Section 4Monitoring, alerting, on-call

• Dashboards
  • CloudWatch + Grafana
  • Embargo gets a read-only dashboard of its own traffic: throughput, latency, error rate, webhook delivery
• On-call and incident response
  • Incident coverage, response targets and escalation contacts will be agreed before production launch
  • Proposed: 24/7 engineering rota (PagerDuty) from launch; P1 acknowledged within 15 min; Embargo technical contact notified within 30 min with updates every 30 min; written post-mortem within 5 business days; public status page
• Changes
  • Rolling deploys, 10% canary, automatic rollback on error-rate increase; rollback < 5 min
  • Database engine upgrades in agreed maintenance windows, announced to Embargo in advance

Section 5Operating targets

Figures are design targets; each is validated by the named test or drill before production launch. Contractual service levels are set in the partnership agreement.

Section 6Questions from Embargo

Server type, OS, count, CPU

• No servers under management. AWS Fargate containers, Amazon Linux 2023 base.
• Baseline 18 tasks: 31 vCPU / 62 GB across three zones (§2.2). Per-role autoscaling maximums are listed in §2.2 and are configuration limits.
• Database: RDS PostgreSQL db.r6g.xlarge (4 vCPU / 32 GB, Graviton) Multi-AZ + one read replica; gp3 1 TB with storage autoscaling.

Language / framework

• Go 1.26; standard-library HTTP server; pgx for PostgreSQL; no heavy framework.
• Version policy: latest stable minus one — we move up within a release of each new Go version, never running an unsupported one.
• Predictable latency under load; ~15 MB static binary per service; cold start < 1 s.

Load balancer

• AWS Application Load Balancer, cross-zone, TLS 1.2+ termination.
• Health check every 10 s; two failures remove a target; 30 s connection draining on deploy.
• AWS WAF in front: managed rule sets + per-IP rate rule.

Rate limiting

• Per partner — token bucket per client_id; 429 + Retry-After + X-RateLimit-* headers. One partner cannot affect another.
• If Redis is unavailable
  • After a short Redis timeout each api task switches to its own in-memory token bucket
  • The partner's rate and burst are divided by the maximum number of api tasks that can run at once, including deploy overlap, and new buckets start with a conservative fill — during the failure a partner can get somewhat less than its limit, never more
  • Shared limiting resumes when Redis recovers; the fallback is alerted and tested with Redis failure, restarts and rollout overlap
• Embargo's limits — reads 2,000 req / s (burst 5,000); writes 500 req / s (burst 1,000); bulk 10 req / min. Raised on request as volume grows; the same figures are published in the API reference.
• Edge
  • The API is public-facing for the Embargo backend and merchant systems, behind AWS WAF: managed rule sets, 2,000 requests / 5 min per IP for unknown sources; partner IPs allow-listed
  • AWS Shield Standard is always on. If volumetric DDoS becomes a pattern, AWS Shield Advanced is the escalation — 24/7 DDoS response team and cost protection

Idempotency and duplicate handling

Every entry point has one key on which a message is processed at most once at a time; a duplicate replays the original result instead of acting again. Duplicate prevention relies on durable operation identifiers and database constraints; timed-out provider operations remain pending until their outcome is confirmed.

• Top-up sent twice
  • Same Idempotency-Key (a retry): the same topup_id and payment session come back — one PSP payment, one credit
  • Different keys for the same intent (a client bug): two independent top-ups, by design; the monthly cap and the app's single-in-flight rule bound the damage, and both are visible in GET /topups
• Spend or withdrawal
  • Authorisation happens once, at the issuer; a duplicated transaction.authorized webhook is dropped by event_id — one ledger line
  • Cashback payout and discount return are credits with their own credit_id; a retried job cannot pay twice

In-flight operations and crash recovery

The case that matters: a process dies between writing to the database, sending a message and calling a provider. The design makes such an operation detectable and resumable rather than assuming it does not happen.

• Record first, act second — transactional outbox
  • Every operation with a side effect — top-up, provider credit, outbound webhook — is written as a row with status pending in the same database transaction as the ledger change; the message send, provider call or delivery is performed from that row, never only from memory
  • A process that dies after the commit leaves a pending row; one that dies before it leaves nothing — either way there is no half-applied state
• Inbound events survive a crash
  • events-ingest returns 200 to the provider only after the publish to SNS FIFO has succeeded; the topic archives the event for 90 days and delivers it to the SQS FIFO queue, which keeps it for 14 days
  • ledger-worker deletes a message only after its database transaction commits; a worker that dies mid-way lets the message reappear after the visibility timeout, and the unique processed_events row makes the second apply a no-op
• Provider calls that time out stay pending
  • The operation identifier is written before the call is made — the Idempotency-Key for Stripe, the card external_id or transfer reference for Wallester, and our top-up id as the Plaid payment reference, which Plaid does not treat as an idempotency key
  • A recovery job runs every 30 s: for each operation pending longer than 60 s it establishes the outcome through each provider's own mechanism — Stripe: repeat the request with the same Idempotency-Key, which replays the stored result within its 24-hour window, and after that window find the PaymentIntent by our top-up id in its metadata; Wallester: look the card up by external_id or the transfer by our reference; Plaid: read the payment by payment_id when one was returned, otherwise match our reference in the payment list before any new create — and only then completes, retries or marks it failed; nothing is retried without a safe key or a lookup
  • Operations still pending after the recovery window are surfaced on the reconciliation dashboard and alert on-call; they are never silently dropped or marked done
• Replay from the archive
  • Events for a chosen time range are replayed through a separate recovery subscription and FIFO queue, at a limited rate, with the same consumer code and uniqueness constraints — events a consumer missed are applied, events already applied are skipped
  • A corrected reward rule is applied by a recalculation job that reads the archived events and posts reviewed adjustment entries; purchases are never applied twice
  • Replay is restricted to recovery operators; each run records range, queue, rate and result
  • The archive complements provider reconciliation and backups; it does not replace regional recovery
• Outbound webhooks are stored, not fire-and-forget
  • Each delivery is a row with an attempt count and next-attempt time; the dispatcher marks it delivered only on a 2xx; undelivered rows are picked up by the next dispatcher, including after a crash
• What this promises — and what it does not
  • An unconfirmed money operation cannot disappear from view and cannot be applied twice by our own retries
  • It does not promise that a provider never duplicates on its side — which is why the provider's own uniqueness key is always set and why reconciliation against provider statements runs nightly

Balances — served from our event projection

• Where a balance read comes from
  • The balance projection in PostgreSQL, kept hot in Redis; the read path never calls Wallester
  • Each wallet returns balance_minor, available_balance_minor and updated_at, so the app always knows how current the figure is
• How it stays current
  • Wallester dispatches authorisation, reversal, release, clearing and transfer events immediately after processing (Wallester, 14 September 2026); ledger-worker applies them in wallet order
  • A card payment normally shows in the balance within seconds
  • Our own credits — top-up load, cashback payout — change the balance when Wallester confirms the transfer, and are shown as pending until then
• Why a lagging projection cannot cause overspend
  • Wallester authorises every payment against its own account balance; the projection only displays
  • The worst case is a balance in the app that is a few seconds old, with its updated_at to show it
• How drift is caught
  • For each account with recent activity, the sum of our wallets on that Wallester account is compared with GET /v1/accounts/{account_id} — balance, blocked_amount, available_amount — at a rate-limited pace outside peaks
  • Nightly reconciliation against Wallester statements covers every account; a missed event is recovered from statement-by-cursor
  • Any difference corrects the projection to Wallester's figure, is logged with both values and alerts on-call
• During a Wallester outage
  • Balances are still served from the projection with their updated_at; card payments depend on Wallester either way
• Why not a live read from Wallester on each request
  • Balance reads peak at up to 1,250 / s for about two minutes after an offer push — above the whole-platform peak Wallester reports
  • Wallester publishes no API latency figures, so a live read would put an unmeasured dependency in every app open
  • Merchant-locked wallets are a split that exists only in our ledger; Wallester holds the account total

Refunds, reversals and cashback

• Cashback accrues on clearing, never on authorisation
  • A purchase TransactionClearing creates the accrual with status pending for the merchant's pending_days, which covers the usual refund window
  • An authorisation reversed or released before clearing — ReversalAuthorization, ReleaseAuthorization — never earned cashback; the transaction becomes reversed and nothing is paid
• Refund while cashback is pending
  • A clearing with group = Refund is matched to the original purchase by authorization_id when present, otherwise by card, merchant and acquirer reference number
  • Full refund: the accrual becomes reversed. Partial refund: the accrual is recalculated on the amount that remains
  • No money has moved, so nothing is taken back; Embargo receives transaction.reversed and the updated cashback_status
• Refund after cashback was paid
  • The paid amount is not debited from the customer's card account; it is recorded as a negative reward entry and offset against the customer's next cashback
  • The reward's funder — merchant, partner or Retail Wallet — carries the reversal in the settlement report (reversed_minor); an offset not recovered within the cashback expiry period is written off and reported
• Unmatched refunds and chargebacks
  • A refund that cannot be matched to exactly one purchase, and any chargeback, goes to the reconciliation queue and changes cashback only after review
  • Each reversal is an append-only ledger entry keyed on the Wallester transaction id, so a repeated event cannot reverse cashback twice

How a Wallester refund references the original purchase is being confirmed with Wallester. After payout the programme rule applies: a reversed cashback is offset against future cashback and is never debited from the card.

Database sharding

• Not sharded — not needed at this volume. One dedicated Multi-AZ PostgreSQL per partner programme; Embargo shares no database with another partner.
• Reporting runs on a PostgreSQL read replica with indexes, monthly partitions and precomputed aggregates.
• Heavy exports run asynchronously with bounded parallelism, never on the primary.

Message queues — receiving and processing events

• Amazon SNS FIFO topic with a 90-day archive in front of an Amazon SQS FIFO queue; message group = wallet_id, so one wallet's events apply in order.
• Ingest: verify per provider (Stripe HMAC signature, Plaid JWT, Wallester Basic-auth credentials + source IP allow-list) → publish with the provider's event identity as deduplication ID → 200 after the publish succeeds; no business logic before acknowledgement. Wallester retries only the event types where guaranteed delivery is enabled, and the Mandatory policy is used wherever it is offered, so gaps are closed by reconciliation against its statement API, not by waiting for retries.
• Workers apply idempotently, with the lasting duplicate check in PostgreSQL; 5 retries with backoff, then the processing dead-letter queue + page. A separate subscription dead-letter queue catches SNS-to-SQS delivery failures. Queue retention 14 days; archive 90 days, replayable by time range.

Microservices

• Six roles from one codebase: api, events-ingest, ledger-worker, webhook-dispatcher, scheduler, reporting (§2.1).
• Independent scaling; shared state only through PostgreSQL, SNS and SQS; one repository, one build, one pipeline.

What request rate must the platform support?

• Partner API, from Embargo's backend
  • Reads: ~18 / s average · 125 / s at lunch · 1,250 / s for ~2 minutes when an offer push lands
  • Writes: ≤ 27 / s (onboarding wave + top-ups)
  • Embargo's production tier sits above this: 2,000 / s sustained, 5,000 / s burst
• Inbound events from Wallester, Stripe, Plaid
  • ~7 / s average · 28 / s at lunch
  • 100 / s when a slot offer opens across 3,000 venues
  • 74 / s for an hour in the nightly settlement batch
• Outbound webhooks to Embargo
  • 34 / s at lunch · 130 / s in the evening burst · 126 / s during settlement
  • Embargo's webhook endpoint should accept ≥ 150 requests / s; below that we buffer and retry — delivery slows; deliveries are stored and retried for 24 h, then surfaced for re-delivery
• Design targets
  • 5,000 API reads / s · 1,000 writes / s · 1,000 events / s in · 1,000 webhooks / s out · 3,000 DB writes / s — to be validated under mixed load in the pre-launch load test
  • End-to-end throughput depends on provider quotas and partner processing capacity

Transaction capacity

• Design target: 1,000 events / s sustained (≈ 86M / day). Capacity figures are design targets to be validated under mixed load; end-to-end throughput depends on provider quotas and partner processing capacity.
• Delivery target: p95 under 5 seconds from durable receipt by Retail Wallet to acknowledgement by Embargo during normal operation; retry delivery may take longer. Latency from the processor to Retail Wallet is measured separately and is not part of this target.
• Embargo at 5M users: 3 purchases / s average, 22 / s lunch peak, 100 / s burst.
• Sharp peaks: the minimum task count per role is configuration — raised ahead of planned pushes or campaign days so a burst is absorbed at once; autoscaling covers the rest.

Does Embargo integrate with Google, Apple, Stripe or Plaid?

• No. One integration: the Partner API server-to-server, plus the Retail Wallet Mobile SDK in the app. No Embargo requests go to Google, Apple, Stripe, Plaid or the issuer.
• Wallet provisioning: the SDK reads device identifiers, the Partner API returns the issuer's encrypted payload (POST /cards/{id}/google-pay), the SDK adds the card; card.wallet_status confirms.
• Two administrative items only the app owner can supply: Android package name + SHA-256 signing fingerprint (Google allow-listing via the issuer); Apple in-app provisioning entitlement for Embargo's developer account (issuer sponsorship letter from us; Apple's lead time, typically weeks).

Monitoring and alerts · failure plan, RPO / RTO · penetration test · SLA

• Monitoring and alerts: §4 · Failure plan: §3 — regional recovery targets RPO ≤ 5 min / RTO ≤ 60 min, validated by an end-to-end drill before launch; zone failure expected to recover automatically in 1–2 min · Penetration test: external firm before go-live and annually · SLA: §5.