Obed Osei Frimpong — home
All work

/ Case study · Employer

2,000 TPS messaging middleware for a global retailer

A high-availability messaging layer in Go and Java that filtered, processed and logged customer messages at scale — plus the Kubernetes migration that cut deploy time by two-thirds.

Role
Senior backend engineer
Context
Employer
Status
Shipped
Domain
Retail · Customer messaging · Platform
Messaging middleware architectureUpstream producers send messages through Kafka to an ingest service, then through filter, process and enrich, and log steps (the message log is stored in ELK) before dispatch to delivery channels. A React template and tracking dashboard reads from the log and writes templates to the process step. All services run inside a Kubernetes cluster managed with Rancher and deployed through Jenkins CI/CD.Kubernetes (Rancher) · Jenkins CI/CDreadstemplatesUpstreamproducersIngest(Kafka)FilterProcess /enrichLog(ELK)Dispatch(channels)Template & tracking dashboard(React)filtered and logged before dispatchdeployment boundary
Architecture overview · scroll sideways to see all of it

The problem

Decathlon needed to message customers reliably and at volume, with filtering, processing and logging applied before anything was sent. Product owners also needed to create message templates and see what had gone out, without an engineer in the loop.

Constraints

  • High availability: messaging is customer-facing, and dropped or duplicated messages are visible.
  • Several upstream producers with different formats and reliability.
  • Deploys needed to be fast and routine, not events.

Architecture

  • Ingestion services accept messages from upstream systems.
  • A pipeline of Go and Java microservices filters, enriches and logs each message before it is dispatched.
  • Kafka carries messages between the services, and the message log is stored in ELK (Elasticsearch, Logstash, Kibana).
  • Delivery goes out through channel adapters.
  • A React dashboard lets product owners manage templates and track sent messages.
  • The platform runs on Kubernetes (managed through Rancher) with Jenkins CI/CD.

Key decisions

  • Go for the hot path, Java where existing domain services lived — matching the language to the job and the team, not to fashion.
  • Filtering and logging before dispatch, so every message sent is explainable after the fact.
  • Migrating from VM-based Docker to Kubernetes to make deploys boring.

How it’s evaluated

  • Throughput and availability under production load (sustained ~2,000 transactions per second).
  • Deploy lead time before and after the migration.

Outcome

  • Messaging middleware handling about 2,000 TPS (transactions per second), improving customer interactions.
  • Deploy time: 30 → 10 min after the Kubernetes migration.
  • Product owners self-serve templates and tracking through the dashboard.

What I’d do next

The same discipline — filter, log, then act — is exactly what AI agents need before they touch customers. Today I’d add an LLM classification step behind an eval suite, with the existing log as the audit trail.

Stack

  • Go
  • Java (Spring)
  • Microservices
  • Kafka
  • ELK
  • Kubernetes
  • Rancher
  • Jenkins
  • Docker
  • React
  • MySQL
  • Redis