Building Microservices That Actually Scale
After migrating a payment service from a legacy monolith to microservices at Wallet Codes, I learned some things the hard way. Here are the patterns that actually worked.
Start with the Boundary, Not the Technology
The biggest mistake teams make is choosing a tech stack before defining service boundaries. We spent two weeks on event storming and domain mapping before writing a single line of code — and it paid off.
A good service boundary answers:
- What business capability does this own?
- What data is it the source of truth for?
- What happens if it goes down?
The Database Per Service Rule
Shared databases between services create hidden coupling. We enforced strict database isolation — each microservice owns its data and exposes it only through an API.
For the payment service, this meant:
- Transaction service — PostgreSQL, owns payment records
- Wallet service — Redis-backed for low-latency balance checks
- Notification service — simple PostgreSQL queue
Async Communication with Guarantees
We used RabbitMQ with a dead letter exchange pattern. Every message that fails processing is routed to a DLQ with contextual metadata — service name, original payload, error reason, and timestamp. This made debugging cross-service failures tractable.
Graceful Degradation
Not every service needs to be up for the system to function. We defined three tiers:
- Critical path — transaction processing (must be up)
- Supporting — notification, reporting (can degrade)
- Best-effort — analytics, recommendation (can fail silently)
Each service has a circuit breaker. If a downstream service is down, the upstream has a fallback — cached response, queued task, or degraded UI.
What I’d Do Differently
- Observability from day one — structured logging, metrics, and tracing should be non-negotiable from the first microservice, not retrofitted
- Fewer services — start with 2-3 coarse-grained services, split later when the boundary is proven
- Contract testing — Pact-style consumer-driven contracts catch breaking changes before deployment
Microservices aren’t the right choice for every project. But when you genuinely need independent deployability and team autonomy, the patterns above keep things from turning into a distributed monolith.