1. Hashing (Consistent Hashing, HashMaps)
Where it shows up:
Java
HashMap,ConcurrentHashMapAWS DynamoDB partitioning
Load balancing (sticky sessions)
Why it matters:
Fast lookup = O(1).
Without it, most systems collapse under scale.
Real-world example:
DynamoDB uses partition key hashing → spreads data across nodes
Consistent hashing → avoids massive reshuffling when nodes change
2. LRU / LFU Cache Eviction
Where it shows up:
Caffeine (Spring Boot default cache)
Redis eviction policies
CDN edge caching
Why it matters:
Memory is finite → eviction strategy determines performance.
Senior insight:
Bad cache strategy = hidden latency explosion
3. Rate Limiting (Token Bucket / Leaky Bucket)
Where it shows up:
API Gateway (AWS)
NGINX rate limiting
Spring filters
Why it matters:
Protects systems from abuse + traffic spikes.
Common algorithm:
Token Bucket → allows bursts but controls sustained rate
4. Circuit Breaker (State Machine)
Where it shows up:
Resilience4j (Spring Boot)
Envoy / Istio
AWS SDK retries
Algorithm nature:
CLOSED → OPEN → HALF-OPEN transitions
Why it matters:
Prevents cascading failures across microservices.
5. Retry with Exponential Backoff
Where it shows up:
AWS SDK (built-in retries)
Kafka consumers
REST clients
Why it matters:
Naive retries = amplify outages
Backoff = stabilize system under failure
6. Thread Scheduling (Work Stealing)
Where it shows up:
Java ForkJoinPool
Parallel streams
Virtual threads (Loom scheduling model)
Algorithm:
Work-stealing queues
Why it matters:
Maximizes CPU utilization with minimal contention.
7. Bloom Filters (Probabilistic Checks)
Where it shows up:
Caches (avoid cache penetration)
Databases (Cassandra, Bigtable)
CDN systems
Why it matters:
Fast “probably exists” check without hitting DB.
Tradeoff:
False positives allowed
No false negatives
8. Load Balancing Algorithms
Where it shows up:
AWS ALB / NLB
Service mesh (Istio, Envoy)
Common algorithms:
Round Robin
Least Connections
Weighted Routing
Senior insight:
Choice affects latency distribution under uneven load.
9. Distributed Consensus (Raft / Paxos)
Where it shows up:
Kafka (metadata quorum)
Zookeeper (legacy)
etcd (Kubernetes control plane)
Why it matters:
Keeps distributed systems consistent.
Reality:
You don’t implement it — but your system depends on it.
10. Sampling (Observability / Monitoring)
Where it shows up:
OpenTelemetry tracing
AWS X-Ray
Logging pipelines
Why it matters:
You can’t log everything → sampling decides what you see
Senior insight:
Bad sampling = debugging blind spots
Subscribe for the series.