harshaNOTES FROM THE PROCESS
Thinking,
out loud.
A space for ideas that don’t fit in a repository. Notes on engineering, learning, and making sense of things.
AI infrastructure
Nine billion tokens, and the work behind the number
What a shared platform’s throughput says—and what it leaves out.
AI infrastructure
An AI gateway is a policy boundary
Routing a request is easy to draw. Deciding whether it should be routed takes more care.
Reliability
Failover should preserve the contract
A backup provider is useful only when the application understands what changed.
Real-time systems
Streaming is a lifecycle, not a text effect
The first token is the beginning of a conversation with the interface, not the end of the engineering.
Real-time systems
The message you missed still matters
Choosing a delivery mechanism means choosing what the system remembers.
Real-time systems
Cancellation is resource management
A stop button should have a meaning all the way through the stack.