OpenAI details Habitat storage redesign and lower median read latency
OpenAI published an October 23 engineering account of Habitat, its online application storage platform. The company says the platform serves more than one billion users across 40 geographic regions and more than 500 petabytes of data. These are OpenAI's reported operating figures, not an independent audit or a new product offering.
The post describes a shift from one shared Azure Cosmos DB container to separate containers for critical workloads, and then back toward shared multi-tenant containers. Isolation reduced the impact of some failures but made each product team responsible for forecasting and provisioning launch capacity, creating operational overhead.
OpenAI says it worked with Microsoft's Azure team on hot-key throttling, request-priority controls and an improved serverless mode. The reported design allows physical partitions to scale to 10,000 request units per second and split based on throughput usage as well as storage usage. That unit is database request capacity, not a model token rate or a per-user guarantee.
The company reports that its largest Cosmos DB account holds tens of petabytes and serves more than 20 million queries per second. That is the stated load of one account, distinct from Habitat's total stored-data figure. The post also recounts earlier regional failover and connection-surge outages, explaining why shared testing and direct engineering coordination became necessary.
For reads, OpenAI says data placement, replication, Valkey caching and a Rust rewrite helped reduce global median latency from roughly 120 milliseconds to roughly 10 milliseconds, a twelvefold improvement. Median read latency is not a promise about every request, ChatGPT response generation or the slowest requests; the post says uncached, cross-region reads remain a challenge.
The practical lesson is about running shared storage under rapid growth: workload isolation, capacity planning and cloud-provider coordination interact. The article is a description of OpenAI's own infrastructure and tradeoffs. It does not announce general availability, pricing or contractual service levels for a customer-facing Habitat service.