Perplexity has moved the core serving tier of its AI search engine off Amazon DynamoDB and onto CobbleDB, a distributed key-value store the company wrote itself in Rust. As InfoQ reported, drawing on Perplexity's own engineering write-up, the change cut batch-read latency roughly fivefold and lowered storage costs by at least 20 percent. The story is interesting for two reasons: what it reveals about the storage needs of AI applications, and what it says about when building your own infrastructure starts to make sense.
Why AI search stresses a database differently
A traditional search engine returns short snippets and links. An AI answer engine has to feed a language model, which means it retrieves full passages and vector embeddings for many pages per query. According to the InfoQ report, each Perplexity query produces around 100 to 120 page keys, split into parallel batches of 10 to 20, and the average record is roughly 50 kilobytes. With traffic above 200,000 requests per second, that adds up to a huge volume of bytes read.
DynamoDB is an excellent managed service, but two things made it a poor fit here. First, cost: its pricing is based on usage, and reading large records at this volume became very expensive. Second, control: as a fully managed service it hides partition placement, caching and replica routing. Perplexity's engineers could not do much about tail-latency spikes caused by cold reads, cross-zone hops or slow replicas. A single slow batch delays the whole answer, so the p99 matters a great deal. Bulk reprocessing jobs, triggered when the team changed chunking or embedding models, also wrote heavily into the same tables and competed with live traffic.
Three systems instead of one
Rather than replace DynamoDB with one bigger database, Perplexity split responsibilities across three components.
Pillar is the durable system of record, built on YTsaurus, the open-source big data platform originally developed at Yandex, running on high-capacity hard drives. It stores versioned tables of page metadata, passages and embeddings, and uses transactions so that crawl updates and export queues commit together.
Lorry is a stateless batch layer. It reads exports from Pillar, groups them into files aligned with CobbleDB partitions, uploads them to Amazon S3 and notifies CobbleDB that new data is ready.
CobbleDB is the hot serving store. Its nodes pull those batch files from S3 on their own schedule, which keeps heavy write traffic away from the machines answering user queries.
How CobbleDB is built
CobbleDB is deliberately narrow. It is optimized for one job: batched key lookups. Each partition has three replicas on separate nodes, and each node uses RocksDB as its storage engine on local NVMe drives with memory-mapped caching. A stateless router hashes page identifiers to partitions, prefers replicas in the same availability zone to avoid cross-zone network costs, and hedges slow requests by sending a parallel read to another replica when one is lagging. Within a node, it uses RocksDB's batched MultiGet to fetch many keys in one call.
Just as important is what CobbleDB leaves out. There are no distributed transactions and no synchronous consensus protocol. Replicas apply updates asynchronously, at their own pace. That would be unacceptable for a bank ledger, but search results can tolerate a little staleness, and dropping those guarantees removes a lot of complexity and coordination overhead.
The results, with caveats
Perplexity reported that median batch-read latency fell from 31.4 ms to 5.6 ms, p90 from 56.7 ms to 9.77 ms and p99 from 123 ms to 24.2 ms. Synthetic benchmarks with payloads up to 100 kilobytes reportedly sustained up to 500,000 requests per second.
Other coverage of the announcement points out that the DynamoDB numbers were recorded before the cutover and the CobbleDB numbers afterwards, so this was not a side-by-side test on identical traffic. The figures describe one company's workload, not a general verdict on DynamoDB.
The other headline: AI-assisted development
According to InfoQ, Perplexity CEO Aravind Srinivas said the roughly 40,000 lines of Rust were written in about two months by two systems engineers working with a large group of AI coding agents, which helped with integration testing, build monitoring and operational runbooks. Perplexity has also said it intends to open-source CobbleDB, though no date has been given. It is a notable data point about what small, experienced teams can attempt with agent support, but the engineers' deep systems knowledge clearly mattered: the design choices are what made the project work.
Trade-offs worth weighing
Leaving a managed database is never free. Perplexity's site reliability engineers now own node lifecycle, backup verification and partition rebalancing, work that AWS used to absorb. Applications must also cope with eventual consistency, since replicas may ingest new batches at different times. And a bespoke database is only worth it when the workload is both very large and very specific. Most products never reach a point where those conditions hold.
Practical takeaways for software teams
Understand your read pattern in detail. Payload size, batch size and how latency compounds across parallel calls can matter more than raw request counts.
Separate durable storage, bulk ingestion and hot serving when write-heavy pipelines start to hurt user-facing latency. This pattern works with off-the-shelf tools too.
Look at tail latency, not just averages. Techniques like zone-aware routing and request hedging can help on many platforms.
Ask which guarantees you actually need. Relaxing consistency where the product allows it can simplify the system dramatically.
Build your own only when cost and control problems are structural, and budget for the operational work that comes with it.
Treat headline benchmarks carefully, especially before-and-after comparisons published by the team that did the work.
CobbleDB is a good example of AI workloads reshaping familiar infrastructure. For most teams the lesson is not to write a database, but to look closely at how their data is really read and to design storage layers around that.
Source: Olimpiu Pop, InfoQ, original article linked below.
