🎯Basically, shared dictionaries help websites load faster by reducing the amount of data sent over the internet.
What Happened
Cloudflare has announced a new feature called shared dictionaries, aimed at improving web performance through smarter compression techniques. As web pages continue to grow heavier each year, the need for efficient data transfer becomes increasingly crucial. Shared dictionaries allow servers to send only the changes made to files instead of entire files, significantly speeding up page load times for users, especially those on slower connections.
The Problem
Web pages have been increasing in size by 6-9% annually, driven by more interactive and media-rich content. This growth is compounded by the rise of agentic crawlers and other automated tools that frequently request full pages, often just to extract small pieces of information. As a result, the web faces a bandwidth and CPU bottleneck, leading to wasted resources and slower load times for users.
What Are Shared Dictionaries?
Shared dictionaries serve as a reference point between the server and the client. Instead of compressing responses from scratch, the server can recognize what the client has cached and only send the new data. This method drastically reduces the amount of data transferred, as it avoids sending redundant information. For instance, if a minor change is made to a JavaScript bundle, shared dictionaries enable the server to send only the differences, or deltas, instead of the entire file.
Delta Compression
Delta compression is a key feature of shared dictionaries. When a resource is first served, the server instructs the browser to cache it for future use. On subsequent requests, the browser informs the server of its cached content, allowing the server to compress the new version against the old one. This method ensures that only the necessary changes are sent, leading to significant bandwidth savings.
Why the Wait?
Despite the potential benefits, shared dictionaries have not been widely adopted due to past failures, notably Google's SDCH in 2008, which faced security issues and architectural challenges. The new standard, RFC 9842, addresses these concerns by enforcing stricter rules on dictionary usage, thereby enhancing security and compatibility. While Chrome and Edge have implemented support, full cross-browser adoption is still in progress.
How Cloudflare is Building Support
Cloudflare is rolling out shared dictionary support in phases, starting with passthrough support that allows headers and encodings required for shared dictionaries to be forwarded without modification. This phased approach aims to make the implementation accessible to all users, paving the way for improved web performance across the board.
🔒 Pro insight: Shared dictionaries could revolutionize web performance, but careful implementation is key to avoid past security pitfalls.
