Avatar
T. Chardin

The road to a decentralized CDN

The road to a decentralized CDN

If you’re familiar with IPFS, you might know that most content on the IPFS network is accessed via a gateway: an HTTP server that acts like a bridge between your browser and the network. This is because browsers only support HTTP and WebRTC protocols natively, hence unless the web page is running a Javascript IPFS node or interfacing with a local go IPFS node, it is not able to load content from the IPFS network directly. This “gateway setup” is how apps currently interface with most Web3 protocols (e.g. Ethereum, Arweave, etc.) and introduces a host of problems. Most notably these gateways create latency, bandwidth bottlenecks and single points of failure into p2p networks, all of which are detrimental to our mission to build a performant and robust decentralized content delivery network (dCDN).

Myel points of presence (PoPs) are similar to IPFS nodes in that they use the same networking protocol (Libp2p) and data format (IPLD). These data and networking standards enable full interoperability with the IPFS and Filecoin networks but introduce the same challenges when trying to interface from browsers and mobile apps. To compound this, PoPs also require access to a wallet to secure the private keys used for peer identity and payments (payments for content delivery are currently issued in Filecoin’s currency FIL). Though native support for these standards in browsers is still in its infancy, our aim from the start has been to tackle these interoperability challenges head on to provide the best experience for developers using our dCDN. We’ve been laying down a path from today’s browsers and native platforms towards a web browsing future with native integration of p2p networking and wallet stacks.

A core component of this path is our browser client, which implements some of the functionality of PoPs into Javascript. As a first step to improving on the “gateway setup” described above, we’re taking advantage of Cloudflare’s edge network and running Myel browser clients directly inside Cloudflare workers. This lowers latency and greatly improves performance while maintaining resilience and interoperability relative to using gateways. Developers can run their own worker client and import their private key as a secret to pay for their users’ data transfers or use our public worker instance which subsidizes the costs of retrieving content (but may throttle requests). On a more technical level, IPLD data blocks are directly streamed from a PoP to the browser over a long running connection that runs via the worker. We’ve implemented an IPLD blockstore interface using the worker KV (key-value) API, which allows for the caching of popular data blocks on the Cloudflare edge. This diminishes data transfer transactions upon repeat requests. For instance, if a user reloads a page and the browser isn’t caching the HTTP response, the worker can stream the blocks directly from its cache instead of making a request to a PoP.

Cloudflare worker latency map

This same Javascript client implementation can also run in a service worker. Requests for content are proxied via the service worker which itself then requests that content directly from a PoP, enabling direct browser-network connections ! This approach brings the best performance and most control but also requires access to a local wallet. As there are currently no Filecoin wallet APIs available, this solution is not usable for paid transfers of content. However, once the Metamask integration of Filecoin becomes publicly available, the service worker client will be a great option for developers who want to cut the costs of a Cloudflare worker and bring the highest level of decentralization to their app’s content delivery.

In summary, we are building a hybrid dCDN network with a variety of options for accessing cached content. Until wallet UXs greatly improve, Cloudflare workers are a great solution for bridging dApps with our decentralized content network whilst maintaining performance. When the protocols we use are native to browsers, one could imagine a dApp loading its app source code and client service worker via a Cloudflare worker and then loading larger content directly from the Myel network; or a UI which decides whether to fetch content using a local service worker or Cloudflare worker based on whether a user has a wallet installed. This ability to combine approaches will give developers a more rich and flexible toolkit that combines the strengths of existing centralized infrastructure with the resilience and robustness of decentralized architectures.