Hypercore was originally written in JavaScript, this talk is about rewriting it in Rust and how that allows us to use it in ways not possible before: as a
cross-language, cross-platform library — as infrastructure. This is the story of how
that happened, what it took, and what we should build with it.

Hypercore is a peer-to-peer append-only log. Think of it like BitTorrent, but you can add
data to it. Like BitTorrent it enables censorship-resistant, decentralized data sharing.

I started working with Hypercore in 2023 when I wanted to build a peer-to-peer RSS protocol.
I built it in JavaScript using the existing Hypercore libraries. It
worked: I had a standalone client / aggregator / podcast app, and tools for converting RSS feeds into Hypercore feeds. See my blog post here.

It worked but it hit a hard wall. I couldn't give existing RSS clients my code as a library (unless their app was also written in JavaScript). To have a healthy swarm of peers we want as many clients as possible, but JavaScript is a walled garden when it comes to native integration. So integration into existing tools/clients/apps is difficult or impossible.

So I found the Datrs project — a group working to rewrite Hypercore in Rust. It was mostly
inactive, with most code half-finished implementations of now obsolete versions. But it had the potential to solve this problem of integration and portability so I dove into completing it.

Since then I've built out the parts needed for core functionality. I
implemented from scratch: Hyperbee, Hyperblobs, Corestore. I updated and
dramatically refactored every layer of peer-discovery, as well as replication
logic.

There are a lot of lessons to share: translating JavaScript's dynamic patterns into Rust's type system, reverse engineering undocumented wire protocols, converting existing Rust async code to use structured concurrency, and other fun things I found along the way.

Then there are things beyond just reimplementing Hypercore. I'll discuss experimenting with using Mozilla's UniFFI to generate libraries for: Python, Swift, and Kotlin. I'll also talk about building tooling for running inline JavaScript code from within Rust for integration testing.

As of today Hypercore is usable for basic cases: A peer can create a Hypercore, other peers can discover it and replicate the data. There are still rough edges and many TODO's in the code.

Toward the end I'd like to open discussion about what to build next — like peer-to-peer RSS or peer-to-peer package managers — and how Rust Hypercore fits with other P2P projects like p2panda, Holochain, IPFS, and iroh.