I am creating a successor
Amidst the site being down, I started thinking about creating a successor. However, I don't have, nor am I willing to spend, my own funds on it. A project like this faces three main challenges:
- Legal
- Service
- Money
The site would need to be resistant to takedowns, which heavily depends on the hosting, storage, and network providers being used. Storage is a major concern, as archiving content at scale requires a significant amount of space.
Ads could be a solution if they cover operating costs, but the service side remains an open question. Who provides DMCA-resistant storage and networking? Cloudflare can provide networking, but it is not DMCA-resistant. Alternatives could include torrents, P2P systems, or some form of domain-specific volunteer mesh storage.
The technical side is arguably the simplest problem. The architecture would consist of a server-rendered static frontend that is easy to cache at the edge or on the client, importers for various paywalled sites, and CAPTCHA/OAuth for abuse prevention. Most of this is already implemented.
The importers are currently mocked because I don't have subscriptions to the paywalled sites needed for testing. For storage, I currently have three implementations:
- Local files
- Cloudflare R2
- Telegram
Each storage backend handles file chunking according to its medium's limitations. Telegram, for example, has a 2 GB file size limit.
The Telegram implementation is backed by the users performing imports. Users provide access to both the paywalled site and a dedicated Telegram account. The importer digitally signs imported files and stores them in that Telegram account. The server then serves the persisted data, which can be cached at the edge. Finally, the frontend verifies the digital signature before presenting the file to the user, ensuring the content has not been tampered with.
The Telegram approach comes from the fact that Telegram offers effectively free storage. However, this use case is likely against their Terms of Service. Running everything through a single Telegram account would make it easy for Telegram to shut the project down. By distributing storage across user-provided accounts, storage becomes decentralized and more resistant to large-scale account takedowns, although it does not eliminate the risk of individual accounts being removed for ToS violations, resulting in partial, small outages, but the user network can recover it quickly.
Of course, if someone is willing to volunteer DMCA-resistant storage for the project, that would remove the need for Telegram-backed storage entirely.
I've also seen discussions about other groups attempting to build a successor. I'm interested in joining forces with anyone working toward the same goal.