Bittorrent From 100 Hours to 1 000 000 Torrents

2024-03-29
KBS
Web DevelopmentLinuxNetworkingCloudAnalysis
Bittorrent From 100 Hours to 1 000 000 Torrents

Without sounding unduly severe, it is reasonable to assume that if you have never encountered the ONE MILLION TORRENTS

After approximately 100 hours of continuous crawling, the database surpassed one million unique torrents. The SQLite file occupies just over 4 GiB:

CLI output showing database size
Figure 2 – Command-line confirmation of database size.

Additional statistics are illustrated below:

Further CLI data
Figure 3 – Supplementary dataset statistics.


NOTEWORTHY FINDINGS

Roughly 80 % of torrents circulating in the DHT are pornographic—an observation that likely speaks to human proclivities more than technological constraints. Yet, with targeted queries, one can uncover genuinely valuable material. By searching for uncommon compression formats such as .zst, I routinely find multi-terabyte archives of rare content; discovery has become something of a hobby.

A particularly notable trove is available here: https://rabbit-hole.ca/db.sqlite.csv. This dataset appears to be an internal index from a major public torrent site, leaked less than a year ago. After basic sanitization (all adult content removed), the file enumerates nearly three million unique torrents—primarily television and film.

CSV preview screenshot
Figure 4 – Sample rows from the leaked CSV (headers trimmed for clarity).

While publishing such a catalog exceeds typical notions of “public” sharing, the information is already circulating widely; my contribution is merely to present it in a cleaner, more accessible format.

“Oh well…” meme image

Draw your own conclusions.