Monday, September 22, 2008

Choosing a Server

BitTorrent trackers are strange animals. Servers are generally pretty well-rounded in terms of system resources (hard drive is proportionate to memory/processor/bandwidth), but trackers don't use each resource equally.

Trackers use very little bandwidth and disk space, but are comparatively heavy on processor and memory. The announce/response and scrape/response cycles contain very little data and are only infrequently initiated with any given peer. By bandwidth alone, it would be theoretically possible to track tens of thousands of simultaneous peers on a residential internet line. Obviously, a consumer ISP would be unable up to handle that volume of connections, but that's a pretty compelling illustration of the bandwidth situation. Similarly, since even a complete BitTorrent site (index + tracker) only needs to serve small metainfo (.torrent) files, the site and database combined for a reasonably-sized tracker should be no more than a few gigabytes. However, the sheer amount of traffic involved in running a tracker makes for a fair bit of work for the memory and processor.

To estimate the traffic on any tracker, remember that each peer will connect to the announce roughly once every 30 minutes, or however long your tracker's announce interval is. The protocol specifies that clients should connect more often when sending event=started/completed/stopped, so you'll have to round up a bit to compensate. Basically, just divide the number/estimated number of peers by the announce interval in seconds (30 minutes * 60 = 1800 seconds). This will give you a rough idea of the number of hits per second on your announce. On TorrentFries, the announce accounts for roughly 50% of the site traffic, 30% for scrape, and 20% for all other activity. Simply double the result to get a rough idea of the total number of hits on the web server.

hits per second = (peers / announce interval) * 2

This doesn't tell you a whole lot in terms of servers, though. However, since site growth is generally pretty gradual, there's no need to worry about what you'll eventually be running on. Just start basic and upgrade as the need strikes.

Our first server had an AMD Athlon 64 3500+ processor with 512 MiB of memory, a 1500 GiB bandwidth cap on a 100 Mbps line, and a 160 GB SATA drive. It was pretty rudimentary, but worked fine to about 8,000 peers (10 hits/sec). Right now we're tracking 25,000 peers on a dual Xeon machine with 4 GiB of memory and two hard drives in a RAID 1 array, and we're looking at about 10% utilization. It's got a bit of room to grow.

If you're curious to see what makes bigger sites tick, take a look at The Pirate Bay's setup.

4 comments:

Sean said...

Any suggestions on hosts? It seems anyone that used to tolerate torrent sites has been shut down.. Also anyway of contacting you other than these comments?

CurlyFries said...

We've avoided discussing hosts in much detail because we don't want to endorse any one host, especially if they end up being more hostile than our assessment would seem to indicate.

No, at the moment, the only way to contact us is by commenting. I suppose we could set up a Gmail address, but we're not really in the business of giving out personal advice.

Bryant said...

I know you don't want to endorse any particular host, but, for informational purposes, can you tell us which host you started on? If not, could you give us some general information on choosing one (like country, location, maybe general pricing info)? I just want to get in the habit of being able to choose good hosts but I'm not too sure how to do so.

CurlyFries said...

http://torrentfries.blogspot.com/search/label/hosting

;)

Clicky Web Analytics