Question: Upper limit to number of tracked files?

[wezell]

I've been playing with the ttorrent code and as wondering the applicability to serve/share large numbers of files. Specifically, the tracker/seeder code - is there an upper limit to the number of files that it could both seed and track? I haven't dug into the code but was it designed with the intent to track large amounts of files, by which I am talking 1m files or more? Or was it not designed with targets like that in mind?

[Dead-off]

memory/perfomance usage on tracker side must be low. So i think tracker can track 1m files.

about seeder-side:
In 1.5 version (v1,6 branch in repo) seeding many files is not possible. There was many problems, for example:

1. seeding one torrent holds .torrent file in memory. So it can spend about 20kb memory for each torrent always (even if nobody downloads the torrent)
2. library binds separate server socket for each torrent (java.io.IOException: No available port for the BitTorrent client! #132)
3. Also library creates instance of java.lang.Thread for each torrent

So in 2.0 version many of these problems were fixed. Torrents will be lazy-inited. So seeding of one file takes about 400 bytes (if nobody currently downloads the torrent). Also all incoming connections are listened on one port. In case when many peers download from the seeder different files i think bottleneck is networking.
Finally we didn't test 1m files but our internal server seeds 10k files without performance problems. Also same server tracks these 10k torrents

[Dead-off]

if you don't have more questions you can close issue:)

[wezell]

Thanks for the quick and complete reply! Looks like there are a lot of nice improvements coming in 2.0.