ngosang/trackerslist with support the tracklisttorrent.json, orbit-db?

[ghost]

Hi everyone.

I'm creating a file format and would like to know what you all think. What do you all think of the tracklistorrrent file format in the project: ngosang/tracklist?

what is tracklistorrrent file?

I'm thinking of creating a tracklisttorrent.json file format - this file format contains a list of metadata to propagate in torrent files.

why write this issue here?

1. One thing that caught my attention was the open source project: 'github/ngosang/tracklist' - the idea of putting together a txt file with a list of torrent trackers. But... one thing I noticed is that you all use txt and not json for trackerslist.
2. So... I'm thinking of creating the tracklisttorrent.json format in the RFC format and publishing a technical documentation on the advantages and disadvantages of this file format.
3. So... I write this topic asking if there is interest from the open project community: ngosang/tracklist, in having the associated tracklisttorrent format as well.
4. So... I believe we can have creative forces to produce something amazing in this open development process. As I said at the beginning, the open project: ngosang/tracklist is a project that caught my attention, in part because I am creating a tracklisttorrent file format that is similar to the ngosang/tracklist project, but it is a file format to be stored or distributed, and not just a software to be used or an open project that gathers a list of trackers in txt.

what relationship is there in the open project github/ngosang/tracklist and the tracklisttorrent format?

1. Initially tracklisttorrent as I said earlier is a json based file format that contains a list of metadata to propagate in torrent files. So the technical documentation about tracklisttorrent should be based on json with some technical differences like the size of the tracklisttorrent file, how the tracklisttorrent file is stored, the target audience that will use the tracklisttorrent file, in addition to whether or not it supports encryption in tracklisttorrent file.
2. The first advantage of tracklisttorrent is that it is a format that, when opened in the browser, becomes a table. So, in the tracklisttorrent technical documentation, I will include or may include some part of the 'github/ngosang/tracklist' code.
3. Both the file format and the 'github/ngosang/tracklist' project - have common goals: to gather a list of trackers for torrent file. The difference between these goals is the way this is done. In the first case I have a specific file format for this and in the second case I use a text file, which is a non-standard format, a general format.
4. The advantage of the first case is that if it's something standardized, you expect everyone to have support for that file format regardless of software or operating system. Of course, in the process of file format standardization, most programs may be interested, whether it is an operating system or general software such as a plug-in, etc., to support this file format, but this is done gradually.

Why create a file format for trackerslistorrent for torrent?

1. My conceptual idea with tracklisttorrent is to have a distributed file format, decentralized from trackers to torrent, the initial advantage is torrent replication in the trackers. It's like you have a list of contacts to which you share your torrent frequently.
2. Another initial advantage is that, as it is a file format, anyone can maintain it independently of software.
3. Another head start is that I am creating this file format for use in the distributed, decentralized database orbit-db, which runs on the ipfs protocol. The advantage of this is that by having a list of trackers you can make this information public for different nodes on the network. The downside is that if there is no node or no data available, the tracklisttorrent file loses its meaning.

What is orbit-db?

OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It's an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and local-first web applications.

question/feedback

What do you all think of the idea?

[ngosang]

Thank you for the ideas. I will try to respond.

Is this project alive?

Yes it's. It has been updated every day for 7 years. It has 37.200 stars in GitHub. The lists are used by bittorrent clients like qBittorrent, several public torrent sites and thousands of users. It's hard to known how many millions of request this repository receives but I had to publish CDN/mirror URLs because a notice from GitHub. You can consider this repositoy the reference for public trackers.

One thing that caught my attention was the open source project: 'github/ngosang/tracklist' - the idea of putting together a txt file with a list of torrent trackers. But... one thing I noticed is that you all use txt and not json for trackerslist.

The use of the txt format was invented by uTorrent client like 20 years ago. It's de-facto standard used by all bittorrent clients and third-party-tools.

So... I'm thinking of creating the tracklisttorrent.json format in the RFC format and publishing a technical documentation on the advantages and disadvantages of this file format.

I appreciate your energy, but defining an RFC is not enough. Bittorrent clients and third-party scripts need to implement it. The bittorrent ecosystem is very old and things move slowly. It takes me 5 minutes to read the RFC and 10 minutes to change the code to upload JSON files. The thing is, if the bittorrent clients don't use that format, we are just creating fragmentation in the ecosystem and a few lines of code that I have to maintain.

Both the file format and the 'github/ngosang/tracklist' project - have common goals: to gather a list of trackers for torrent file. The difference between these goals is the way this is done. In the first case I have a specific file format for this and in the second case I use a text file, which is a non-standard format, a general format.

The goal of this project is to keep alive the bittorrent ecosystem providing a reliable list of public trackers. Trackers come and go but you will always find the best ones on this list.

I don't agree with the last phrase. The "tracker list format" is really well defined. It's a text format with one tracker per line. Lines are separated with 2 "new lines" in Windows format \r\n\r\n. All bittorrent clients use this format. Is so simple that you don't need a RFC nor a special software to open it. This format is readable by humans and machines. Maybe you should document this format in a RFC, I will link to it.

Another head start is that I am creating this file format for use in the distributed, decentralized database orbit-db, which runs on the ipfs protocol.

You can make a script to publish the files in this project to IPFS or in OrbitDB. The files in this project are public domain and they can be used in any way, even without quoting me. I don't know much about those technologies, but, if you think it can be useful I can take a look and create a IPFS mirror for the files.

[ghost]

Hi ngosang.

I appreciate your energy, but defining an RFC is not enough. Bittorrent clients and third-party scripts need to implement it. The bittorrent ecosystem is very old and things move slowly. It takes me 5 minutes to read the RFC and 10 minutes to change the code to upload JSON files. The thing is, if the bittorrent clients don't use that format, we are just creating fragmentation in the ecosystem and a few lines of code that I have to maintain.

1. So... OrbitDB works with json, I thought of creating a json file format that is compatible with orbit-db and at the same time tracklist torrent. That's a good reason I forgot to mention earlier when creating this issue.
2. My goal was to support tracklistorrent for any solution that uses torrent, or most of the solutions that are open and have a target audience in the repository. But... I forgot to mention that second reason for the file format... but... but your feedback was interesting and maybe it could be wrong.
3. So... for me the big problem of creating the RFC is the implementation, not even the RFC documentation has a good implementation or is complete. For example, there is an rfc about email formatting being done in markdown, but most apps or email clients still don't use this recent format.
4. So when you say that it's not enough to create an RFC, but there must be some support and implementation later or earlier than the RFC, I think your feedback is valid, in view of the case I mentioned recently of changing the email format that was not implemented in every application or email client.
5. But just to give some general feedback, the RFC itself is not meant to be a 100% correct implementation, it's just a technical document that changes over the years by certain technology etc.
6. So... I also think what you said about fragmentation is valid, when a new format appears, the first thing that happens is fragmentation, there will be a bittorrent client with the "new format" and a bittorrent client with the "old format".
7. I forgot to mention these 6 points in the issue about the advantages and disadvantages of this file format. So overall, thanks for your concise feedback.
8. Note: But... note... a general way to solve this is to create a webapp that converts tracklisttorrent to tracklist.txt, or tracklist.txt to tracklisttorrent. This can be done with an orbit-db database and a website stored in ipfs. You would need a plugin or feature just to read this file format in any torrent client. Or even there would be no need for that, considering that both files like: tracklist.txt and tracklisttorrent are corresponding, something like typescript and javascript.

The goal of this project is to keep alive the bittorrent ecosystem providing a reliable list of public trackers. Trackers come and go but you will always find the best ones on this list.
I don't agree with the last phrase. The "tracker list format" is really well defined. It's a text format with one tracker per line. Lines are separated with 2 "new lines" in Windows format \r\n\r\n. All bittorrent clients use this format. Is so simple that you don't need a RFC nor a special software to open it. This format is readable by humans and machines. Maybe you should document this format in a RFC, I will link to it.

1. I was trying to make the 'sales pitch' process, the idea of convincing, of saying good things in a file format. But...
2. From a technical point of view from what I have practical and/or job knowledge over the years, in certain scenarios, in terms of file performance, a binary file format is better than a text file format.
3. the reason for this is that a binary format is read in block and not sequentially as it happens when reading text files by line.
4. The downside is that the binary format is machine readable and the text format is human readable.

Anyway, everything you said is right, I just tried to say a few things to complement your answer.

You can make a script to publish the files in this project to IPFS or in OrbitDB. The files in this project are public domain and they can be used in any way, even without quoting me. I don't know much about those technologies, but, if you think it can be useful I can take a look and create a IPFS mirror for the files.

thank you for feedback... so... this idea is great. Note: IPFS stores the information on multiple servers, or nodes, all around the world. This distribution increases stability by creating numerous points of access that can act as backups no matter the state of a single node.

[ngosang]

#417

[ghost]

Hi ngosang.

#417

thank you for feedback. I will close this issue, you clarified my doubt.

Have a great day, and thank you for your patience in answering my question.