What is torrenting and how does it work?
Torrenting architecture and Choking Algo
Torrenting is a way of distributing large files.
You’d have downloaded some large files or movies through a torrent client like u-torrent or BitTorrent with the help of a .torrent file or a magnet-torrent link. If you had observed, you’d end up spending 20% more data than the size of the file, or sometimes even worse, more than 50%. So why is this happening and how? In this blog, we’re going to explore the BitTorrent protocol and the mechanism of torrenting which is the cause of the previously discussed scenario.
BitTorrent P2P file sharing protocol is a peer-to-peer architecture protocol. Simply put, you can download files from other clients if they have the file you’re looking for. The file is broken into chunks called pieces, which are in turn broken into blocks. When you’re downloading, you’re actually collecting all the pieces that make up your file.
Let’s explore the above terminologies.
Pieces and Blocks: Original file that is stored in the network is split into equal-sized pieces ranging from 256KB to 2MB. SHA1 of each piece is added in the .torrent file. Each piece is in turn split into Blocks. In one transfer, a block is transferred but a piece is served by a peer. Once a piece is downloaded by a peer, it informs the other peers in the swarm so that they can download it from each other.
Peer set: Each peer maintains a list of peers it can send pieces and this is called a peer set. Also known as swarm.
Active peer set: A peer can only send data to a subset of its peer set and this is called an active peer set.
Seeders and Leechers: A peer can be a seeder or a leecher. Leecher: when a peer is downloading. Seeder: when a peer has all the pieces of the content.
Tracker: A server that tracks all the information regarding peers, pieces, and file torrents.
Torrent file and magnet torrent: A torrent file is the session of transfer of single content to a set of peers. Each torrent is independent. This is like a metafile that contains information such as filename, file size, and pieces of information about the file that we’ll be downloading. This file ends with the .torrent extension and can be found on the web. Magnet-torrent is a hyperlink that is used by BitTorrent clients to fetch the details of the hashed file. This link can be generated if you have the hash code of the BitTorrent file.
Say you have got a torrent file and you open that file in the u-torrent application. Downloading starts. What exactly is happening? Now comes the most interesting part.
As soon as you open the torrent file, the first step is tracker announce. Announce or a “Tracker Announce”, is a request sent to a tracker regarding the info about the peers who have the pieces of the file. A request is sent, a connection to the tracker is established, information related to peers having the same file is exchanged, then the connection is closed.
Now that you know who has the pieces, you’ll have to request them to send pieces to you. And in turn, you must send pieces if you have pieces that are requested by others. This is the process of reciprocation.
PIECE SELECTION ALGORITHMS
Since a file is split into a number of pieces and in turn blocks, piece selection strategy is very important to ensure that the swarm has all the required parts of the file. In case a certain piece of the file is missing from the swarm then the download will never finish for all the peers in the swarm which is not desirable. To ensure that this does not happen, the piece selection algorithm is implemented which tries to ensure that all the pieces availability by replicating the pieces as quickly as possible.
Random first piece — The first piece to be downloaded is chosen at random. This way a peer can receive and start quickly uploading.
Rarest First — Sometimes, the original seeder is removed from the swarm due to cost reasons. As the name itself suggests, the next piece to select for downloading is the rarest piece in the swarm. This way, this rarest piece can be spread in the swarm quickly and risks of losing a piece are avoided.
Strict policy — Since pieces are again subdivided into blocks. When a ‘block’ is requested, the remaining blocks of the same piece are requested before any other piece. With this, pieces can be assembled quickly and will be available for other peers.
Endgame mode — When the pieces are requested from peers, sometimes a peer with a low upload speed is selected. This can be amplified during the end of the download as the download reaches completion due to the tendency for the remaining pieces to be downloaded from peers with saturated connections. This is avoided by sending a request to all peers for the piece instead of waiting for a single peer. The overhead for this is negligible as the endgame mode is relatively short, but cannot be used for all pieces as that would cause considerable overhead.
Can there be a peer who’ll only download and never upload? Of course yes. If you decide not to upload to anyone, you can set “maximum upload rate = 0” in the client app. But isn’t this a loss to the network? How to solve this issue?
THE CHOKE ALGORITHM: THE POWER OF BITTORRENT
Since BitTorrent works on P2P network, there is no central resource allocation unit. Then, how to ensure maximum download speed and let no peer abuse the network? The choke algorithm was introduced to guarantee that everybody gets a fair share and reciprocation. This algorithm is a variant of tit-for-tat.
Free riders are peers that download from the network but never upload. They should be penalized. Hence, our criteria to choose a peer to send our content to cannot be simple, it should be based on reciprocation. Choking and Interested: Choking is the refusal to upload. This becomes necessary to reduce TCP congestion and prevent abuse and starvation. Choking and unchoking aren’t perpetual but periodic. An interested peer is someone who wants a piece that you have. At the very beginning, every peer is by default choked by you. You’ll unchoke a peer if that peer has a piece that you need. Any peer will upload to peers, who give the best download rate. This prohibits free riders and encourages peers to let others download. The choke algorithm works differently in different cases as when a peer is — leecher, seeder. The mechanism is huge. If you want to understand the whole mechanism, you can read it here.