General approach » History » Version 16

Greg Burri, 05/15/2012 02:59 AM

1 1 Greg Burri
h1. General approach
3 5 Greg Burri
h2. Generalities 
5 8 Greg Burri
Each file the user want to share is cut sequentially in logical pieces called _chunk_. Each chunk has the same size except the last one which can be smaller.
6 1 Greg Burri
7 8 Greg Burri
For each chunk there is a footprint called _hash_ which identify the chunk across the network. We admit that two chunks with the same hash contain the same data. The hash is computed with a cryptographic hash function like "SHA-1": .
8 2 Greg Burri
9 1 Greg Burri
h2. To know who are the other peers
11 5 Greg Burri
12 6 Greg Burri
Each peer sends periodically a message to tell to the other peers that it exists. This message can contain different information like the peer name, the amount of sharing data and also the wanted chunk hashes, see _Download_ section below. Multicast UDP is used to broadcasting the message to all other peers simultaneously.
13 5 Greg Burri
h2. Browsing 
Remote entries can be browsed by asking the content of a given remote directory. A listed file can contain one or more of its hashes.
There is a special case which the peer doesn't know about remote entries and wants to retrieve the root directories.
h2. Search for entries (by terms)
A multicast UDP datagram is sent to all other peer with the searched terms. The remote peers which have one or more matched entry will send them back with a unicast UDP datagram.
A file result can contain on or more of its hashes.
h2. Download
31 14 Greg Burri
A chunk can only be downloaded if we know its hash. The hashes associated to a specific file can be explicitly asked to the peer owner of the file. If the remote peer doesn't know the hashes of one of its file when they are asked, it will compute these hashes on the fly and send them one by one as soon as possible.
32 5 Greg Burri
33 3 Greg Burri
The list of the files to download is ordered by the user, thus the top files must be downloaded first. We will try to know first the hashes of the top files. As soon as we know a hash we can try to download the associated chunk. There can be many parallel downloads but only from different remote peers. The number of parallel downloads is limited.
34 1 Greg Burri
35 15 Greg Burri
To choose a chunk to download we will take the first file having at least one un-downloaded chunk which has at least one free peer, then among theses chunks we will choose the one which has the minimum number of peer (rarest part first), if there is several eligible chunks we choose one randomly.
36 4 Greg Burri
37 6 Greg Burri
To known which remote peer has which wanted chunk we asked periodically to all remote peers if they have the next chunks we want to download by sending them the corresponding hashes. The number of hash sent is limited. This can be achieved by putting theses hashes in the message described in section _To know who are the other peers_.
38 1 Greg Burri
39 5 Greg Burri
To initiate a download we ask a remote peer by sending a hash and an offset relative to the beginning of the chunk.
40 1 Greg Burri
41 5 Greg Burri
h2. Communicate by chatting
42 1 Greg Burri
43 16 Greg Burri
A multicast UDP datagram containing an arbitrary message can be sent to all other peers.
44 7 Greg Burri
45 9 Greg Burri
h2. Controlling the Core
The core is controlled via a TCP socket, there is three kinds of message:
49 10 Greg Burri
* The *state message* is sent periodically by the core to the controller it provides the whole state of the core including the known peers, the current downloads, uploads and some settings. There is no dependency between two states.
50 9 Greg Burri
* The *event message* is a message sent by the core to the controller for some event like a new chat message or a new log message.
51 12 Greg Burri
* The *command message* is a message sent by the controller to the core. An answer may be provided by the core. Commands may be used to add a new download, to move a download in the queue, to search some entries among the other peers, to send a chat message, etc. For some commands a tag may be provide by the core to identify some deferred answers, for example there can be many answers to a search command and they can be send several seconds after the initial command.
52 9 Greg Burri
A non-local connection must first, right after the connection is established, send a command to authenticate the controller with a hashed password. If the password is wrong the connection is closed by the core.
56 5 Greg Burri
h2. Glossary
57 3 Greg Burri
58 7 Greg Burri
Remote _<thing>_: <thing> is associated to another peer.
An entry: Can be a file or a directory.
A peer: A computer on the network running D-LAN.
A free peer: A peer from which we don't have any active downloads.
A chunk: A part of a file.
A hash: A identifier for a chunk.