Project

General

Profile

Persistent data » History » Version 18

Greg Burri, 04/19/2012 03:32 PM

1 10 Greg Burri
h1. Persistent data
2 1 Greg Burri
3 4 Greg Burri
h2. Files cache
4 1 Greg Burri
5 17 Greg Burri
This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of D-LAN. If some paths of some files doesn't match the shared directory they are discarded.
6 1 Greg Burri
7 9 Greg Burri
This file is saved as a binary file serialized by protocol buffer.
8
9 7 Greg Burri
You can see the proto file here : source:/application/Protos/files_cache.proto
10 1 Greg Burri
11 4 Greg Burri
h3. Size
12
13
Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account.
14
15
<pre>
16
Nb of files : 50'0000
17
Total size : 1 TB
18 18 Greg Burri
Chunk Size : 64 MB
19 4 Greg Burri
Average file per directory : 50
20
Average filename and dirname length : 50 char
21
Size of a hash : 20B
22
</pre>
23
24
<pre>
25
AverageFileSize = 1*1024^4 / 50'000 = 21MB
26 18 Greg Burri
Sha1Size = floor(AverageFileSize / 64MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB
27 4 Greg Burri
FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char)
28
DirnameSize = (50'000 / 50) * 50 * 2 = 97kB
29
30
TotalSize = 977kB + 4.8MB + 97kB = 5.9MB
31
</pre>
32
33 6 Greg Burri
h2. Queued files and directories
34 1 Greg Burri
35 9 Greg Burri
A file containing the queued files and directories is saved in user space. An item from the list may contains a remote peer id and/or the hashes which identify a file. 
36
37
As the file cache file (see above), this file is saved as a binary file serialized by protocol buffer.
38
39
For more details see the file : source:/application/Protos/queue.proto
40 8 Greg Burri
41 4 Greg Burri
42
h2. Configuration file
43
44 13 Greg Burri
h3. Core side
45
46 11 Greg Burri
here are the persisted data :
47
48
* The nick
49 14 Greg Burri
* The peer ID
50 12 Greg Burri
51 13 Greg Burri
As the other data described above these data are saved as a file serialized by _Protocol Buffer_ but in a human readable form.
52 1 Greg Burri
53 13 Greg Burri
For more details see the message _Settings_ in file : source:/application/Protos/common.proto
54 11 Greg Burri
55 13 Greg Burri
h3. Client Side
56 11 Greg Burri
57 13 Greg Burri
here are the persisted data :
58 11 Greg Burri
59 13 Greg Burri
* Core address
60
* Password for remote core