Persistent data » History » Version 18
Greg Burri, 04/19/2012 03:32 PM
1 | 10 | Greg Burri | h1. Persistent data |
---|---|---|---|
2 | 1 | Greg Burri | |
3 | 4 | Greg Burri | h2. Files cache |
4 | 1 | Greg Burri | |
5 | 17 | Greg Burri | This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of D-LAN. If some paths of some files doesn't match the shared directory they are discarded. |
6 | 1 | Greg Burri | |
7 | 9 | Greg Burri | This file is saved as a binary file serialized by protocol buffer. |
8 | |||
9 | 7 | Greg Burri | You can see the proto file here : source:/application/Protos/files_cache.proto |
10 | 1 | Greg Burri | |
11 | 4 | Greg Burri | h3. Size |
12 | |||
13 | Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account. |
||
14 | |||
15 | <pre> |
||
16 | Nb of files : 50'0000 |
||
17 | Total size : 1 TB |
||
18 | 18 | Greg Burri | Chunk Size : 64 MB |
19 | 4 | Greg Burri | Average file per directory : 50 |
20 | Average filename and dirname length : 50 char |
||
21 | Size of a hash : 20B |
||
22 | </pre> |
||
23 | |||
24 | <pre> |
||
25 | AverageFileSize = 1*1024^4 / 50'000 = 21MB |
||
26 | 18 | Greg Burri | Sha1Size = floor(AverageFileSize / 64MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB |
27 | 4 | Greg Burri | FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char) |
28 | DirnameSize = (50'000 / 50) * 50 * 2 = 97kB |
||
29 | |||
30 | TotalSize = 977kB + 4.8MB + 97kB = 5.9MB |
||
31 | </pre> |
||
32 | |||
33 | 6 | Greg Burri | h2. Queued files and directories |
34 | 1 | Greg Burri | |
35 | 9 | Greg Burri | A file containing the queued files and directories is saved in user space. An item from the list may contains a remote peer id and/or the hashes which identify a file. |
36 | |||
37 | As the file cache file (see above), this file is saved as a binary file serialized by protocol buffer. |
||
38 | |||
39 | For more details see the file : source:/application/Protos/queue.proto |
||
40 | 8 | Greg Burri | |
41 | 4 | Greg Burri | |
42 | h2. Configuration file |
||
43 | |||
44 | 13 | Greg Burri | h3. Core side |
45 | |||
46 | 11 | Greg Burri | here are the persisted data : |
47 | |||
48 | * The nick |
||
49 | 14 | Greg Burri | * The peer ID |
50 | 12 | Greg Burri | |
51 | 13 | Greg Burri | As the other data described above these data are saved as a file serialized by _Protocol Buffer_ but in a human readable form. |
52 | 1 | Greg Burri | |
53 | 13 | Greg Burri | For more details see the message _Settings_ in file : source:/application/Protos/common.proto |
54 | 11 | Greg Burri | |
55 | 13 | Greg Burri | h3. Client Side |
56 | 11 | Greg Burri | |
57 | 13 | Greg Burri | here are the persisted data : |
58 | 11 | Greg Burri | |
59 | 13 | Greg Burri | * Core address |
60 | * Password for remote core |