Persistent data » History » Version 18
Greg Burri, 04/19/2012 03:32 PM
| 1 | 10 | Greg Burri | h1. Persistent data |
|---|---|---|---|
| 2 | 1 | Greg Burri | |
| 3 | 4 | Greg Burri | h2. Files cache |
| 4 | 1 | Greg Burri | |
| 5 | 17 | Greg Burri | This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of D-LAN. If some paths of some files doesn't match the shared directory they are discarded. |
| 6 | 1 | Greg Burri | |
| 7 | 9 | Greg Burri | This file is saved as a binary file serialized by protocol buffer. |
| 8 | |||
| 9 | 7 | Greg Burri | You can see the proto file here : source:/application/Protos/files_cache.proto |
| 10 | 1 | Greg Burri | |
| 11 | 4 | Greg Burri | h3. Size |
| 12 | |||
| 13 | Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account. |
||
| 14 | |||
| 15 | <pre> |
||
| 16 | Nb of files : 50'0000 |
||
| 17 | Total size : 1 TB |
||
| 18 | 18 | Greg Burri | Chunk Size : 64 MB |
| 19 | 4 | Greg Burri | Average file per directory : 50 |
| 20 | Average filename and dirname length : 50 char |
||
| 21 | Size of a hash : 20B |
||
| 22 | </pre> |
||
| 23 | |||
| 24 | <pre> |
||
| 25 | AverageFileSize = 1*1024^4 / 50'000 = 21MB |
||
| 26 | 18 | Greg Burri | Sha1Size = floor(AverageFileSize / 64MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB |
| 27 | 4 | Greg Burri | FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char) |
| 28 | DirnameSize = (50'000 / 50) * 50 * 2 = 97kB |
||
| 29 | |||
| 30 | TotalSize = 977kB + 4.8MB + 97kB = 5.9MB |
||
| 31 | </pre> |
||
| 32 | |||
| 33 | 6 | Greg Burri | h2. Queued files and directories |
| 34 | 1 | Greg Burri | |
| 35 | 9 | Greg Burri | A file containing the queued files and directories is saved in user space. An item from the list may contains a remote peer id and/or the hashes which identify a file. |
| 36 | |||
| 37 | As the file cache file (see above), this file is saved as a binary file serialized by protocol buffer. |
||
| 38 | |||
| 39 | For more details see the file : source:/application/Protos/queue.proto |
||
| 40 | 8 | Greg Burri | |
| 41 | 4 | Greg Burri | |
| 42 | h2. Configuration file |
||
| 43 | |||
| 44 | 13 | Greg Burri | h3. Core side |
| 45 | |||
| 46 | 11 | Greg Burri | here are the persisted data : |
| 47 | |||
| 48 | * The nick |
||
| 49 | 14 | Greg Burri | * The peer ID |
| 50 | 12 | Greg Burri | |
| 51 | 13 | Greg Burri | As the other data described above these data are saved as a file serialized by _Protocol Buffer_ but in a human readable form. |
| 52 | 1 | Greg Burri | |
| 53 | 13 | Greg Burri | For more details see the message _Settings_ in file : source:/application/Protos/common.proto |
| 54 | 11 | Greg Burri | |
| 55 | 13 | Greg Burri | h3. Client Side |
| 56 | 11 | Greg Burri | |
| 57 | 13 | Greg Burri | here are the persisted data : |
| 58 | 11 | Greg Burri | |
| 59 | 13 | Greg Burri | * Core address |
| 60 | * Password for remote core |