Persistent data » History » Version 17
  Greg Burri, 03/17/2011 12:16 AM 
  
| 1 | 10 | Greg Burri | h1. Persistent data  | 
|---|---|---|---|
| 2 | 1 | Greg Burri | |
| 3 | 4 | Greg Burri | h2. Files cache  | 
| 4 | 1 | Greg Burri | |
| 5 | 17 | Greg Burri | This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of D-LAN. If some paths of some files doesn't match the shared directory they are discarded.  | 
| 6 | 1 | Greg Burri | |
| 7 | 9 | Greg Burri | This file is saved as a binary file serialized by protocol buffer.  | 
| 8 | |||
| 9 | 7 | Greg Burri | You can see the proto file here : source:/application/Protos/files_cache.proto  | 
| 10 | 1 | Greg Burri | |
| 11 | 4 | Greg Burri | h3. Size  | 
| 12 | |||
| 13 | Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account.  | 
||
| 14 | |||
| 15 | <pre>  | 
||
| 16 | Nb of files : 50'0000  | 
||
| 17 | Total size : 1 TB  | 
||
| 18 | Chunk Size : 32 MB  | 
||
| 19 | Average file per directory : 50  | 
||
| 20 | Average filename and dirname length : 50 char  | 
||
| 21 | Size of a hash : 20B  | 
||
| 22 | </pre>  | 
||
| 23 | |||
| 24 | <pre>  | 
||
| 25 | AverageFileSize = 1*1024^4 / 50'000 = 21MB  | 
||
| 26 | Sha1Size = floor(AverageFileSize / 32MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB  | 
||
| 27 | FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char)  | 
||
| 28 | DirnameSize = (50'000 / 50) * 50 * 2 = 97kB  | 
||
| 29 | |||
| 30 | TotalSize = 977kB + 4.8MB + 97kB = 5.9MB  | 
||
| 31 | </pre>  | 
||
| 32 | |||
| 33 | 6 | Greg Burri | h2. Queued files and directories  | 
| 34 | 1 | Greg Burri | |
| 35 | 9 | Greg Burri | A file containing the queued files and directories is saved in user space. An item from the list may contains a remote peer id and/or the hashes which identify a file.  | 
| 36 | |||
| 37 | As the file cache file (see above), this file is saved as a binary file serialized by protocol buffer.  | 
||
| 38 | |||
| 39 | For more details see the file : source:/application/Protos/queue.proto  | 
||
| 40 | 8 | Greg Burri | |
| 41 | 4 | Greg Burri | |
| 42 | h2. Configuration file  | 
||
| 43 | |||
| 44 | 13 | Greg Burri | h3. Core side  | 
| 45 | |||
| 46 | 11 | Greg Burri | here are the persisted data :  | 
| 47 | |||
| 48 | * The nick  | 
||
| 49 | 14 | Greg Burri | * The peer ID  | 
| 50 | 12 | Greg Burri | |
| 51 | 13 | Greg Burri | As the other data described above these data are saved as a file serialized by _Protocol Buffer_ but in a human readable form.  | 
| 52 | 1 | Greg Burri | |
| 53 | 13 | Greg Burri | For more details see the message _Settings_ in file : source:/application/Protos/common.proto  | 
| 54 | 11 | Greg Burri | |
| 55 | 13 | Greg Burri | h3. Client Side  | 
| 56 | 11 | Greg Burri | |
| 57 | 13 | Greg Burri | here are the persisted data :  | 
| 58 | 11 | Greg Burri | |
| 59 | 13 | Greg Burri | * Core address  | 
| 60 | * Password for remote core  |