Persistent data » History » Version 5
Greg Burri, 07/30/2009 04:29 PM
| 1 | 1 | Greg Burri | h1. Structures |
|---|---|---|---|
| 2 | |||
| 3 | 4 | Greg Burri | h2. Files cache |
| 4 | 1 | Greg Burri | |
| 5 | 5 | Greg Burri | This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of Aybabtu. If some paths of some files doesn't match the shared directory they are discarded. |
| 6 | 1 | Greg Burri | |
| 7 | 4 | Greg Burri | You can see the proto file here : source:/protos/files_cache.proto |
| 8 | 1 | Greg Burri | |
| 9 | 4 | Greg Burri | h3. Size |
| 10 | |||
| 11 | Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account. |
||
| 12 | |||
| 13 | <pre> |
||
| 14 | Nb of files : 50'0000 |
||
| 15 | Total size : 1 TB |
||
| 16 | Chunk Size : 32 MB |
||
| 17 | Average file per directory : 50 |
||
| 18 | Average filename and dirname length : 50 char |
||
| 19 | Size of a hash : 20B |
||
| 20 | </pre> |
||
| 21 | |||
| 22 | <pre> |
||
| 23 | AverageFileSize = 1*1024^4 / 50'000 = 21MB |
||
| 24 | Sha1Size = floor(AverageFileSize / 32MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB |
||
| 25 | FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char) |
||
| 26 | DirnameSize = (50'000 / 50) * 50 * 2 = 97kB |
||
| 27 | |||
| 28 | TotalSize = 977kB + 4.8MB + 97kB = 5.9MB |
||
| 29 | </pre> |
||
| 30 | |||
| 31 | |||
| 32 | h2. Configuration file |
||
| 33 | |||
| 34 | TODO.. |