Persistent data » History » Version 4
Greg Burri, 07/21/2009 01:33 PM
| 1 | 1 | Greg Burri | h1. Structures |
|---|---|---|---|
| 2 | |||
| 3 | 4 | Greg Burri | h2. Files cache |
| 4 | 1 | Greg Burri | |
| 5 | 4 | Greg Burri | This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of Aybabtu. |
| 6 | 1 | Greg Burri | |
| 7 | 4 | Greg Burri | You can see the proto file here : source:/protos/files_cache.proto |
| 8 | 1 | Greg Burri | |
| 9 | 4 | Greg Burri | h3. Size |
| 10 | |||
| 11 | Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account. |
||
| 12 | |||
| 13 | <pre> |
||
| 14 | Nb of files : 50'0000 |
||
| 15 | Total size : 1 TB |
||
| 16 | Chunk Size : 32 MB |
||
| 17 | Average file per directory : 50 |
||
| 18 | Average filename and dirname length : 50 char |
||
| 19 | Size of a hash : 20B |
||
| 20 | </pre> |
||
| 21 | |||
| 22 | <pre> |
||
| 23 | AverageFileSize = 1*1024^4 / 50'000 = 21MB |
||
| 24 | Sha1Size = floor(AverageFileSize / 32MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB |
||
| 25 | FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char) |
||
| 26 | DirnameSize = (50'000 / 50) * 50 * 2 = 97kB |
||
| 27 | |||
| 28 | TotalSize = 977kB + 4.8MB + 97kB = 5.9MB |
||
| 29 | </pre> |
||
| 30 | |||
| 31 | |||
| 32 | h2. Configuration file |
||
| 33 | |||
| 34 | TODO.. |