Project

General

Profile

Persistent data » History » Version 4

Greg Burri, 07/21/2009 01:33 PM

1 1 Greg Burri
h1. Structures
2
3 4 Greg Burri
h2. Files cache
4 1 Greg Burri
5 4 Greg Burri
This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of Aybabtu.
6 1 Greg Burri
7 4 Greg Burri
You can see the proto file here : source:/protos/files_cache.proto
8 1 Greg Burri
9 4 Greg Burri
h3. Size
10
11
Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account.
12
13
<pre>
14
Nb of files : 50'0000
15
Total size : 1 TB
16
Chunk Size : 32 MB
17
Average file per directory : 50
18
Average filename and dirname length : 50 char
19
Size of a hash : 20B
20
</pre>
21
22
<pre>
23
AverageFileSize = 1*1024^4 / 50'000 = 21MB
24
Sha1Size = floor(AverageFileSize / 32MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB
25
FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char)
26
DirnameSize = (50'000 / 50) * 50 * 2 = 97kB
27
28
TotalSize = 977kB + 4.8MB + 97kB = 5.9MB
29
</pre>
30
31
32
h2. Configuration file
33
34
TODO..