Project

General

Profile

Persistent data » History » Version 8

Greg Burri, 09/02/2009 09:21 PM

1 1 Greg Burri
h1. Structures
2
3 4 Greg Burri
h2. Files cache
4 1 Greg Burri
5 5 Greg Burri
This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of Aybabtu. If some paths of some files doesn't match the shared directory they are discarded.
6 1 Greg Burri
7 7 Greg Burri
You can see the proto file here : source:/application/Protos/files_cache.proto
8 1 Greg Burri
9 4 Greg Burri
h3. Size
10
11
Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account.
12
13
<pre>
14
Nb of files : 50'0000
15
Total size : 1 TB
16
Chunk Size : 32 MB
17
Average file per directory : 50
18
Average filename and dirname length : 50 char
19
Size of a hash : 20B
20
</pre>
21
22
<pre>
23
AverageFileSize = 1*1024^4 / 50'000 = 21MB
24
Sha1Size = floor(AverageFileSize / 32MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB
25
FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char)
26
DirnameSize = (50'000 / 50) * 50 * 2 = 97kB
27
28
TotalSize = 977kB + 4.8MB + 97kB = 5.9MB
29
</pre>
30
31 6 Greg Burri
h2. Queued files and directories
32
33 8 Greg Burri
A file containing the queued files and directories is saved in user space. An item from the list may contains a remote peer id and/or the hashes. For more details see the file : source:/application/Protos/queue.proto
34
35 4 Greg Burri
36
h2. Configuration file
37
38
TODO..