Project

General

Profile

Persistent data » History » Version 9

Greg Burri, 09/02/2009 09:23 PM

1 1 Greg Burri
h1. Structures
2
3 4 Greg Burri
h2. Files cache
4 1 Greg Burri
5 5 Greg Burri
This file contains all the shared files and theirs SHA-1 hashes. This file is saved in user space after an update of the file list, it is completely loaded at the start of Aybabtu. If some paths of some files doesn't match the shared directory they are discarded.
6 1 Greg Burri
7 9 Greg Burri
This file is saved as a binary file serialized by protocol buffer.
8
9 7 Greg Burri
You can see the proto file here : source:/application/Protos/files_cache.proto
10 1 Greg Burri
11 4 Greg Burri
h3. Size
12
13
Here is a size approximation for 50'000 files stored in 1 TB. The overhead of the file format is not taken in account.
14
15
<pre>
16
Nb of files : 50'0000
17
Total size : 1 TB
18
Chunk Size : 32 MB
19
Average file per directory : 50
20
Average filename and dirname length : 50 char
21
Size of a hash : 20B
22
</pre>
23
24
<pre>
25
AverageFileSize = 1*1024^4 / 50'000 = 21MB
26
Sha1Size = floor(AverageFileSize / 32MB) * 20 * 50'000 = 1 * 20 * 50'000 = 977kB
27
FilenameSize = 50'000 * 50 * 2 = 4.8MB (2 bytes per char)
28
DirnameSize = (50'000 / 50) * 50 * 2 = 97kB
29
30
TotalSize = 977kB + 4.8MB + 97kB = 5.9MB
31
</pre>
32
33 6 Greg Burri
h2. Queued files and directories
34 1 Greg Burri
35 9 Greg Burri
A file containing the queued files and directories is saved in user space. An item from the list may contains a remote peer id and/or the hashes which identify a file. 
36
37
As the file cache file (see above), this file is saved as a binary file serialized by protocol buffer.
38
39
For more details see the file : source:/application/Protos/queue.proto
40 8 Greg Burri
41 4 Greg Burri
42
h2. Configuration file
43
44
TODO..