Project

General

Profile

Algorithms » History » Version 20

Greg Burri, 08/11/2009 09:03 AM

1 1 Greg Burri
h1. Algorithms
2
3 20 Greg Burri
4
h2. Word indexing
5
6
Theses steps are valid independently of the language.
7
8
# Each file is split in word, there is a set of characters which will use as delimiter between the word like [' ', '#', '.', '?',..].
9
# Some character are replaced by a another, for example : ['é', 'ë',..] => 'e'. The goal is to remove all accent.
10
11
12 1 Greg Burri
h2. Searching
13 2 Greg Burri
14 3 Greg Burri
For a functional description see here : [[Functional definition#The-search-window]]
15 15 Greg Burri
16 1 Greg Burri
!aybabtu_search.png!
17 18 Greg Burri
_This schema depicts how the results are sorted from one peer. Each peer result are then merged._
18 4 Greg Burri
19 5 Greg Burri
h2. Peer ID
20
21 7 Greg Burri
Each peer owns a peer id which is unique and generated during the first start. This ID is used to identify a peer, it's better than the previous usage of peer IP, considering this situation :
22 5 Greg Burri
23
* _A_ put in queue a file entry _f_ from _B_, _B_ doesn't know the hashes of this file entry.
24
* _B_ change his IP address.
25
* _A_ want to download _f_, it can ask _B_ for the hashes even _B_'s IP changed.
26
27 8 Greg Burri
h2. Core threads
28
29 9 Greg Burri
There are three kind of threads in the core in addition to the main thread :
30
* [[Protocol_core-core#Downloading-thread|Downloading thread]] : @DownloadManager::ChunkDownloader@
31 8 Greg Burri
* Uploading thread : @UploadManager::Uploader@
32 11 Greg Burri
* [[Algorithms#Updating-the-file-cache|Updating file cache thread]] : @FileManager::FileUpdater@
33 5 Greg Burri
34 19 Greg Burri
h2. Updating the file cache (Obsolete, must be rewritten after the seventh prototype ended)
35 10 Greg Burri
36
Here is the algorithm for the thread (@FileManager::FileUpdater@) which will periodically update the file cache and persist it.
37
38
<pre>
39
D : The set of shared directories
40
T : Time during the hashes are computed (for example 30s)
41
F : A set of file initialy empty
42
43 13 Greg Burri
- Add a watcher to the shared directories
44
45
// First synchronize (at start)
46
For each d in D (recursively) :
47 14 Greg Burri
   - Synchronize physical folders and files with d content
48 13 Greg Burri
   - Add in F the files which doesn't have computed hashes
49
50
Loop :   
51 10 Greg Burri
   t : Time.now
52 1 Greg Burri
   For each f in F :
53 13 Greg Burri
      - Compute the unknown hash of files f
54 1 Greg Burri
      - Remove f from F
55 10 Greg Burri
      If (Time.now - t) > T : break
56
   - Wait T - (Time.now - t)
57 13 Greg Burri
   - Wait for changes for a period of P
58
      - When a modification occurs synchronise the file/folder
59
      - Add each new file in F
60 12 Greg Burri
   - Persist the entire cache in a file (only every ~30min)
61 10 Greg Burri
</pre>
62
63 5 Greg Burri
64 4 Greg Burri
h2. Downloading
65
66
See here : [[Protocol_core-core#Downloading-threads]]