Search prototype¶

directory : source:/prototypes/03_Search

Goals¶

• Search with glob pattern.
• Measure the time for searching among 50'000 files. There is always two measures to test the caching capacity of the OS.
  • A solution is to generate randomly empty files and folders and create an iso with it which will be stored in the repo.
• If time is above 1 minute for the first time and above 3 seconds for the seconds then an another solution must be found like an indexing system.

Setup¶

The file source:/prototypes/03_Search/files.iso.bzip2 contains about 50'000 files and can be mounted with the following commands.

$> bunzip2 files.iso.bzip2
$> mkdir files
$> sudo mount -o loop files.iso files

It will bunzip the file, create a directory files and mount the iso file.

Reference times¶

Here is some reference times given by the find command.

<flynux:~/code/projects/aybabtu/prototypes/03_Search $> sudo mount -o loop files.iso files/
[sudo] password for gburri: 

<flynux:~/code/projects/aybabtu/prototypes/03_Search $> time find files/ -iname "*1234*" 
tree2/tree/055cba22/005f1168/20fe14e1/221234aa
tree2/tree/055cba22/0bc10022/de8ccd6d/31234116
tree2/tree/055cba22/78da82a9/c634ba77/881234a0
tree2/tree/055cba22/b7a538f2/caaf9076/212349d5

real    0m1.932s
user    0m0.140s
sys     0m1.716s

<flynux:~/code/projects/aybabtu/prototypes/03_Search $> time find files/ -iname "*1234*" 
tree2/tree/055cba22/005f1168/20fe14e1/221234aa
tree2/tree/055cba22/0bc10022/de8ccd6d/31234116
tree2/tree/055cba22/78da82a9/c634ba77/881234a0
tree2/tree/055cba22/b7a538f2/caaf9076/212349d5

real    0m0.382s
user    0m0.200s
sys     0m0.176s

Times from the prototype¶

Measured in release mode.

source:prototypes/03_Search@36daf15fa072500575ff3f252c55b500e0972f26

<flynux:~/code/projects/aybabtu/prototypes/03_Search $> time ./03_Search search tree "*1234*" 
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/b301a5ef/dd21a1d1/0cc1234f
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/394173f8/5b947a9a/db4fce33/51234260
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/b301a5ef/8e6e6b98/d1deca60/123445ed
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/b301a5ef/9e439990/34a960b1/db261234

real    0m30.329s
user    0m2.460s
sys     0m1.804s

<flynux:~/code/projects/aybabtu/prototypes/03_Search $> time ./03_Search search tree "*1234*" 
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/b301a5ef/dd21a1d1/0cc1234f
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/394173f8/5b947a9a/db4fce33/51234260
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/b301a5ef/8e6e6b98/d1deca60/123445ed
/home/gburri/code/projects/aybabtu/prototypes/03_Search/tree/b301a5ef/9e439990/34a960b1/db261234

real    0m1.912s
user    0m1.060s
sys     0m0.756s

So .. theses times are very bad...