Strigi

Jos van den Oever

daemon and (optional) qt4 gui: strigi version 0.7.8 (2013-02-05) (811 kb) ChangeLog

previous version of daemon and (optional) qt4 gui: strigi version 0.5.11 (2008-07-20) (857 kb)

additional kde3 gui: strigiapplet version 0.5.7 (2007-10-26) (32 kb)

KDE-Apps entry

Wiki page

Akademy 2006 presentation on Strigi (pdf)

Introduction

For my personal use I've written the jstreams classes that allow one to easily read nested files. These have proven very fast and have been included in the clucene c++ search engine. These classes would also be a cool extension to the kio plugins for allowing the user to browse e.g. files in a zip file that is stored in an email attachment. Another use would be to write a crawler that can gather information from all files in the filesystem even if they are hidden in emails or archives. I intended to add this feature to Kat, but because of the slowdown in the Kat project the latest Kat development version is not complete and does not build.

So I developed a small daemon that can index information using the new crawler. Now i've reached a point that the crawler is very stable and fast. How fast exactly depends on you system. It comes complete with a simple gui to controll the daemon and to search. I've named the program Strigi, which is short for Strigiformes.

Here are the main features of Strigi:

• very fast crawling
• very small memory footprint
• no hammering of the system
• pluggable backend, currently clucene and hyperestraier, sqlite3 and xapian are in the works
• communication between daemon and search program over an abstract interface, this is currently a simple socket but implementation of dbus is a possibility. There's a small perl program in the code as an example of how to query. This is so easy that any KDE app could implement this.
• simple interface for implementing plugins for extracting information. we'll try to reuse the kat plugins, although native plugins will have a large speed advantage
• calculation of sha1 for every file crawled (allows fast finding of duplicates)

Installation

To install the strigi daemon you need to install one of the four possible backends. Currently, the CLucene backend is fastest. You need at least version 0.9.15, which is available here. In addition you need libmagic on your system.

To install strigi, just download the tarball and do the cmake dance.

Bug reporting

Should you encounter any bugs in Strigi, please report them here