The MUFIN Project

Objective

In general, searching can be described as an activity of looking thoroughly in order to find something or someone, an investigation seeking answers, an operation that determines whether one or more of a set of items has a specified property. The goal of the project Multi-Feature Indexing Network (MUFIN) is to develop a general purpose technology solution to the problem of searching in various and very large databases.

Vision

It is generally agreed that the search problem is a triple of: (1) data and queries, (2) indexing structure, and (3) computing infrastructure on which the search is executed. We believe that future search systems must be born on the divergence of scale and determinism. In particular, a more and more desirable property of any search system is its ability to either handle growing amounts of work in a graceful manner, or to be readily enlarged. On the other hand, the necessity of search processes to be determined by an unbroken chain of pre-defined steps is in search processes becoming less and less important. The effects of the divergence of scale and determinism on the development of search structures are illustrated in the following figure.

Scalability

• exponential growth in data volume
• number of users (queries) increasing fast
• variety of data types, emergence of various digital databases and libraries
• multi-lingual (feature, modal) queries

Determinism

• from exact match to similarity
• from precise query evaluation to approximate evaluation
• from unvaried answer to satisfactory answer
• from fixed queries to personalized queries
• from dedicated hardware to dynamic hardware mapping

Building pillars

MUFIN represents a joint research effort towards a scalable and extensible similarity search system for many applications. Its extensibility is achieved by accepting the metric space model of similarity, so the technology works for any metric distance measure and finds applications as diverse as biology, geography, multimedia, data cleaning and integration, etc. In order to scale into billions of object searched on-line for hundreds of queries processed real-time, structured peer-to-peer (P2P) similarity search networks are applied. In order to tune performance, MUFIN keeps a clear separation between the logical P2P structure and the hardware physical infrastructure.