PHP-based indexing and search implementation

[kristo5747]

Is there such thing?

 

I designed a while back a rudimentary form based app for my users.

 

We receive from our suppliers hardware manufacturing data in XML files: file name is made of eleven fields separated by tildes, with each field having its own meaning.

 

R&D guys wanted to be able to search each field of the file names so I used regex() with decent results.

 

Problem is that we have now in the upwards of 2.5 million files. And my app can't hack it anymore.

 

I looked at Apache Lucene & Solr. Though it seemed like the best solution to my problem, the fields in the filenames are not peers to the file content. Big no-no with Solr.

 

What is the best way to implement a PHP app with indexing and search capability with such large number of files?

 

Do I have to buy Zend and use Zend_Search? Is it the only way?

 

Thanks for your input.

[lastkarrde]

At the very least you will want to put that data into a database (MySQL, MongoDB, Postgres), searching through the file is horribly inefficient. You could then attempt to use the database's built in full text search.

 

Best bet for performance and accuracy is to use something 3rd party like Lucene.

[QuickOldCar]

I second using a database.

 

Then trying one of the following methods

 

http://sphinxsearch.com/

http://lucene.apache.org/

http://www.zackgrossbart.com/hackito/search-engine-python/

 

Or even use mysql full text indexing

http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html

 

 

[kristo5747]

mysql full text indexing seems the way to go, alongside with Lucene.

 

Thanks for your input.

[ignace]

Do I have to buy Zend and use Zend_Search?

 

Zend_Search (as in Zend framework?) is free to use as is the entire framework.