我正在建设一个Django网站,我正在寻找一个搜索引擎。
一些候选人:
Lucene/Lucene with Compass/Solr
斯芬克斯
Postgresql内置全文搜索
MySQl内置全文搜索
选择标准:
结果相关性和排名
搜索和索引速度
易于使用,易于与Django集成
资源需求——站点将托管在VPS上,所以理想情况下搜索引擎不需要大量的RAM和CPU
可伸缩性
额外的功能,如“你的意思是?”,相关搜索等
任何使用过上述搜索引擎或其他不在列表中的引擎的人——我很想听听你的意见。
编辑:至于索引需求,由于用户不断地向站点输入数据,这些数据将需要不断地进行索引。它不必是实时的,但理想情况下,新数据在索引中显示的延迟不超过15 - 30分钟
很高兴看到有人插话谈论Lucene——因为我对此一无所知。
而斯芬克斯,我很了解,所以看看我能不能帮上忙。
Result relevance ranking is the default. You can set up your own sorting should you wish, and give specific fields higher weightings.
Indexing speed is super-fast, because it talks directly to the database. Any slowness will come from complex SQL queries and un-indexed foreign keys and other such problems. I've never noticed any slowness in searching either.
I'm a Rails guy, so I've no idea how easy it is to implement with Django. There is a Python API that comes with the Sphinx source though.
The search service daemon (searchd) is pretty low on memory usage - and you can set limits on how much memory the indexer process uses too.
Scalability is where my knowledge is more sketchy - but it's easy enough to copy index files to multiple machines and run several searchd daemons. The general impression I get from others though is that it's pretty damn good under high load, so scaling it out across multiple machines isn't something that needs to be dealt with.
There's no support for 'did-you-mean', etc - although these can be done with other tools easily enough. Sphinx does stem words though using dictionaries, so 'driving' and 'drive' (for example) would be considered the same in searches.
Sphinx doesn't allow partial index updates for field data though. The common approach to this is to maintain a delta index with all the recent changes, and re-index this after every change (and those new results appear within a second or two). Because of the small amount of data, this can take a matter of seconds. You will still need to re-index the main dataset regularly though (although how regularly depends on the volatility of your data - every day? every hour?). The fast indexing speeds keep this all pretty painless though.
我不知道这是否适用于您的情况,但Evan Weaver比较了一些常见的Rails搜索选项(Sphinx, Ferret (Lucene的Ruby移植)和Solr),运行了一些基准测试。我想可能有用。
我还没有深入研究MySQL的全文搜索,但我知道它在速度和功能方面都无法与Sphinx、Lucene或Solr竞争。
我会把mnoGoSearch添加到列表中。非常高性能和灵活的解决方案,它作为谷歌:索引器从多个站点获取数据,您可以使用基本标准,或发明自己的钩子,以获得最大的搜索质量。它还可以直接从数据库中获取数据。
这个解决方案今天还不太为人所知,但它满足了最大的需求。你可以编译并安装它,或者在独立的服务器上,甚至在你的主服务器上,它不需要像Solr那样多的资源,因为它是用C编写的,即使在小型服务器上也能完美运行。
一开始你需要自己编译,所以需要一些知识。我为Debian做了一个小脚本,可能会有帮助。欢迎任何调整。
当你使用Django框架时,你可以在中间使用PHP客户端,或者在Python中找到解决方案,我看到了一些文章。
当然,mnoGoSearch是开源的,GNU GPL。
我不了解Sphinx,但是对于Lucene和数据库全文搜索,我认为Lucene的性能是无与伦比的。只要你正确地设置了Lucene索引,无论你要搜索多少条记录,你都应该能够在10毫秒内完成几乎所有的搜索。
最大的障碍来了:就我个人而言,我认为在你的项目中集成Lucene并不容易。当然,设置它并不难,这样你就可以做一些基本的搜索,但如果你想充分利用它,获得最佳性能,那么你绝对需要一本关于Lucene的好书。
至于CPU和内存的需求,在Lucene中执行搜索并不会给CPU带来太多的任务,尽管索引你的数据是,尽管你不经常这样做(可能一天一次或两次),所以这不是一个很大的障碍。
它不能回答你所有的问题,但简而言之,如果你有大量的数据要搜索,并且你想要出色的性能,那么我认为Lucene绝对是一条出路。如果你没有那么多的数据要搜索,那么你也可以使用数据库全文搜索。在我的书中,设置MySQL全文搜索绝对更容易。
很高兴看到有人插话谈论Lucene——因为我对此一无所知。
而斯芬克斯,我很了解,所以看看我能不能帮上忙。
Result relevance ranking is the default. You can set up your own sorting should you wish, and give specific fields higher weightings.
Indexing speed is super-fast, because it talks directly to the database. Any slowness will come from complex SQL queries and un-indexed foreign keys and other such problems. I've never noticed any slowness in searching either.
I'm a Rails guy, so I've no idea how easy it is to implement with Django. There is a Python API that comes with the Sphinx source though.
The search service daemon (searchd) is pretty low on memory usage - and you can set limits on how much memory the indexer process uses too.
Scalability is where my knowledge is more sketchy - but it's easy enough to copy index files to multiple machines and run several searchd daemons. The general impression I get from others though is that it's pretty damn good under high load, so scaling it out across multiple machines isn't something that needs to be dealt with.
There's no support for 'did-you-mean', etc - although these can be done with other tools easily enough. Sphinx does stem words though using dictionaries, so 'driving' and 'drive' (for example) would be considered the same in searches.
Sphinx doesn't allow partial index updates for field data though. The common approach to this is to maintain a delta index with all the recent changes, and re-index this after every change (and those new results appear within a second or two). Because of the small amount of data, this can take a matter of seconds. You will still need to re-index the main dataset regularly though (although how regularly depends on the volatility of your data - every day? every hour?). The fast indexing speeds keep this all pretty painless though.
我不知道这是否适用于您的情况,但Evan Weaver比较了一些常见的Rails搜索选项(Sphinx, Ferret (Lucene的Ruby移植)和Solr),运行了一些基准测试。我想可能有用。
我还没有深入研究MySQL的全文搜索,但我知道它在速度和功能方面都无法与Sphinx、Lucene或Solr竞争。