Enterprise search in plone using solr

Enterprise Search in
Plone using Solr
Calvin Hendryx-Parker
Plone Conference 2010
Wednesday, October 27, 2010

PLONE CONFERENCE 2010
• Java Based
• Full-Text Search
• Web Services API
• Standards Based Interfaces
• Scalable
• XML Conﬁguration
• Extensible
What is Solr?

• Indexing
• Query
Playing with Solr

• Data Schema
• Faceted Search
• Administrative
Interface
• Incremental Updates
• Supports Sharding
• Index Databases, Local
Files and Web Pages
• Supports Multiple
Indexes
Solr Features

• Stopwords
• Synonyms
• Highlighted Context
Snippets
• Spelling Suggestions
• More Like This
Suggestions
• Supports Rich
Documents
Solr Features

PLONE CONFERENCE 2010Solr Performance
• Wiktionary Dataset
• 49.5 Millions lines of XML
• 1.3 GB of data
• 1.7 Million Pages Indexed in 5.5 hours
• ZODB Size after import 1.1GB

collective.solr
Integration Options
with Plone

• Monkey Patching
• Relies on collective.indexing
• Duplicates all indexes
• Sub-Optimal Integration with Zope Transactions
• Relies on Thread Locals
collective.solr Issues

What to do?

Reevaluate

• No Monkey Patching
• Simpler Code
Solr Integration as a
Catalog Index

• ZCatalog Index
• Doesn't depend on
Plone
• Utilizes new
foreign_connections
Connection Method
• Pass through Solr
Queries
• Direct access to the
Solr Response
Enter alm.solrindex

• Still handled by the ZCatalog
• Could change in the future
Sorting

• Handle Parsing Attributes for Indexing
• Translate ﬁeld-speciﬁc queries to Solr
• Registered as Zope Utilities
alm.solrindex Field
Handlers

<html>
<body>
<h3>Code Sample</h3>
<p>Replace this text!</p>
</body>
</html>
Example Handler
class TextFieldHandler(DefaultFieldHandler):
def parse_query(self, field, field_query):
name = field.name
request = {name: field_query}
record = parseIndexRequest(request, name, ('query',))
if not record.keys:
return None
query_str = ' '.join(record.keys)
if not query_str:
return None
return {'q': u'+%s:%s' % (name, quote_query(query_str))}

• GenericSetup Proﬁle
• Tests
• Uses solrpy instead of
the unsupported
solr.py
Other alm.solrindex
Features

• Can replace several ZCatalog indexes
• Remove any indexes you have replaced
• Use it for all Text Indexes
• Still Utilize the ZCatalog Indexes for Everything Else
Tips

Demo
Project Gutenburg Data

Questions?

Check out
sixfeetup.com/demos

Enterprise search in plone using solr

More Related Content

Similar to Enterprise search in plone using solr

More from Calvin Hendryx-Parker

Recently uploaded

Enterprise search in plone using solr