3S: How it works & what it does

Architecture and components

Indexing Broker

The Indexing Broker is a highly configurable multi-threaded queuing, transformation and processing engine allowing for large quantities of content to be normalized and ingested by the search engine. It has 2 ingestion modes: 1) A REST interface for synchronous ingestion and 2) a watch folder system for asynchronous batch processing.

Semantic Search Engine

The Semantic Search Engine is the heart of the system. It consists of an enterprise-class information retrieval technology paired with Nstein Text Mining Engine and custom algorithms to increase the relevancy of results and the accuracy of suggested documents. The system comes with extensible pre-configured reference schemas. Documents ingested by the system are automatically enriched with metadata containing people names, organizations, subjects, topics, places and landmarks, industry terms, subjectivity, sentiment and an automatic summary.

Back-office

The Back-office provides the tools necessary to support operations of the system. It allows content editors to intuitively tweak search algorithms and force content in specific positions in the search results. Sales operations can manage campaigns via keyword-based rules and drive truly contextual advertizing. System administrators are provided with the maintenance tools required for managing and maintaining the search cluster.

Presentation Framework

The Presentation Framework is a highly cached and easy-to-extend API and template system aimed at delivering large volume of query results through a flexible presentation layer. It can be extended to build custom search-driven Web applications or used as is with the out-of-the-box themes. The presentation layer leverages the semantic metadata as well any other metadata stored in the indices.

A non-programmer can design and integrate a new search-driven theme with very little time and effort while a programmer will enjoy the extensible library architecture and the hook mechanisms to build-in custom business logic. The system ships with 4 white-label ready-to-use themes: 1) Guided search, 2) Topic pages, 3) Discovery engine and 4) Mobile delivery.

Semantic Widget Server

The Semantic Widget Server is a service sitting on an API that provides results fragments to the Presentation Framework and to other external systems. All data and metadata stored in the search indices or compiled by the system as well as features supported by the core API are exposed via the Semantic Widget Server: Semantic clusters, search results, popular documents, recent documents, popular queries, etc. This conveniently allows developers to build applications leveraging 3S’s business logic in remote systems.

The system natively serves fragment in XML, JSON and XHTML but is extensible to support other formats. It also ships with a reference JavaScript widget engine allowing content to be served across properties via interactive Web widgets.

Enterprise clustering

Although Nstein 3S components can reside on 1 machine for low-cost operations, all components are also distributable in high availability clusters for high volume/traffic deployments.

The system is designed to support organizations with broad requirements but limited operational budgets as well as large enterprises with complex master/slave configurations.

Suggested low-end configuration: One master with all components with a daily backup on an external media

Mid range configuration: One master with all components with a second as failover with a second as failover

Enterprise configuration: Components scattered across clusters