SenseiDB
discontinuedby LinkedIn · since 2010 · also known as Sensei, LinkedIn Sensei
LinkedIn's distributed, real-time faceted-search database on Lucene, Zoie and Bobo; archived since 2014.
discontinued. Development ended around 2013 and LinkedIn's repository is archived (LinkedInAttic). Use Elasticsearch, OpenSearch or Apache Solr.
Overview
SenseiDB was a distributed, elastic database for semi-structured data that LinkedIn built and open-sourced around 2010, and which powered parts of the LinkedIn homepage and the LinkedIn Signal product. It combined Lucene for full-text search, Zoie for real-time indexing and Bobo for fast faceted browsing, sharded and replicated across nodes coordinated by ZooKeeper, and offered BQL, an SQL-like language with MATCH ... AGAINST, GROUP BY and BROWSE BY facets. Data streamed in through gateways for Kafka, JDBC, JMS and files, or was bootstrapped from HDFS with a MapReduce indexing job. Development stopped after the 1.5.x releases around 2012-2013, and LinkedIn archived the repository; Elasticsearch and OpenSearch cover the same ground today.
Strengths & trade-offs
Strengths
- +Real-time indexing: updates searchable within moments of arriving
- +Fast faceted browsing via Bobo alongside full-text relevance
- +SQL-like BQL with facets, grouping and full-text predicates
- +Ingest gateways for Kafka, JDBC and JMS plus Hadoop bootstrapping
- +Proven at LinkedIn scale in its day
Trade-offs
- −Archived and unmaintained for over a decade
- −Built on old Lucene versions with no upgrade path
- −No transactions or strong consistency
Great for
- +Historical reference for real-time faceted search architecture
- +Migrating legacy Sensei deployments to Elasticsearch or OpenSearch
Think twice for
- −Any new search deployment
- −Transactional data
Capabilities
Native or official first-party support · 5 of 25 tracked
Scale & consistency
- Comfortable data size
- Medium · ≤ 10 TB
- Horizontal scaling
- Native
- Multi-region
- No
- Transactions
- None
- Consistency
- Eventual
- Schema
- Flexible
- Architecture
- Distributed, Primary replica
- Query languages
- BQL (Browse Query Language)
Sharded and replicated across nodes with ZooKeeper; replicas converge eventually.
Deployment
Ecosystem
Alternatives to SenseiDB
Lucene-based distributed search and analytics engine for text, vectors, logs and metrics.
senseidb vs elasticsearch →Apache-licensed, foundation-governed fork of Elasticsearch for search, vectors and observability.
senseidb vs opensearch →The original Lucene search server: battle-tested, Apache-governed, now with GPU-accelerated vectors.
senseidb vs apache-solr →Big-data serving engine for search, recommendation and RAG with tensors and ML ranking.
senseidb vs vespa →Fast SQL-first search database, forked from Sphinx, with columnar storage and vector KNN.
senseidb vs manticore-search →Lightweight single-binary full-text search engine in Go with Elasticsearch-compatible ingest.
senseidb vs zincsearch →SenseiDB FAQ
What is SenseiDB?
SenseiDB is a search engine database by LinkedIn, first released in 2010. SenseiDB was a distributed, elastic database for semi-structured data that LinkedIn built and open-sourced around 2010, and which powered parts of the LinkedIn homepage and the LinkedIn Signal product. It combined Lucene for full-text search, Zoie for real-time indexing and Bobo for fast faceted browsing, sharded and replicated across nodes coordinated by ZooKeeper, and offered BQL, an SQL-like language with MATCH ... AGAINST, GROUP BY and BROWSE BY facets. Data streamed in through gateways for Kafka, JDBC, JMS and files, or was bootstrapped from HDFS with a MapReduce indexing job. Development stopped after the 1.5.x releases around 2012-2013, and LinkedIn archived the repository; Elasticsearch and OpenSearch cover the same ground today.
Is SenseiDB open source?
Yes. SenseiDB is open source. License: Apache License 2.0.
Does SenseiDB support ACID transactions?
No. SenseiDB does not provide ACID transactions; its consistency model is eventual.
Does SenseiDB scale horizontally?
SenseiDB scales horizontally out of the box, with built-in sharding or partitioning. It is comfortable at medium scale (≤ 10 TB).
How can I run SenseiDB?
You can self-host it.
What is SenseiDB best for?
SenseiDB is a strong fit for historical reference for real-time faceted search architecture and migrating legacy Sensei deployments to Elasticsearch or OpenSearch. Think twice for any new search deployment.
What are the best alternatives to SenseiDB?
Common alternatives to SenseiDB include Elasticsearch, OpenSearch, Apache Solr, Vespa and Manticore Search. The right choice depends on your workload, scale and hosting; WhichDB's smart match ranks them against your requirements.
Is SenseiDB still maintained?
SenseiDB is discontinued. Development ended around 2013 and LinkedIn's repository is archived (LinkedInAttic). Use Elasticsearch, OpenSearch or Apache Solr.
How popular is SenseiDB?
SenseiDB ranks #611 of 844 databases on the WhichDB Score (0.8/100), and #25 among search engine databases. The score combines public adoption signals such as package downloads, GitHub activity, job posts and developer discussion.