Apache ZooKeeper
activeby Apache Software Foundation · since 2008 · also known as ZooKeeper, ZK
The original coordination service: hierarchical znodes, watches and ephemeral nodes for clusters.
Overview
ZooKeeper provides a replicated, in-memory tree of small data nodes (znodes) with ordered writes via the Zab protocol, watches and ephemeral nodes that disappear when a client session dies. Those primitives became the standard way to build leader election, locks, group membership and configuration for Hadoop, HBase, Solr, Pulsar and many other systems. Kafka removed its ZooKeeper dependency in 4.0 in favour of KRaft, and newer platforms tend to pick etcd, so ZooKeeper's role is now mostly supporting existing ecosystems. It is still a dependable, well-understood component where it already fits.
Strengths & trade-offs
Strengths
- +Ephemeral and sequential znodes make leader election and locks straightforward
- +Totally ordered writes with sequential consistency for clients
- +Mature recipe library (Apache Curator)
- +Deep integration with Hadoop, HBase, Solr, Pulsar and others
Trade-offs
- −Whole dataset lives in memory on every member; 1 MB znode limit
- −Session and GC tuning issues cause hard-to-debug outages
- −Declining adoption: Kafka dropped it and new systems prefer etcd
- −Watches are one-shot unless using persistent watches (3.6+)
Great for
- +Coordination for HBase, Solr, Pulsar and legacy Kafka
- +Leader election and distributed locks for JVM services
- +Cluster membership and small shared configuration
Think twice for
- −Application data storage
- −Large or frequently updated datasets
- −New cloud-native platforms (consider etcd)
Capabilities
Native or official first-party support · 6 of 25 tracked
Scale & consistency
- Comfortable data size
- Small · ≤ 100 GB
- Horizontal scaling
- None
- Multi-region
- No
- Transactions
- Limited ACID
- Consistency
- Strong
- Schema
- Schemaless
- Architecture
- Distributed, In memory, Client server
- Query languages
- ZooKeeper API
Each ensemble member keeps the full tree in memory; observers can scale reads.
Deployment
Ecosystem
Alternatives to Apache ZooKeeper
Raft-based, strongly consistent key-value store that holds Kubernetes' cluster state.
apache-zookeeper vs etcd →The key-value store built into HashiCorp Consul for service config, locks and discovery.
apache-zookeeper vs hashicorp-consul-kv →Apple's ordered, strictly serializable distributed key-value store, built to have layers on top.
apache-zookeeper vs foundationdb →etcd's maintained fork of BoltDB: a simple, pure-Go B+tree store in a single file.
apache-zookeeper vs bbolt →D. J. Bernstein's constant database: immutable on-disk hash tables with two-read lookups and atomic swaps.
apache-zookeeper vs cdb →Tiny embeddable C string key-value database from the radare2 project, built on djb's cdb format.
apache-zookeeper vs sdb →Apache ZooKeeper FAQ
What is Apache ZooKeeper?
Apache ZooKeeper is a hierarchical database by Apache Software Foundation, first released in 2008. ZooKeeper provides a replicated, in-memory tree of small data nodes (znodes) with ordered writes via the Zab protocol, watches and ephemeral nodes that disappear when a client session dies. Those primitives became the standard way to build leader election, locks, group membership and configuration for Hadoop, HBase, Solr, Pulsar and many other systems. Kafka removed its ZooKeeper dependency in 4.0 in favour of KRaft, and newer platforms tend to pick etcd, so ZooKeeper's role is now mostly supporting existing ecosystems. It is still a dependable, well-understood component where it already fits.
Is Apache ZooKeeper open source?
Yes. Apache ZooKeeper is open source. License: Apache License 2.0.
Does Apache ZooKeeper support ACID transactions?
Partly. Apache ZooKeeper offers limited ACID guarantees (for example, single-record or single-partition transactions), with strong consistency.
Does Apache ZooKeeper scale horizontally?
Apache ZooKeeper does not scale horizontally; it runs on a single node, so you scale it up rather than out. It is comfortable at small scale (≤ 100 GB).
How can I run Apache ZooKeeper?
You can self-host it.
What is Apache ZooKeeper best for?
Apache ZooKeeper is a strong fit for coordination for HBase, Solr, Pulsar and legacy Kafka; leader election and distributed locks for JVM services; and cluster membership and small shared configuration. Think twice for application data storage.
What are the best alternatives to Apache ZooKeeper?
Common alternatives to Apache ZooKeeper include etcd, Consul KV, FoundationDB, bbolt and cdb. The right choice depends on your workload, scale and hosting; WhichDB's smart match ranks them against your requirements.
How popular is Apache ZooKeeper?
Apache ZooKeeper ranks #45 of 844 databases on the WhichDB Score (39.9/100), and #1 among hierarchical databases. The score combines public adoption signals such as package downloads, GitHub activity, job posts and developer discussion.