ArcticDB
activeby Man Group · since 2023 · also known as Arctic
Serverless DataFrame database from Man Group: Pandas in, versioned columnar data on S3 out.
Overview
ArcticDB is a C++ storage engine with a Python API that stores Pandas and Polars DataFrames directly on S3, Azure Blob or local LMDB, with no server to run. Every write is versioned, so you can read a symbol as of any earlier version or snapshot, and data is chunked by row and column so slicing by date range and columns stays fast even on very wide tables. It was built by Man Group as the successor to its MongoDB-based Arctic library and open-sourced in 2023 under the Business Source License, with each version converting to Apache-2.0 two years after release. It is a research data store for Python quants, not a SQL database.
Strengths & trade-offs
Strengths
- +No server: Python clients talk straight to S3, Azure Blob or LMDB
- +Every write is versioned, with snapshots for reproducible research
- +Handles very wide DataFrames (hundreds of thousands of columns)
- +Fast date-range and column slicing
- +Built and used in production by a large hedge fund
Trade-offs
- −Python-centric; no SQL interface
- −BSL licence restricts offering it as a database service
- −Concurrent writers to the same symbol need care
- −Not suited to low-latency streaming or many small writes
Great for
- +Quant research data stores
- +Versioned financial time series
- +Feature and factor libraries for backtesting
- +Scientific DataFrame storage on object storage
Think twice for
- −SQL users and BI tools
- −Real-time ingestion
- −Multi-language application backends
Capabilities
Native or official first-party support · 4 of 25 tracked
Scale & consistency
- Comfortable data size
- Large · ≤ 1 PB
- Horizontal scaling
- Native
- Multi-region
- No
- Transactions
- Limited ACID
- Consistency
- Eventual
- Schema
- Flexible
- Architecture
- Embedded, Serverless, Shared storage
- Query languages
- Python API, QueryBuilder
Clients read and write object storage directly, so throughput scales with clients and the store.
Deployment
Ecosystem
Alternatives to ArcticDB
Column-store time-series engine with the q language, a fixture of capital-markets tick data.
arcticdb vs kdb-plus →Columnar time-series SQL database built for market data and very high ingest rates.
arcticdb vs questdb →Distributed time-series database and scripting platform for quant finance and industrial analytics.
arcticdb vs dolphindb →In-process columnar SQL database: the SQLite of analytics, running inside your app or notebook.
arcticdb vs duckdb →Embedded, serverless retrieval database on the Lance columnar format for multimodal AI data.
arcticdb vs lancedb →Embeddable multidimensional array storage engine for dense and sparse data on object storage.
arcticdb vs tiledb →ArcticDB FAQ
What is ArcticDB?
ArcticDB is a time-series database by Man Group, first released in 2023. ArcticDB is a C++ storage engine with a Python API that stores Pandas and Polars DataFrames directly on S3, Azure Blob or local LMDB, with no server to run. Every write is versioned, so you can read a symbol as of any earlier version or snapshot, and data is chunked by row and column so slicing by date range and columns stays fast even on very wide tables. It was built by Man Group as the successor to its MongoDB-based Arctic library and open-sourced in 2023 under the Business Source License, with each version converting to Apache-2.0 two years after release. It is a research data store for Python quants, not a SQL database.
Is ArcticDB open source?
Not in the OSI sense. ArcticDB is source-available: you can read the code, but the license restricts some uses, typically offering it as a competing managed service. License: Business Source License 1.1 (converts to Apache-2.0 after 2 years).
Does ArcticDB support ACID transactions?
Partly. ArcticDB offers limited ACID guarantees (for example, single-record or single-partition transactions), with eventual consistency.
Does ArcticDB scale horizontally?
ArcticDB scales horizontally out of the box, with built-in sharding or partitioning. It is comfortable at large scale (≤ 1 PB).
How can I run ArcticDB?
You can self-host it, embed it in your application and run it serverless.
What is ArcticDB best for?
ArcticDB is a strong fit for quant research data stores; versioned financial time series; and feature and factor libraries for backtesting. Think twice for SQL users and BI tools.
What are the best alternatives to ArcticDB?
Common alternatives to ArcticDB include kdb+, QuestDB, DolphinDB, DuckDB and LanceDB. The right choice depends on your workload, scale and hosting; WhichDB's smart match ranks them against your requirements.
How popular is ArcticDB?
ArcticDB ranks #232 of 844 databases on the WhichDB Score (14.4/100), and #13 among time-series databases. The score combines public adoption signals such as package downloads, GitHub activity, job posts and developer discussion.