whichdb

CrowdDB

discontinued

by UC Berkeley and ETH Zurich · since 2011

Berkeley/ETH research database that routed parts of SQL queries to human workers on Mechanical Turk.

relationalresearch prototypeproprietarylegacy
Website Compare Find similar

discontinued. Academic prototype (SIGMOD 2011, UC Berkeley and ETH Zurich); never released.

Overview

CrowdDB was a research system from UC Berkeley and ETH Zurich, presented at SIGMOD 2011 by Michael Franklin, Donald Kossmann, Tim Kraska, Sukriti Ramesh and Reynold Xin, that treated people as a query-processing resource. Its CrowdSQL dialect let schemas mark tables or columns as crowdsourced, so a query could fill in missing values, add missing rows, or answer subjective comparisons and entity-resolution questions that machines handle poorly. The optimizer added crowd operators such as CrowdProbe, CrowdJoin and CrowdCompare, generated task user interfaces automatically from the schema, and posted them to Amazon Mechanical Turk, weighing cost, latency and answer quality. It was a prototype, never released as a product, but it started a line of work on human-in-the-loop query processing.

Analytics (OLAP)

Strengths & trade-offs

Strengths

  • +Extended SQL so missing data and subjective questions could be answered by people
  • +Generated crowd task UIs automatically from the schema
  • +Optimizer balanced monetary cost, latency and answer quality

Trade-offs

  • −Never released; no code to run
  • −Human answers are slow, costly and noisy

Great for

  • +Studying human-in-the-loop query processing
  • +Background for crowdsourced data cleaning and labelling research

Think twice for

  • −Any production use

Capabilities

Native or official first-party support · 1 of 25 tracked

✓Joins
–Transactions
–Secondary indexes
–Full-text search
–Vector search
–Geospatial
–JSON
–Change data capture
–Triggers
–Stored procedures
–Materialized views
–TTL / expiry
–Pub/Sub
–Columnar storage
–In-memory
–Durable persistence
–Compression
–Encryption at rest
–Role-based access
–Time travel
–Branching
–Online schema changes
–Streaming ingest
–Offline sync
–Runs in WebAssembly

Scale & consistency

Comfortable data size
Small · ≤ 100 GB
Horizontal scaling
None
Multi-region
No
Transactions
Limited ACID
Consistency
Eventual
Schema
Strict
Architecture
Single node
Query languages
CrowdSQL

Deployment

Self-hosted Vendor-managed cloud Serverless Embedded Edge Kubernetes operator

Alternatives to CrowdDB

Compare all→
PostgreSQLRelational93.2

The extensible, standards-first open-source relational database.

crowddb vs postgresql →
DuckDBColumnar / OLAP63.7

In-process columnar SQL database: the SQLite of analytics, running inside your app or notebook.

crowddb vs duckdb →
IBM NetezzaRelational15.1

IBM's FPGA-accelerated MPP warehouse, now offered as NPS on-prem, in Cloud Pak and as a service.

crowddb vs ibm-netezza →
MondrianRelational12.4

Java ROLAP engine that answers MDX queries by generating SQL against your relational warehouse.

crowddb vs mondrian →
Google F1Relational0.0

Google's internal distributed SQL database on Spanner that replaced sharded MySQL for AdWords.

crowddb vs google-f1 →
Apache KylinColumnar / OLAP16.9

OLAP engine that pre-computes cubes and aggregate indexes for sub-second SQL on huge datasets.

crowddb vs apache-kylin →

CrowdDB FAQ

What is CrowdDB?

CrowdDB is a relational database by UC Berkeley and ETH Zurich, first released in 2011. CrowdDB was a research system from UC Berkeley and ETH Zurich, presented at SIGMOD 2011 by Michael Franklin, Donald Kossmann, Tim Kraska, Sukriti Ramesh and Reynold Xin, that treated people as a query-processing resource. Its CrowdSQL dialect let schemas mark tables or columns as crowdsourced, so a query could fill in missing values, add missing rows, or answer subjective comparisons and entity-resolution questions that machines handle poorly. The optimizer added crowd operators such as CrowdProbe, CrowdJoin and CrowdCompare, generated task user interfaces automatically from the schema, and posted them to Amazon Mechanical Turk, weighing cost, latency and answer quality. It was a prototype, never released as a product, but it started a line of work on human-in-the-loop query processing.

Is CrowdDB open source?

No. CrowdDB is proprietary, closed-source software. License: Unreleased research code.

Does CrowdDB support ACID transactions?

Partly. CrowdDB offers limited ACID guarantees (for example, single-record or single-partition transactions), with eventual consistency.

Does CrowdDB scale horizontally?

CrowdDB does not scale horizontally; it runs on a single node, so you scale it up rather than out. It is comfortable at small scale (≤ 100 GB).

What is CrowdDB best for?

CrowdDB is a strong fit for studying human-in-the-loop query processing; and background for crowdsourced data cleaning and labelling research. Think twice for any production use.

What are the best alternatives to CrowdDB?

Common alternatives to CrowdDB include PostgreSQL, DuckDB, IBM Netezza, Mondrian and Google F1. The right choice depends on your workload, scale and hosting; WhichDB's smart match ranks them against your requirements.

Is CrowdDB still maintained?

CrowdDB is discontinued. Academic prototype (SIGMOD 2011, UC Berkeley and ETH Zurich); never released.

How popular is CrowdDB?

CrowdDB has little measurable public adoption today, so it ranks near the bottom of the WhichDB Score.