WW warin.me

DATA SYSTEMS · INTERACTIVE LAB

Query Parquet directly, observe pushdown, windows, rollups and precomputation.

The useful question is not “which database wins?” It is “what work did the system avoid, defer, store or repeat?”

Checking isolated teaching runtimes…
ROWS10K
100K
1Mone deterministic sales model

START WITH ONE QUESTION

How much work can be avoided before analytical operators begin?

Recommended sequence

  1. Select 100,000 rows and run CSV scan. Treat this as the row-oriented file baseline.
  2. Without changing size, run Parquet scan. Compare the same answer, observed latency and physical representation.
  3. Run Filter + projection pushdown. Inspect the plan for filters and selected columns being applied near the scan.
  4. Run Window calculation to retain detail rows while adding analytical context, then run ROLLUP to produce summaries at several grains.
  5. Run Precomputed summary last. Separate faster reuse of a prepared result from improvements caused only by the file format.
What this runtime actually demonstrates

DuckDB starts as a new local process for each request on this teaching site. Its displayed time includes process startup.

Read the timing carefully

The displayed value is observed lab latency, not a universal database benchmark. PostgreSQL reuses an open PDO connection; DuckDB starts a subprocess for each request; ClickHouse communicates over HTTP and reports its own engine elapsed time. At only 10K–1M rows, much of the dataset may also remain in cache. Compare query plans, rows and bytes read, freshness and maintenance cost before comparing milliseconds.

CONCEPT

A database can be a library, not a server.

DuckDB runs inside the existing web container and opens a local database or Parquet file. Projection and filter pushdown move selection into the scan, so unused columns and irrelevant row groups can be avoided before higher operators run.

CSV / Parquet→embedded vector engine→result / local table

BOUNDED EXPERIMENT

Whitelist only · 25 s timeout · 2 CPU / 2 GB ClickHouse ceiling · one operation at a time

—execution time
—rows scanned
—bytes read
—rows returned

SQL

Select a bounded experiment.

Result

No result yet.

Explain / query plan

The physical plan appears after execution.
What happened?

Run an experiment, then connect the measurement to physical work.

Why CSV and Parquet do different work

CSV

Portable and visible, but values arrive as text. The engine must parse delimiters, recognize records and establish types before analytical operators receive vectors.

Parquet

Column types, compressed pages, statistics and row groups travel with the file. DuckDB can project needed columns and push eligible filters into the scan. File layout still matters.

Embedded boundary

No database server is listening. The controlled PHP endpoint starts a local DuckDB process against fixed paths and fixed SQL. This lowers service count, but concurrency and file ownership must remain bounded.