What is a Parquet file?

You have a .parquet file sitting in a bucket. You open it the way you'd open a CSV, and you get a few readable words drowning in binary noise. Somewhere near the start there are four bytes spelling PAR1, and the same four bytes at the very end.

That's the whole story in miniature: Parquet is not a text file with a different extension. It's a binary format designed so that a query engine can skip almost all of it.

Row by row, or column by column

A CSV stores one record per line. Everything about the first customer, then everything about the second:

id,country,amount,created_at
1,FR,42.00,2026-01-04
2,DE,17.50,2026-01-04

Parquet turns that ninety degrees. All the id values are stored together, then all the country values, then all the amount values. Conceptually:

id:         1, 2, 3, 4, …
country:    FR, DE, FR, FR, …
amount:     42.00, 17.50, 8.25, …
created_at: 2026-01-04, 2026-01-04, …

Same data, different physical layout. Everything else about Parquet follows from this.

Why anyone bothers

You read only the columns you ask for. A table with 60 columns, and your query touches three of them. A CSV reader has to walk every byte of every row to find the commas. A Parquet reader seeks to the three column chunks it needs and ignores the rest. On wide tables this alone is often a 10x to 20x difference.

Similar values compress absurdly well. A column of country codes is thousands of repetitions of a handful of strings. Stored together, it compresses to almost nothing. Parquet also applies encodings before compression. Dictionary encoding replaces repeated values with small integers, and run-length encoding collapses repeats into a count. A CSV compresses the whole file as one soup of mixed types; Parquet compresses each column with the scheme that suits it. It is routinely 5 to 10 times smaller than the same data as CSV.

It knows what it contains. Parquet carries a schema. A column is an INT64 or a BYTE_ARRAY (UTF8) or a TIMESTAMP, and that is written in the file. There is no argument about whether 007 is a number or a string, no date that parses differently on another machine. A CSV is just text, and every reader guesses.

It can skip data it knows is irrelevant. Rows are grouped into blocks called row groups, typically 128 MB or so. For every column in every row group, Parquet stores statistics: min, max, null count. Ask for WHERE amount > 1000 and the reader checks each row group's max for amount. If the max is 300, it skips the entire block without decompressing a byte. This is called predicate pushdown, and on sorted or clustered data it can eliminate most of the file.

The structure, briefly

PAR1
  Row group 1
    Column chunk: id       → pages
    Column chunk: country  → pages
    Column chunk: amount   → pages
  Row group 2
    …
  Footer  ← schema + per-column statistics + byte offsets
PAR1

The footer at the end is the key. It holds the schema and the location of everything. A reader fetches the last few kilobytes first, learns the layout, then issues targeted range requests for the pieces it actually needs.

This is also why Parquet is awkward to stream. You can append to a CSV forever. A Parquet file is not complete until its footer is written. That is why tools that write Parquet produce many files rather than one growing file, and why a bucket full of part-00000-xxx.parquet files is such a common sight.

Parquet, CSV, Avro

The rule of thumb: Avro to move data, Parquet to query it. Avro is row-oriented, so it handles record-by-record writes and schema evolution well, which fits event pipelines. Parquet is columnar, so it wins the moment something scans the data.

And Delta Lake, Iceberg, Hudi? They are not competing formats. They are table formats built on top of Parquet files, adding a transaction log so you get atomic writes, time travel and schema evolution. A Delta table is a folder of ordinary Parquet files plus a _delta_log/ directory. Open any file inside it and you'll find PAR1.

When Parquet is the wrong answer

• Small files. Under a few megabytes the metadata overhead outweighs the benefit, and you have made a readable file unreadable for nothing.
• You need to read whole rows, one at a time. Fetching a single complete record means touching every column chunk. Row formats do that better.
• Someone has to open it in a spreadsheet. Then it's CSV, and there is nothing to discuss.
• Constant appends. The footer problem. You'll end up with thousands of tiny files and a compaction job you didn't plan for.

Actually looking inside one

import pandas as pd
print(pd.read_parquet("data.parquet").head())

-- DuckDB, no import step, reads straight from the bucket
SELECT * FROM 'gs://my-bucket/data.parquet' LIMIT 10;

# parquet-tools: schema and row-group statistics without reading the data
parquet-tools inspect data.parquet

Start with the schema, not the rows. Nine times out of ten the question is what is in here, and the footer answers it in a few kilobytes.

A note on where I'm coming from. I build Cloud File Viewer, a Chrome extension that previews files in Google Cloud Storage without downloading them. It handles CSV, JSON, logs and gzip today. Not Parquet, not yet. Peeking at a Parquet file in a bucket still means pulling it down or firing up DuckDB.

Whether that's worth fixing is the question I'm currently trying to answer. If you'd use it, say so: contact@mlidevstudio.com. It's the most requested thing so far, and messages are what decide the order I build in.