File format

Overview

Dendrite’s file format is very similar to Parquet’s file format.

TODO: add glossary and description. Discuss units of parallelization.

Primitives

Varints

Strings

UTF-8 byte arrays. VarSint + bytes

Encodings

Metadata

Record-group

Column-chunk

Schema

Column

Collection

Record

Field

Custom Types

Custom metadata

Record Group

Column Chunk

Dictionary vs regular columns

Page

Data

Dictionary