File format
Overview
Dendrite’s file format is very similar to Parquet’s file format.
TODO: add glossary and description. Discuss units of parallelization.
Primitives
Varints
- length variants
- zigzag (signed)
Strings
UTF-8 byte arrays. VarSint + bytes
Encodings
Metadata
Record-group
Column-chunk
Schema
Column
Collection
Record
Field
Custom Types
Custom metadata
Record Group
Column Chunk
Dictionary vs regular columns