diff --git a/Encodings.md b/Encodings.md index 0a07100a..2a9778b2 100644 --- a/Encodings.md +++ b/Encodings.md @@ -77,13 +77,17 @@ endian integer, followed by the bytes. The dictionary encoding builds a dictionary of values encountered in a given column. The dictionary will be stored in a dictionary page per column chunk. The values are stored as integers using the [RLE/Bit-Packing Hybrid](#RLE) encoding. If the dictionary grows too big, whether in size -or number of distinct values, the encoding will fall back to the plain encoding. The dictionary page is -written first, before the data pages of the column chunk. +or number of distinct values, the writer may stop using dictionary encoding and fall back to another +valid encoding for subsequent data pages. The fallback encoding is chosen by the writer and recorded +in the data page header's `encoding` field; readers must use that field to determine whether each +data page is dictionary encoded or uses another encoding. The dictionary page is written first, +before the data pages of the column chunk. Dictionary page format: the entries in the dictionary using the [plain](#PLAIN) encoding. -Data page format: the bit width used to encode the entry ids stored as 1 byte (max bit width = 32), -followed by the values encoded using the RLE/Bit-Packing described above (with the given bit width). +Dictionary-encoded data page format: the bit width used to encode the entry ids stored as 1 byte +(max bit width = 32), followed by the values encoded using the RLE/Bit-Packing described above +(with the given bit width). Using the `PLAIN_DICTIONARY` enum value is deprecated, use `RLE_DICTIONARY` in a data page and `PLAIN` in a dictionary page for new Parquet files.