Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ every such clause. The count rose from 16 notes to 44 across the three parts,
which was not a regression: the new clauses — validation, security, inference,
test vectors — are places the previous structure had no room to admit a gap in.

A pass over the vendored upstream documentation has since closed 18 gaps,
A pass over the vendored upstream documentation has since closed 20 gaps,
taking Part 1 from 34 editorial notes to 24 and the three parts together from
44 to 34. (Twelve notes went; two new ones came, both in the narrow-type
conversion rules, where reading the source closely raised a question the
Expand Down
61 changes: 34 additions & 27 deletions sources/part1/sections/06-type-system.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -144,33 +144,19 @@ leading bits; the two differ in sign as well as in magnitude. State one.
[[packing]]
==== Packing

A tensor whose element type is narrower than eight bits SHALL be stored as a
bit stream, the elements in order, each element occupying the next free bits of
the current octet counting from its least significant bit.

For a 4-bit element type, two elements occupy one octet: the first in the four
least significant bits, the second in the four most significant bits. For
consecutive elements `x` and `y`, the octet `z` is

[%unnumbered]
----
z = ((y & 0x0F) << 4) | (x & 0x0F)
----

and the elements are recovered as `x = z & 0x0F` and `y = (z >> 4) & 0x0F`.

For a 2-bit element type, four elements occupy one octet, the element at index
`i` of the group occupying bits `2i` and `2i + 1`.

For a 6-bit element type, the codes are packed into a contiguous bit stream in
the same direction; an element therefore MAY span two octets.

The storage size in octets of a tensor of `N` elements of a type of width `w`
bits SHALL be the least integer not less than `N * w / 8`. Any bits of the
final octet not occupied by an element SHALL be zero, and a consumer SHALL
ignore them.

.Storage size
A tensor carries its elements either as an octet string, called raw storage
here, or in a field typed for its elements, called typed storage. The two pack
a narrow element type differently, and a tensor uses one or the other, not
both.

In raw storage, the elements SHALL be a bit stream, in order, each element
occupying the next free bits of the current octet counting from its least
significant bit. The storage size in octets of a tensor of `N` elements of a
type of width `w` bits SHALL be the least integer not less than `N * w / 8`.
Any bits of the final octet not occupied by an element SHALL be zero, and a
consumer SHALL ignore them.

.Storage size in raw storage
[example]
====
A tensor of 7 `INT4` elements occupies 4 octets, of which the four most
Expand All @@ -179,6 +165,27 @@ elements occupies 6 octets, of which the two most significant bits of the sixth
octet are padding.
====

In typed storage, a narrow element type uses a sequence of 32-bit integers, and
<<tbl-typed-storage>> states how many elements each integer carries and which
of its bits they occupy. Every other bit of such an integer SHALL be zero.

[[tbl-typed-storage]]
.Narrow element types in typed storage
[cols="2,1,4"]
|===
| Element width | Elements per integer | Bits occupied

| 2 bits | 4 | bits 0 to 1, 2 to 3, 4 to 5 and 6 to 7, in element order
| 4 bits | 2 | bits 0 to 3 and 4 to 7, in element order
| 6 bits | 1 | bits 0 to 5
|===

A producer SHOULD use raw storage for a 6-bit element type. Typed storage costs
at least one octet per element there, so it is not a packing at all.

A floating-point element type narrower than 32 bits SHALL be written to typed
storage as the unsigned integer whose bits are those of the element.

[[string-elements]]
=== String elements and character encoding

Expand Down
29 changes: 29 additions & 0 deletions sources/part1/sections/14-versioning.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

=== Version axes

==== General

ONNX has three independent version axes:

IR version:: the version of the structural and serialization rules given in
Expand All @@ -14,6 +16,33 @@ published in <<onnx-part-2>>.
Model version:: an identifier assigned by the <<producer>>, opaque to this
document.

[[ir-version-specified]]
==== The IR version this edition specifies

This edition specifies IR version 14, as published in ONNX release 1.23.0.

<<tbl-ir-versions>> is the version history.

[[tbl-ir-versions]]
.IR versions, the release each was published in, and what each introduced
[cols="1,1,4"]
|===
| IR version | ONNX release | Introduced

| 8 | 1.10.0 | the sparse tensor and optional types; model-local functions
| 9 | 1.14.0 | attribute default values in a function; the four FP8 types
| 10 | 1.16.0 | `UINT4` and `INT4`; the function overload; metadata properties
| 11 | 1.18.0 | `FLOAT4E2M1`; multi-device configuration
| 12 | 1.19.0 | `FLOAT8E8M0`
| 13 | 1.20.0 | `UINT2` and `INT2`
| 14 | 1.23.0 | `FLOAT6E2M3` and `FLOAT6E3M2`; the opaque type outside the ONNX-ML build
|===

NOTE: The release is the publication event. The schema source carries a
publication date against each IR version as well, and for IR version 14 it
still reads "TBD" although 1.23.0 shipped it; the date in the schema is not
maintained and is not relied on here.

=== IR version evolution

The IR version is a positive integer that increases monotonically.
Expand Down
6 changes: 6 additions & 0 deletions sources/part1/sections/annex-c-known-gaps.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,12 @@ transcription rather than take it on trust, and so that a refresh of
| <<element-types>> (complete enumeration)
| `onnx-ml.proto`, `TensorProto.DataType`

| <<packing>> (typed storage)
| `onnx-ml.proto`, `TensorProto.int32_data`

| <<tbl-ir-versions>> (the IR version history)
| `docs/Versioning.md`, "Released Versions"; `onnx-ml.proto`, `Version`

| <<reduced-precision>> (narrow types and their packing)
| `docs/docsgen/source/technical/`: `float4.md`, `float6.md`, `float8.md`,
`int2.md`, `int4.md`
Expand Down
Loading