Compute and store position of TNodes during parsing for improved error messages.

[ianjosephwilson]

Design Notes

Goals

• don't overload the cache (and don't make the current cache incompatible)
• don't add a bunch of info to the core apis that we have to work around
• allow the tracking to be disabled (potentially)

Translation

(template_part_index, offset_in_template_part) = translate(parser_line_number, offset_in_parser_line)

The underlying HTMLParser provides a line position via getpos() that is relative to the entire template flattened into a continuous string with interpolations replaced with placeholders. That line position is translated to a "part" position in the Template using unified indexing (combined strings and interpolations). This part position applies to all templates with the same tuple of strings.

Note that the purpose of the "unified indexing" (all this 2*len(t.strings) - 1 or index // 2 madness) is that it allows for exact storage of the part location and also has less complicated usage and more direct iteration. This is especially true when starting at a part and ending at a part compared to storing and trying to iterate from a s_index or an i_index.

Source Position and Info

The translation from line position to part position is performed for all nodes during parsing and then stored with that tnode. For tags (elements and components) more info is wanted. This extra source info is stored in a lookup to avoid overloading the cache and core apis. A new TTree structure is used to store the root: TNode and the lookup packed as sinfos. This keeps the core TNode subclasses simple as they only have one extra field: source_pos: PartPosition. This assumes that if a node needs extra source info then its source position is unique within that template which seems sensible for now.

Next Steps

Assuming we don't hit any brick wall blockers we'd keep going along until we get to semi-functioning exception handling:

• part 4: add TemplateRef slicing
• part 5: add SourceReader and printable repr of Templates
• part 6: parser exceptions and handling
• part 7: processor exceptions and handling

[ianjosephwilson]

@davepeck Thanks for merging the last part. This part is ready to review.