Source freshness results from dbt v2 store a null max_loaded_at_time_ago_in_s (process_freshness_result reads age) #1098

Describe the bug

With dbt v2, every row that upload_source_freshness writes to dbt_source_freshness_results has max_loaded_at_time_ago_in_s = NULL. max_loaded_at and snapshotted_at are filled in as usual.

process_freshness_result reads the value from result_dict.age (L44). dbt-core's freshness result has age, but the result dbt v2 passes to the on-run-end hook does not. It has the value under max_loaded_at_time_ago_in_s, the name dbt uses in sources.json. Logging result.to_dict() in an on-run-end hook with dbt v2 2.0.11 shows these keys:

['adapter_response', 'criteria', 'execution_time', 'max_loaded_at', 'max_loaded_at_time_ago_in_s', 'node', 'snapshotted_at', 'status', 'thread_id', 'timing', 'unique_id']

with, for example, max_loaded_at_time_ago_in_s = 49349.0, while result.age is undefined.

The null hides the freshness details in the Elementary report, and freshness alerts show "Time Elapsed: N/A".

To Reproduce

1. In a project with the Elementary package, configure freshness on a source.
2. Run dbt source freshness with dbt v2.
3. In dbt_source_freshness_results, the new row has max_loaded_at_time_ago_in_s = NULL.

On BigQuery, over 7 days of our scheduled dbt v2 runs, all 1046 rows had a null max_loaded_at_time_ago_in_s, and none had a null max_loaded_at or snapshotted_at.

Expected behavior

max_loaded_at_time_ago_in_s is stored whichever engine ran the freshness check.

Environment:

• Elementary dbt package version: 0.26.1
• dbt version: dbt v2 2.0.11 (also seen with 2.0.8)
• Data warehouse: BigQuery

Additional context

This came up in elementary-data/elementary#2402, where the capitalized dbt v2 statuses (Pass) blanked the report's test-results page. That part is fixed on the CLI side in elementary-data/elementary#2403.

That issue also mentioned that process_freshness_result compares status == "runtime error" case-sensitively. With dbt v2 2.0.11 this doesn't come up: when a source's freshness check errors, dbt v2 leaves the source out of results (and out of sources.json), so the hook never sees it.

Would you be willing to contribute a fix for this issue?

Yes, PR to follow.