Summary
The US Fundamental Data (Morningstar) pages do not state that the SecurityReference status fields are current reference values stamped onto every historical daily snapshot, not point-in-time values. Users qualifying point-in-time universe selection read share_class_status on a historical date and treat D (Deactive) as "delisted as of that date", which silently excludes every later-delisted company at formation time.
What we observed (LEAN 2.5.0.0.18057, cloud backtest, history[Fundamental](symbol, 2005-01-01, 2024-01-01))
| ticker |
rows |
security_reference.share_class_status |
security_reference.delisting_date |
| CELG (delisted 2019-11-21) |
3748 (2005-01-04 .. 2019-11-21) |
D on every row |
2019-11-26 on every row, including 2005 |
| TWTR (delisted 2022-10-28) |
2260 (2013-11-08 .. 2022-10-29) |
D on every row |
2022-10-28 on every row |
| AAPL |
4781 |
A on every row |
none |
| XOM |
4781 |
A on every row |
none |
The explicit FundamentalService.Get<string>(date, sid, FundamentalProperty.SecurityReference_ShareClassStatus) route returns the same: CELG is D on 2010-06-30, 2015-06-30, 2019-06-28 and 2019-11-20, and null from 2019-11-21 (no snapshot after the last trading day). delisting_reason, company_status, is_primary_share (TWTR: False on every row) and exchange_id behave the same way. This is consistent with the earlier finding that the sector / industry codes are today's assignment backfilled across history (see QuantConnect/Lean#9777's discussion of provenance).
Field definition in Lean: Common/Data/Fundamental/Generated/SecurityReference.cs (ShareClassStatus, Morningstar DataId 1014: "(A) Active, (D) Deactive, (I) Inactive, or (O) Obsolete").
Proposed documentation change
On 03 Writing Algorithms/14 Datasets/04 Morningstar/01 US Fundamental Data (Introduction and/or Data Point Attributes), add a short note along the lines of:
The financial statement and earnings-report values are as-originally-reported per snapshot date. The reference and classification fields (security_reference.share_class_status, delisting_date, delisting_reason, company_status, is_primary_share, exchange_id, and the asset_classification sector/industry codes) are Morningstar's current values and are the same on every historical snapshot. Do not use them to decide historical membership; a company delisted in 2019 reads D with its 2019 delisting date on its 2005 rows. For point-in-time membership use the daily universe itself (history[Fundamentals] / universe_history, which include later-delisted companies) and the Delisting events for cessation dates.
Optionally cross-link this from the Fundamental Universes page, which is where users build formation-date selections.
Open question for the data team
Whether Morningstar supplies effective-dated status / delisting history that could be written into the daily files instead. If so, this becomes a data change rather than a docs note.
Intercom conversation: 215475854522100.
Summary
The US Fundamental Data (Morningstar) pages do not state that the
SecurityReferencestatus fields are current reference values stamped onto every historical daily snapshot, not point-in-time values. Users qualifying point-in-time universe selection readshare_class_statuson a historical date and treatD(Deactive) as "delisted as of that date", which silently excludes every later-delisted company at formation time.What we observed (LEAN 2.5.0.0.18057, cloud backtest,
history[Fundamental](symbol, 2005-01-01, 2024-01-01))security_reference.share_class_statussecurity_reference.delisting_dateDon every rowDon every rowAon every rowAon every rowThe explicit
FundamentalService.Get<string>(date, sid, FundamentalProperty.SecurityReference_ShareClassStatus)route returns the same: CELG isDon 2010-06-30, 2015-06-30, 2019-06-28 and 2019-11-20, andnullfrom 2019-11-21 (no snapshot after the last trading day).delisting_reason,company_status,is_primary_share(TWTR:Falseon every row) andexchange_idbehave the same way. This is consistent with the earlier finding that the sector / industry codes are today's assignment backfilled across history (see QuantConnect/Lean#9777's discussion of provenance).Field definition in Lean:
Common/Data/Fundamental/Generated/SecurityReference.cs(ShareClassStatus, Morningstar DataId 1014: "(A) Active, (D) Deactive, (I) Inactive, or (O) Obsolete").Proposed documentation change
On
03 Writing Algorithms/14 Datasets/04 Morningstar/01 US Fundamental Data(Introduction and/or Data Point Attributes), add a short note along the lines of:Optionally cross-link this from the Fundamental Universes page, which is where users build formation-date selections.
Open question for the data team
Whether Morningstar supplies effective-dated status / delisting history that could be written into the daily files instead. If so, this becomes a data change rather than a docs note.
Intercom conversation: 215475854522100.