import format_width and format_digits for SAS

[belegdol]

Please read the README, particularly the known limitations section!

Describe the issue
Currently, meta.original_variable_types only contains the format names but not their widths. Looking at https://github.com/WizardMac/ReadStat/blob/05cadd8906f6cbd4ac22966d1f4b0ae2743ae8bf/src/sas/readstat_sas7bdat_read.c#L23-L24, it appears that readstat can read the latter.

To Reproduce
Import a sas dataset created with a format such as b8601da8. or 10.2.

File example
Not currently possible but any file should do.

Expected behavior
A clear and concise description of what you expected to happen.

Setup Information:
How did you install pyreadstat? pip
Platform (windows, macOS, linux, 32 or 64 bit): any
Python Version 3.11
Python Distribution (System, plain python, Anaconda): plain python
Using Virtualenv or condaenv? venv

[belegdol]

This could in theory make integer identification possible (#295). Saurfang (https://github.com/saurfang/spark-sas7bdat) does it by checking format width and precision:

• inferInt (Default: false)
  • Boolean: infer numeric columns with <=4 bytes, format width >0 and format precision =0, as Int
• inferLong (Default: false)
  • Boolean: infer numeric columns with <=8 bytes, format width >0 and format precision =0, as Long
• inferShort (Default: false)
  • Boolean: infer numeric columns with <=2 bytes, format width >0 and format precision =0, as Short

[belegdol]

At the first glance it looks like the following change might be sufficient:
WizardMac/ReadStat#361

[ofajardo]

thanks for the suggestion! Once the PR at Readstat gets approved, then the field you need will automatically appear in pyreadstat (once the Readstat sources are updated of course), or are there other changes needed?

[belegdol]

It does not look like any further changes are required. Here is how meta.original_variable_types looks like with the PR change included:

{'i': None, 'var1': '6', 'var2': '9.2', 'var3': 'BEST6', 'var4': 'BEST9.2'}

Without it with 1.3.3 it looks as follows:

{'var1': None, 'var2': None, 'var3': 'BEST6', 'var4': 'BEST9.2', 'i': None}

For the record, it appears that my original assumption on lengths not being read at all was incorrect - at least not entirely or it is dependent on how the SAS file in question has been created. I might need to investigate this further, but given the proprietary nature of SAS it is not exactly trivial.

[belegdol]

Lengths not being read should be fixed with WizardMac/ReadStat#364.

[ofajardo]

good, I wait for the update on Readstat, and then will update here

[belegdol]

Both PRs have been merged now.

[belegdol]

Fixed in 434e9fd.