Problem with processing many gt3x files

[josefheidler]

Hey,

I'm having trouble processing a batch of files. They're failing silently during the process, and I only get an error message when I attempt to export the summary.

Any idea why the logs might be empty or what’s causing the export hitches?

Welcome to the actimetric R package!
actimetric version 0.1.6
actimetricModels version 0.1.1

If you experiment any technical issue or wish to contribute to actimetric,
please contact Jairo at jairo@jhmigueles.com or Matthew at matthew.ahmadi@sydney.edu.au

_______________________________________________________________________________________________

Classifier: Preschool hip random forest free living lag-lead

_______________________________________________________________________________________________

Reading /work/Other/test_files_actimetric/41674_4 (2017-03-01).gt3x ...
File Size: 32.2 MB



----


Calculating and exporting the summary statistics per day and per recording...
Error in `[<-.data.frame`(`*tmp*`, , ci, value = "Preschool hip random forest free living lag-lead") : 
  replacement has 1 row, data has 0
5.
stop(sprintf(ngettext(N, "replacement has %d row, data has %d",
"replacement has %d rows, data has %d"), N, n), domain = NA)
4.
`[<-.data.frame`(`*tmp*`, , ci, value = "Preschool hip random forest free living lag-lead")
3.
`[<-`(`*tmp*`, , ci, value = "Preschool hip random forest free living lag-lead")
2.
aggregate_per_date(tsDir = tsDir, epoch = epoch, classifier = classifier,
classes = classes, boutdur = boutdur, boutcriter = boutcriter,
boutmaxgap = boutmaxgap, visualreport = visualreport)
1.
runActimetric(input_directory = "/work/Other/test_files_actimetric",
output_directory = "/work/Other/out_actimetric", studyname = "test",
classifier = "Preschool Hip Random Forest Free Living Lag-Lead",
do.sleep = FALSE, do.enmo = FALSE, do.actilifecounts = FALSE, ...

Test file: https://1drv.ms/u/c/06AFF54AD2EAE6D0/IQDXxMnwmc5GR4hnWO9WqdkBAYBW2q9s5gDYuErG76xzvZM?e=QB7w7o

[jhmigueles]

Hi @josefheidler,

Thank you for reporting this and providing the test file.

I have replicated the issue. The failure and subsequent crash during export occur because the file contains large gaps of missing data, likely due to the ActiGraph Idle Sleep Mode being enabled.

The Fix
The issue is within the ReadAndCalibrate function. I have proposed a fix here.

Please, test this and let me know if it works before proceeding with a pull request:

install.packages("remotes") # only if not installed
remotes::install_github("PhysicalActivityOpenTools/actimetric", ref = "issue74_readingLargeGaps")

Then, run actimetric as usual.

[josefheidler]

Hi @jhmigueles,

Thanks, seems resolved. Still processing, but for those specific files the issue is gone.

One more thing, not critical, but when there's basically no data in a file, instead of skipping it and logging info, the process crashes. You then have to manually remove the file and restart, which is fine since you have the skip attribute.

If you want to debug that one too, see the test file: https://1drv.ms/u/c/06AFF54AD2EAE6D0/IQBLDKrK7zUVRI0BTPpseysUAV4mCqtf_-9YC4EeguqfTNY?e=sOwFjX

Thank you.