Fix scraper reliability and consolidated output quality

[manuelgrabmayer]

At the moment, the scraper does not reliably extract the required data, and the current pipeline cannot be considered functional. This needs to be fixed before any consolidation, enrichment, or downstream processing can be trusted.