This repo contains the code used to produce the GEHM dataset. It was ran using Google Colab, on varying server requirement configurations (specified within the notebooks).
The user needs to specify target source video and audio destination(s) on Google Drive, which automatically attaches to the runtime after running the notebooks.
whisper_x.ipynb produces PRAAT-compatible files based on input audio. It heavily relies on WhisperX pipeline, which is based on openAI's Whisper.
openpose_keypoint_extraction.ipynd extracts OpenPose features and saves them as JSON files in separate folders. A generalized working example of running OpenPose on Google Colab is available here.
Dataset available separately.