Project Idea: AI-Based Swimming Analytics System

[Honey7505]

Dear Contributors,
I'm currently working on an AI-Based Swimming Analytics System as a Proof of Concept (POC), and I'd greatly appreciate your guidance on the overall architecture and model selection. My goal is to build a system capable of analyzing swimming videos and generating useful performance metrics.

The objective is to automatically analyze a swimmer from a video and provide:

• Stroke Type Detection (Freestyle, Backstroke, Breaststroke, Butterfly)
• Stroke Count
• Lap Count
• Lap-wise Performance
• Swimming Speed
• Lap Time
• Technique Analysis (future enhancement)

Current Technology Stack

• Programming Language: Python
• Computer Vision: OpenCV
• Deep Learning Framework: PyTorch
• Object Detection: YOLO11
• Pose Estimation: YOLO Pose (experimented), planning to evaluate RTMPose
• Temporal Model: CNN + LSTM (initial experiments)
• Development Environment: Google Colab and Jupyter Notebook
• Training Framework: Ultralytics YOLO

Current Approach

I have experimented with multiple approaches:

1. CNN + LSAT

   • Extracted frames from swimming videos.
   • Used a CNN to extract frame-level features.
   • Trained an LSTM for stroke classification.

2. YOLO Pose

   • Used pose estimation to extract body keypoints.
   • Attempted stroke recognition using body movements.

3. YOLO Detection

   • Trained a custom YOLO model on swimming stroke classes.
   • Successfully detected swimmers but stroke classification accuracy was inconsistent.

Challenges Faced

• Underwater videos make pose estimation difficult because many body joints are occluded or not clearly visible.
• YOLO Pose struggles with underwater footage since it is primarily trained on general human pose datasets.
• CNN + LSTM achieved limited accuracy due to the relatively small dataset.
• YOLO detection can classify a single frame, but swimming stroke recognition is fundamentally a temporal problem that requires analyzing movement across multiple frames.
• However, I believe the dataset size is still insufficient for robust temporal learning.

Planned Pipeline

Video
↓
YOLO Swimmer Detection
↓
Swimmer Tracking
↓
RTMPose (Pose Estimation)
↓
Temporal Model (LSTM or another sequence model)
↓
Stroke Type Detection
↓
Swimming Analytics (Stroke Count, Lap Count, Speed, Lap Time)

I'm looking for guidance on whether this architecture is appropriate for a Proof of Concept, or if there is a better approach for reliable underwater swimming analysis. Any recommendations regarding temporal models, pose estimation, dataset preparation, or existing research/projects would be extremely helpful.