Overview
Enhance the SDS YAML configuration and metadata handling to support both Object Detection and Segmentation ML models.
Object Detection Model Support
The detection models will provide bounding box information for each detected object.
Sample Detection Model Output
{
"frame_id": 123,
"detections": [
{
"class_id": 0,
"label": "person",
"confidence": 0.95,
"bounding_box": {
"x1": 120,
"y1": 80,
"x2": 320,
"y2": 450
}
},
{
"class_id": 2,
"label": "car",
"confidence": 0.89,
"bounding_box": {
"x1": 500,
"y1": 200,
"x2": 850,
"y2": 600
}
}
]
}
For every detected object, the model returns:
Bounding Box Coordinates
(x1, y1) – Top-left corner of the bounding box
(x2, y2) – Bottom-right corner of the bounding box
Confidence Score
Indicates the model's confidence in the prediction.
Notes
A single frame may contain N detected objects.
The model will return bounding box metadata for each detected object.
If no objects are detected, the output metadata will be empty (or contain zero detections).
The 2D coordinates are used to draw rectangular bounding boxes around detected objects.
The confidence score represents the reliability of the prediction.
Segmentation Model Support
Segmentation models provide both object detection and pixel-level object masks.
Sample Segmentation Model Output
For every detected object, the model returns:
Bounding box information.
Confidence score.
Segmentation mask representing the object's pixels.
Unlike detection models, segmentation models identify the exact pixels belonging to each detected object rather than only providing a rectangular bounding box.
{
"frame_id": 123,
"segmentations": [
{
"class_id": 0,
"label": "person",
"bounding_box": {
"x1": 120,
"y1": 80,
"x2": 320,
"y2": 450
},
"confidence": 0.95,
"mask": "encoded_mask_data"
},
{
"class_id": 0,
"label": "car",
"bounding_box": {
"x1": 500,
"y1": 200,
"x2": 850,
"y2": 600
},
"confidence": 0.89,
"mask": "encoded_mask_data"
}
]
}
Note:
The segmentation mask size and pixel mapping are dependent on the model's input dimensions. For example, a model with an input shape of 640×640 will generate masks corresponding to that resolution, while a model with an input shape of 1024×1024 will produce masks at the higher resolution. Therefore, the number of masked pixels, mask granularity, and mask representation may vary across segmentation models based on their configured input shape and output format. SDS should support variable-sized masks and should not assume a fixed mask resolution. Bounding boxes are optional for segmentation models and their usage depends on the model's training configuration.
Overview
Enhance the SDS YAML configuration and metadata handling to support both Object Detection and Segmentation ML models.
Object Detection Model Support
The detection models will provide bounding box information for each detected object.
Sample Detection Model Output
For every detected object, the model returns:
Bounding Box Coordinates
(x1, y1) – Top-left corner of the bounding box
(x2, y2) – Bottom-right corner of the bounding box
Confidence Score
Indicates the model's confidence in the prediction.
Notes
A single frame may contain N detected objects.
The model will return bounding box metadata for each detected object.
If no objects are detected, the output metadata will be empty (or contain zero detections).
The 2D coordinates are used to draw rectangular bounding boxes around detected objects.
The confidence score represents the reliability of the prediction.
Segmentation Model Support
Segmentation models provide both object detection and pixel-level object masks.
Sample Segmentation Model Output
For every detected object, the model returns:
Bounding box information.
Confidence score.
Segmentation mask representing the object's pixels.
Unlike detection models, segmentation models identify the exact pixels belonging to each detected object rather than only providing a rectangular bounding box.
Note:
The segmentation mask size and pixel mapping are dependent on the model's input dimensions. For example, a model with an input shape of 640×640 will generate masks corresponding to that resolution, while a model with an input shape of 1024×1024 will produce masks at the higher resolution. Therefore, the number of masked pixels, mask granularity, and mask representation may vary across segmentation models based on their configured input shape and output format. SDS should support variable-sized masks and should not assume a fixed mask resolution. Bounding boxes are optional for segmentation models and their usage depends on the model's training configuration.