Plugin: Implement batch processing in the age and gender detection plugin.

[prasannals]

The age and gender detection plugin is in the src/age_and_gender_detection folder. Currently, the images are processed sequentially. Implement batch processing of images. You can take the batch size as a parameter from the user, with some reasonable defaults.

[nb950]

this is about use of dynamic_axes during onnx model create/export . talk to prasanna to be sure .

[prasannals]

Thanks, jai. I think that is spot on. Here is how I would approach this task -

1. Explore the layers of the ONNX model being used. Consider the input dimensions of the input to each layer of the model. ONNX models can be visualized in various ways, including using this webapp: https://netron.app/
2. The first dimension (batch size) of the tensor going into the various layers should not be fixed. If it is fixed, the model must be converted to support dynamic batch sizes.
3. Consider all the layers that need to have the dyanmic batch sizes. For example, I have attached the visualization of the age_googlenet.onnx from the age and gender detection plugin. This has a fixed batch size of 1. We need to change the batch size of the onnx models coming into the Conv layer, Relu, MaxPool, Dropout etc.

4. Once the model is converted, test it with various batch sizes. Measure and record performance gains (time) vs using just a batch of size 1 (current approach).
5. Perform validation on some dataset to ensure that the performance of the model, in terms of its ability to perform the task it is trained to do (age detection, face detection, etc.) has not changed. The performance measure (accuracy, F1, etc. ) must remain the same or very similar for the original model and the new model that supports batch processing.
6. Integrate the model into RescueBox, change code where necessary. Considering getting the batch size from the UI (possibly as a "parameter" to the plugin models) while setting a reasonable default that works across lots of machines.