Enable per-class AP metric

[mzweilin]

          It would be nice to flatten so we get per class AP numbers. This doesn't work right now though and just returns the average 🙃

Originally posted by @dxoigmn in #192 (comment)