Our current computation of the AUC is wrong, (see tests on branch fixes/general)
We have been computing the AUC using the following formula:
$$
AUC_{ratio} = \frac{AUC_{Real} - AUC_{Optimal}}{AUC_{Optimal} + 10^{-16}}
$$
This is conceptualy, and mathematically, wrong for what we are trying to compute.
The method auc_ratio is making use of a method inside of utils called auc_ratio_df which is calling to _auc, a method that is supposed to compute the $A_{Real}$.
This auxiliary function is making use of:
area_real = np.trapezoid(df[ColumnNames.Y], df[ColumnNames.X])
Which is wrong, as this will give, for example, from a displacement from (0,0) to (10,10) an area of 50.
$$
\int_0^{10} x dx = 50
$$
This metric is senstive to coordinate orientation.
´In the same way, the optimal area, computed as:
area_optimal = np.trapezoid(y_opt, x_opt)
In _auc_optimal is incorrect for the same reason.
The auc_perp is correct, as it is computed as:
auc_perp = np.trapezoid(dists, s_grid)
Which is the geometric definition for the AUC.
$$
AUC = \frac{1}{2} \int d(s) ds
$$
Where d(s) is the perpendicular distance between the ideal path and the user path, in this case computed for each point, which makes it translation and rotation invariant.
This metric now gives values starting from 0 if the line is straight and higher values as the trajectory becomes more deviated.
The code nevertheless, has complexity $O(n^2)$, I think it could be lowered to O(n), but this should call for another Issue.
Link to ChatGPT conversation on this topic.
Our current computation of the AUC is wrong, (see tests on branch fixes/general)
We have been computing the AUC using the following formula:
This is conceptualy, and mathematically, wrong for what we are trying to compute.
The method$A_{Real}$ .
auc_ratiois making use of a method inside of utils calledauc_ratio_dfwhich is calling to_auc, a method that is supposed to compute theThis auxiliary function is making use of:
Which is wrong, as this will give, for example, from a displacement from (0,0) to (10,10) an area of 50.
This metric is senstive to coordinate orientation.
´In the same way, the optimal area, computed as:
In _auc_optimal is incorrect for the same reason.
The auc_perp is correct, as it is computed as:
Which is the geometric definition for the AUC.
Where d(s) is the perpendicular distance between the ideal path and the user path, in this case computed for each point, which makes it translation and rotation invariant.
This metric now gives values starting from 0 if the line is straight and higher values as the trajectory becomes more deviated.
The code nevertheless, has complexity$O(n^2)$ , I think it could be lowered to O(n), but this should call for another Issue.
Link to ChatGPT conversation on this topic.