Rule Execution Benchmark Suite

[dhanur-sharma]

Description

The execution speed increase needs a benchmarking suite to quantify gains.

Implementation Considerations

Deliverable

• Rule execution benchmarking

Dependencies

depends on

[bishwaspraveen]

Running benchmark for Title Pattern...

================================================================================
Detailed Profile for Title Pattern
================================================================================

Dataset Size: 10

APPLY OPERATION
Function Name                                      | Calls    | Total Time   | Cumulative  
--------------------------------------------------------------------------------------
/app/sde_collections/models/delta_patterns.py:477(apply) | 1        | 0.0000       | 0.5870      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:383(__iter__) | 4        | 0.0000       | 0.5800      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1879(_fetch_all) | 5        | 0.0000       | 0.5800      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:1532(execute_sql) | 4        | 0.0000       | 0.5790      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:85(__iter__) | 3        | 0.0000       | 0.5790      
/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py:66(execute) | 4        | 0.0000       | 0.5720      
/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py:76(_execute_with_wrappers) | 4        | 0.0000       | 0.5720      
/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py:82(_execute) | 4        | 0.0000       | 0.5720      
{method 'execute' of 'psycopg2.extensions.cursor' objects} | 4        | 0.5720       | 0.5720      
/app/sde_collections/models/delta_patterns.py:109(update_affected_delta_urls_list) | 1        | 0.0000       | 0.0080      
/usr/local/lib/python3.10/site-packages/django/db/models/manager.py:85(manager_method) | 15/5     | 0.0000       | 0.0070      
/usr/local/lib/python3.10/site-packages/django/db/models/fields/related_descriptors.py:1206(set) | 1        | 0.0000       | 0.0060      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:725(as_sql) | 17/4     | 0.0000       | 0.0060      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:78(pre_sql_setup) | 17/5     | 0.0000       | 0.0060      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:541(compile) | 176/58   | 0.0000       | 0.0040      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:70(setup_query) | 17/5     | 0.0000       | 0.0040      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:229(get_select) | 17/5     | 0.0000       | 0.0040      
/app/sde_collections/models/delta_url.py:60(get_queryset) | 3        | 0.0000       | 0.0030      
/app/sde_collections/models/delta_url.py:13(with_exclusion_status) | 3        | 0.0000       | 0.0030      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/query.py:1557(_add_q) | 21       | 0.0000       | 0.0030      

UNAPPLY OPERATION
Function Name                                      | Calls    | Total Time   | Cumulative  
--------------------------------------------------------------------------------------
/app/sde_collections/models/delta_patterns.py:555(unapply) | 1        | 0.0000       | 0.0130      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:1532(execute_sql) | 6        | 0.0000       | 0.0060      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:383(__iter__) | 2        | 0.0000       | 0.0050      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1879(_fetch_all) | 2        | 0.0000       | 0.0050      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:85(__iter__) | 2        | 0.0000       | 0.0050      
/usr/local/lib/python3.10/site-packages/django/db/models/fields/related_descriptors.py:1173(clear) | 2        | 0.0000       | 0.0040      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:725(as_sql) | 10/2     | 0.0000       | 0.0030      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py:78(pre_sql_setup) | 10/2     | 0.0000       | 0.0030      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1124(delete) | 4        | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py:66(execute) | 6        | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py:76(_execute_with_wrappers) | 6        | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py:82(_execute) | 6        | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/query.py:1557(_add_q) | 22/20    | 0.0000       | 0.0020      
{method 'execute' of 'psycopg2.extensions.cursor' objects} | 6        | 0.0020       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1457(_filter_or_exclude_inplace) | 16       | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/query.py:1532(add_q) | 16       | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1584(annotate) | 4        | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/sql/query.py:1130(add_annotation) | 20       | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1446(_filter_or_exclude) | 16       | 0.0000       | 0.0020      
/usr/local/lib/python3.10/site-packages/django/db/models/query.py:1599(_annotate) | 4        | 0.0000       | 0.0020    

[bishwaspraveen]

================================================================================
COMPARATIVE SUMMARY
================================================================================

Dataset Size: 10
Pattern Type         | Operation | Total Functions | Total Time   | DB Queries
--------------------------------------------------------------------------------
DeltaExcludePattern  | Apply    | 486             | 0.5960       | 0         
                     | Unapply  | 486             | 0.0020       | 0         
DeltaIncludePattern  | Apply    | 486             | 0.5810       | 0         
                     | Unapply  | 481             | 0.0010       | 0         
DeltaDivisionPattern | Apply    | 488             | 0.5890       | 0         
                     | Unapply  | 497             | 0.0020       | 0         
DeltaDocumentTypePattern | Apply    | 488             | 0.5720       | 0         
                     | Unapply  | 497             | 0.0020       | 0         
DeltaTitlePattern    | Apply    | 486             | 0.5720       | 0         
                     | Unapply  | 504             | 0.0020       | 0         

Dataset Size: 100
Pattern Type         | Operation | Total Functions | Total Time   | DB Queries
--------------------------------------------------------------------------------
DeltaExcludePattern  | Apply    | 486             | 0.5780       | 0         
                     | Unapply  | 473             | 0.0020       | 0         
DeltaIncludePattern  | Apply    | 486             | 0.5900       | 0         
                     | Unapply  | 473             | 0.0010       | 0         
DeltaDivisionPattern | Apply    | 488             | 0.5840       | 0         
                     | Unapply  | 483             | 0.0020       | 0         
DeltaDocumentTypePattern | Apply    | 488             | 0.5710       | 0         
                     | Unapply  | 483             | 0.0030       | 0         
DeltaTitlePattern    | Apply    | 486             | 0.5660       | 0         
                     | Unapply  | 482             | 0.0020       | 0         

Dataset Size: 1000
Pattern Type         | Operation | Total Functions | Total Time   | DB Queries
--------------------------------------------------------------------------------
DeltaExcludePattern  | Apply    | 486             | 0.5720       | 0         
                     | Unapply  | 473             | 0.0020       | 0         
DeltaIncludePattern  | Apply    | 486             | 0.5670       | 0         
                     | Unapply  | 473             | 0.0020       | 0         
DeltaDivisionPattern | Apply    | 488             | 0.6130       | 0         
                     | Unapply  | 483             | 0.0020       | 0         
DeltaDocumentTypePattern | Apply    | 488             | 0.5710       | 0         
                     | Unapply  | 483             | 0.0010       | 0         
DeltaTitlePattern    | Apply    | 486             | 0.5760       | 0         
                     | Unapply  | 482             | 0.0020       | 0         

Dataset Size: 10000
Pattern Type         | Operation | Total Functions | Total Time   | DB Queries
--------------------------------------------------------------------------------
DeltaExcludePattern  | Apply    | 486             | 0.5810       | 0         
                     | Unapply  | 473             | 0.0010       | 0         
DeltaIncludePattern  | Apply    | 486             | 0.5900       | 0         
                     | Unapply  | 473             | 0.0010       | 0         
DeltaDivisionPattern | Apply    | 488             | 0.5730       | 0         
                     | Unapply  | 483             | 0.0010       | 0         
DeltaDocumentTypePattern | Apply    | 488             | 0.5650       | 0         
                     | Unapply  | 483             | 0.0020       | 0         
DeltaTitlePattern    | Apply    | 486             | 0.5740       | 0         
                     | Unapply  | 482             | 0.0020       | 0         

Dataset Size: 100000
Pattern Type         | Operation | Total Functions | Total Time   | DB Queries
--------------------------------------------------------------------------------
DeltaExcludePattern  | Apply    | 486             | 0.5850       | 0         
                     | Unapply  | 473             | 0.0020       | 0         
DeltaIncludePattern  | Apply    | 486             | 0.5820       | 0         
                     | Unapply  | 473             | 0.0020       | 0         
DeltaDivisionPattern | Apply    | 488             | 0.5850       | 0         
                     | Unapply  | 483             | 0.0010       | 0         
DeltaDocumentTypePattern | Apply    | 488             | 0.5770       | 0         
                     | Unapply  | 483             | 0.0020       | 0         
DeltaTitlePattern    | Apply    | 486             | 0.5770       | 0         
                     | Unapply  | 482             | 0.0020       | 0         

Dataset Size: 200000
Pattern Type         | Operation | Total Functions | Total Time   | DB Queries
--------------------------------------------------------------------------------
DeltaExcludePattern  | Apply    | 486             | 0.5890       | 0         
                     | Unapply  | 473             | 0.0020       | 0         
DeltaIncludePattern  | Apply    | 486             | 0.5760       | 0         
                     | Unapply  | 473             | 0.0010       | 0         
DeltaDivisionPattern | Apply    | 488             | 0.5900       | 0         
                     | Unapply  | 483             | 0.0020       | 0         
DeltaDocumentTypePattern | Apply    | 488             | 0.5740       | 0         
                     | Unapply  | 483             | 0.0020       | 0         
DeltaTitlePattern    | Apply    | 486             | 0.5920       | 0         
                     | Unapply  | 482             | 0.0020       | 0         
.
----------------------------------------------------------------------
Ran 1 test in 263.684s