Skip to content

mpi: Fixup code generation for Bundles#2581

Closed
FabioLuporini wants to merge 13 commits intomainfrom
hotfix-parlang-lowering-2
Closed

mpi: Fixup code generation for Bundles#2581
FabioLuporini wants to merge 13 commits intomainfrom
hotfix-parlang-lowering-2

Conversation

@FabioLuporini
Copy link
Copy Markdown
Contributor

All tests are in PRO ... because that's the only place where we generate Bundles

@mloubout mloubout added the MPI mpi-related label Apr 10, 2025
@FabioLuporini FabioLuporini force-pushed the hotfix-parlang-lowering-2 branch from dbd4524 to 5b59cec Compare April 10, 2025 14:29
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 10, 2025

Codecov Report

Attention: Patch coverage is 28.46975% with 201 lines in your changes missing coverage. Please review.

Project coverage is 45.91%. Comparing base (19cfcb6) to head (a50a1d4).

Files with missing lines Patch % Lines
devito/passes/iet/definitions.py 7.57% 61 Missing ⚠️
devito/passes/iet/engine.py 15.21% 39 Missing ⚠️
tests/test_ir.py 10.52% 17 Missing ⚠️
devito/mpi/routines.py 5.88% 16 Missing ⚠️
tests/test_buffering.py 0.00% 15 Missing ⚠️
devito/types/array.py 65.78% 12 Missing and 1 partial ⚠️
tests/test_linearize.py 0.00% 6 Missing ⚠️
devito/mpi/halo_scheme.py 58.33% 5 Missing ⚠️
devito/passes/iet/languages/C.py 37.50% 5 Missing ⚠️
devito/passes/iet/languages/CXX.py 37.50% 5 Missing ⚠️
... and 7 more

❗ There is a different number of reports uploaded between BASE (19cfcb6) and HEAD (a50a1d4). Click for more details.

HEAD has 14 uploads less than BASE
Flag BASE (19cfcb6) HEAD (a50a1d4)
16 4
pytest-gpu-nvc-nvidiaX 1 0
pytest-gpu-aomp-amdgpuX 1 0
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2581       +/-   ##
===========================================
- Coverage   91.96%   45.91%   -46.05%     
===========================================
  Files         244      245        +1     
  Lines       47797    47974      +177     
  Branches     4207     4219       +12     
===========================================
- Hits        43957    22029    -21928     
- Misses       3165    25026    +21861     
- Partials      675      919      +244     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX ?
pytest-gpu-nvc-nvidiaX ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread devito/mpi/routines.py
try:
name = "bag_%s" % "".join(f.name for f in candidates)
bag = Bag(name=name, components=candidates)
name = "bag_%s" % "".join(f.name for f in components)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fstring? Might be a rare occasion where it's less neat tbh

# Vector types
return len(dtype)
dlen = len(dtype)
if 'padding0' in dtype.fields:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to search for this hardcoded string? Looks a bit hacky

@FabioLuporini FabioLuporini force-pushed the hotfix-parlang-lowering-2 branch from 9dbaf1f to 4a959f6 Compare April 12, 2025 08:59
@FabioLuporini FabioLuporini force-pushed the hotfix-parlang-lowering-2 branch from 462e604 to f3d7aca Compare April 15, 2025 08:16
Comment thread devito/ir/iet/nodes.py
return printAST(self)

@property
def view_cir(self):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is like a prettier print I guess

)

@property
@cached_property
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the motivation for this?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It went from simply being an alias for an attribute of an attribute, to having internal logic etc

Comment thread devito/mpi/routines.py
except ValueError:
for i in candidates:
for i in components:
name = "bag_%s" % i.name
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fstring

Comment thread devito/mpi/routines.py
@@ -363,10 +372,17 @@ def _make_copy(self, f, hse, key, swap=False):
else:
swap = lambda i, j: (j, i)
name = 'scatter%s' % key
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

bfmap = map_buffered_functions(clusters, key)

# Sort for deterministic code generation
bfmap = {i: bfmap[i] for i in sorted(bfmap, key=lambda i: i.name)}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this sorting go into map_buffered_functions instead?

# NOTE: the `arity` is calculated such as `sizeof(float3)/sizeof(float)`
# for portability reasons (since we don't know the size of compound
# types a priori)
arity_param = Symbol(name='arity', dtype=size_t)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you can't have more than one of these per kernel, so no need to go through the symbol registry?

"""
decl = Definition(obj)

# NOTE: the `arity` is calculated such as `sizeof(float3)/sizeof(float)`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is largely identical to that in _alloc_mapped_array_on_high_bw_mem. Would it be worth pulling this out into a utility function?

call1 = call._rebuild(arguments=args)
efunc1 = Transformer({call: call1}).visit(efunc)

#TODO Propagate index through call stack until haloupdate...
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: probably only temp comments, but not Pep8 compliant

efuncs = update_call_stack(call, efuncs, _filter, cascade=True)

# Store the new Callables
efuncs.up
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do?

@FabioLuporini
Copy link
Copy Markdown
Contributor Author

closing this, and reopening soon from another branch

@FabioLuporini FabioLuporini deleted the hotfix-parlang-lowering-2 branch April 22, 2025 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

MPI mpi-related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants