Improve parallel performance - #177
Open
dbrakenhoff wants to merge 10 commits into
Open
Conversation
For semi-confined aquifers (ilap == 0) in steady models, the vcontour_array method was using aq.z for z-coordinates which includes all layer boundaries (leaky and aquifer), but the arr array only had values for aquifer layers. This caused dimension mismatches or incorrect contour plots. The fix builds a z array that matches the structure of the arr array: - z[0] = top boundary (where head = hstar) - z[1], z[3], ... = aquifer tops - z[2], z[4], ... = aquifer bottoms This ensures dimensions always match and the head values are properly aligned with the z-coordinates. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- send one model copy per worker - modify disvec signature to match head - parallel progressbars updates per completed chunk - add disvec_array method
- use passed in x1 if x1 is finite, otherwise compute x1 from x2 - some offset
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes memory usage issues related to parallel processing. Before the entire model class was sent over to each process for each task, this quickly grew the memory used to significant amounts, especially, for larger models and lots of points at which to compute the heads.
This is now fixed by setting up a single model on each worker, then performing all calculations using those copied models.
Some additional changes:
tqdmis now a required dependency.