Skip to content

Stop using multiprocessing for fitting dispersion models earlier #142

@le-ander

Description

@le-ander

From experience, I can say that the multiprocessing overhead when fitting dispersion models seems to be a lot larger than the code is currently written for.

ie. fitting the last 50 or so models still takes a long time and as soon as multiprocessing is switched off for the last models, things become a lot faster. maybe multiprocessing could be only used when there are more than 10x as many genes left than processors here:

if nproc > 1 and len(idx_update) > nproc:

So something like:
if nproc > 1 and len(idx_update) > 10 * nproc:

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions