Skip to content

Coffea casa analysis notebook (based on Zora Che notebook) - #36

Closed
oshadura wants to merge 11 commits into
TopEFT:masterfrom
oshadura:coffea-casa-analysis
Closed

Coffea casa analysis notebook (based on Zora Che notebook)#36
oshadura wants to merge 11 commits into
TopEFT:masterfrom
oshadura:coffea-casa-analysis

Conversation

@oshadura

Copy link
Copy Markdown

@bryates

bryates commented Jun 17, 2021

Copy link
Copy Markdown
Contributor

Instead of copying or moving the analysis folder, shortcuts might also work. I used a shortcut earlier to get binder working from a subdirectory. However, if everyone is onboard with moving analysis inside the topcoffea folder, that's fine with me.

@oshadura

oshadura commented Jun 17, 2021

Copy link
Copy Markdown
Author

@bryates thanks a lot for comment!
My problem is that I need to install topcoffea from my topcoffea fork git+https://github.com/oshadura/topcoffea.git@coffea-casa-analysis on remote Dask workers (not only notebook). For this I am using next class:

from dask.distributed import Client, Worker, WorkerPlugin
import os
from typing import List
class DependencyInstaller(WorkerPlugin):
    def __init__(self, dependencies: List[str]):
        self._depencendies = " ".join(f"'{dep}'" for dep in dependencies)
    def setup(self, worker: Worker):
        os.system(f"pip install {self._depencendies}")
dependency_installer = DependencyInstaller([
    "git+https://github.com/oshadura/topcoffea.git@coffea-casa-analysis","awkward==1.3.0"
])

Since I need to use coffea processor on dask workers as well, I need to ship it there, so it should be packaged :(

oshadura and others added 3 commits June 20, 2021 13:57
@oshadura

Copy link
Copy Markdown
Author

Most luckily will be replaced by TopEFT/topcoffea#41

@oshadura oshadura closed this Jul 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants