diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 2ec9102b6..723a9897f 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,10 +1,16 @@ # Release notes -## New in git-machete 3.44.2 +## New in git-machete 3.45.0 + +- added: `machete.github.retrieveOnlyMyPullRequests` and `machete.gitlab.retrieveOnlyMyMergeRequests` git config keys + that make PR/MR-listing commands download only the current user's open PRs/MRs instead of all of them, + which can speed up operations considerably in repositories/projects with many open PRs/MRs - fixed: `git machete discover` no longer produces a different branch tree depending on which worktree it is run from; the fresh-branch recency ranking now aggregates HEAD reflogs across all worktrees rather than only the current one (reported by @jasonoura, contributed by @earfman) - fixed: when run from a branch being slid out, `git machete slide-out` no longer checks out that branch's new parent if a child branch is going to be checked out right afterwards anyway for the rebase/merge +- fixed: pull/merge request-reading commands (`anno-prs`, `checkout-prs`, `retarget-pr`, `restack-pr`, `update-pr-descriptions` and their GitLab counterparts) + now address the base/target repository - the one that actually hosts the PR/MR - when the `machete.{github,gitlab}.base*` git config keys are set, rather than the head/source repository ## New in git-machete 3.44.1 diff --git a/docs/man/git-machete.1 b/docs/man/git-machete.1 index 116071d4e..381aeaf67 100644 --- a/docs/man/git-machete.1 +++ b/docs/man/git-machete.1 @@ -30,7 +30,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] .. .TH "GIT-MACHETE" "1" "" "" "git-machete" .SH NAME -git-machete \- git-machete 3.44.2 +git-machete \- git-machete 3.45.0 .sp git machete is a robust tool that \fBsimplifies your git workflows\fP\&. .sp @@ -332,7 +332,8 @@ For example, \fBgit config machete.github.domain git.example.org\fP The name of the git remote (as in \fBgit remote\fP) that git\-machete pushes the head branch to. Unless both \fBmachete.github.organization\fP and \fBmachete.github.repository\fP are set, this remote\(aqs URL is also inspected to derive the GitHub organization and repository that the pull request resides in. -The pull request is operated on through the GitHub API, which addresses that organization/repository rather than a git remote. +Unless the \fBmachete.github.base*\fP keys below point elsewhere, the pull request is operated on through the GitHub API, +which addresses that organization/repository rather than a git remote. By default (when this key is unset), if exactly one remote\(aqs URL corresponds to GitHub, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitHub. For example, \fBgit config machete.github.remote origin\fP @@ -348,6 +349,10 @@ For example, \fBgit config machete.github.repository git\-machete\fP .B \fBmachete.github.baseRemote\fP Like \fBmachete.github.remote\fP, but used to locate the base repository that the pull request targets, which may differ from the head repository (for example, the base in an upstream repository and the head in a fork). +Setting this key is what makes the PR\-reading/\-modifying commands (\fBanno\-prs\fP, \fBcheckout\-prs\fP, \fBretarget\-pr\fP, +\fBrestack\-pr\fP, \fBupdate\-pr\-descriptions\fP) address that base repository rather than the head one. +\fBcreate\-pr\fP does not need it: it infers the base repository from the base branch\(aqs tracking remote, +so it already targets the correct base (even one in a separate fork/upstream repository) even when this key is unset. Defaults to \fBmachete.github.remote\fP when unset. For example, \fBgit config machete.github.baseRemote upstream\fP .TP @@ -400,6 +405,25 @@ from the message body of the first unique commit of the branch, even if \fB\&.gi .UNINDENT .UNINDENT .TP +.B \fBmachete.github.retrieveOnlyMyPullRequests\fP +When set to \fBtrue\fP, commands that need to list open pull requests in the repository +(such as \fBgithub anno\-prs\fP, \fBgithub checkout\-prs\fP and \fBtraverse\fP with GitHub integration) +will only download open PRs authored by the current user (as determined from the GitHub API token), +instead of all open PRs in the repository. +.sp +This can speed up operations considerably in repositories with hundreds or thousands of open PRs, +at the cost of not being able to discover PRs opened by other users when traversing PR chains +(for example, when checking out an entire stack that includes PRs from multiple authors). +.sp +A valid GitHub API token is required when this key is set. +.sp +The \fB\-\-all\fP flag to \fBgithub checkout\-prs\fP and \fBgithub update\-pr\-descriptions\fP +still downloads all open PRs in the repository, regardless of this setting. +.sp +The \fB\-\-by=\fP flag to \fBgithub checkout\-prs\fP and \fBgithub update\-pr\-descriptions\fP +downloads open PRs authored by the given user directly (rather than filtering the current user\(aqs PRs), +so it keeps working for any author even when this key is set. +.TP .B \fBmachete.gitlab.{domain,remote,namespace,project,baseRemote,baseNamespace,baseProject}\fP .INDENT 7.0 .TP @@ -411,7 +435,8 @@ For example, \fBgit config machete.gitlab.domain git.example.org\fP The name of the git remote (as in \fBgit remote\fP) that git\-machete pushes the source branch to. Unless both \fBmachete.gitlab.namespace\fP and \fBmachete.gitlab.project\fP are set, this remote\(aqs URL is also inspected to derive the GitLab namespace and project that the merge request resides in. -The merge request is operated on through the GitLab API, which addresses that namespace/project rather than a git remote. +Unless the \fBmachete.gitlab.base*\fP keys below point elsewhere, the merge request is operated on through the GitLab API, +which addresses that namespace/project rather than a git remote. By default (when this key is unset), if exactly one remote\(aqs URL corresponds to GitLab, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitLab. For example, \fBgit config machete.gitlab.remote origin\fP @@ -427,6 +452,10 @@ For example, \fBgit config machete.gitlab.project hello\-world\fP .B \fBmachete.gitlab.baseRemote\fP Like \fBmachete.gitlab.remote\fP, but used to locate the target project that the merge request targets, which may differ from the source project (for example, the target in an upstream project and the source in a fork). +Setting this key is what makes the MR\-reading/\-modifying commands (\fBanno\-mrs\fP, \fBcheckout\-mrs\fP, \fBretarget\-mr\fP, +\fBrestack\-mr\fP, \fBupdate\-mr\-descriptions\fP) address that target project rather than the source one. +\fBcreate\-mr\fP does not need it: it infers the target project from the target branch\(aqs tracking remote, +so it already targets the correct project (even one in a separate fork/upstream project) even when this key is unset. Defaults to \fBmachete.gitlab.remote\fP when unset. For example, \fBgit config machete.gitlab.baseRemote upstream\fP .TP @@ -479,6 +508,25 @@ from the message body of the first unique commit of the branch, even if \fB\&.gi .UNINDENT .UNINDENT .TP +.B \fBmachete.gitlab.retrieveOnlyMyMergeRequests\fP +When set to \fBtrue\fP, commands that need to list open merge requests in the project +(such as \fBgitlab anno\-mrs\fP, \fBgitlab checkout\-mrs\fP and \fBtraverse\fP with GitLab integration) +will only download open MRs authored by the current user (as determined from the GitLab API token), +instead of all open MRs in the project. +.sp +This can speed up operations considerably in projects with hundreds or thousands of open MRs, +at the cost of not being able to discover MRs opened by other users when traversing MR chains +(for example, when checking out an entire stack that includes MRs from multiple authors). +.sp +A valid GitLab API token is required when this key is set. +.sp +The \fB\-\-all\fP flag to \fBgitlab checkout\-mrs\fP and \fBgitlab update\-mr\-descriptions\fP +still downloads all open MRs in the project, regardless of this setting. +.sp +The \fB\-\-by=\fP flag to \fBgitlab checkout\-mrs\fP and \fBgitlab update\-mr\-descriptions\fP +downloads open MRs authored by the given user directly (rather than filtering the current user\(aqs MRs), +so it keeps working for any author even when this key is set. +.TP .B \fBmachete.overrideForkPoint..to\fP Executing \fBgit machete fork\-point \-\-override\-to[\-parent|\-inferred|=] []\fP sets up a fork point override for \fB\fP\&. .sp @@ -1212,7 +1260,8 @@ For example, \fBgit config machete.github.domain git.example.org\fP The name of the git remote (as in \fBgit remote\fP) that git\-machete pushes the head branch to. Unless both \fBmachete.github.organization\fP and \fBmachete.github.repository\fP are set, this remote\(aqs URL is also inspected to derive the GitHub organization and repository that the pull request resides in. -The pull request is operated on through the GitHub API, which addresses that organization/repository rather than a git remote. +Unless the \fBmachete.github.base*\fP keys below point elsewhere, the pull request is operated on through the GitHub API, +which addresses that organization/repository rather than a git remote. By default (when this key is unset), if exactly one remote\(aqs URL corresponds to GitHub, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitHub. For example, \fBgit config machete.github.remote origin\fP @@ -1228,6 +1277,10 @@ For example, \fBgit config machete.github.repository git\-machete\fP .B \fBmachete.github.baseRemote\fP Like \fBmachete.github.remote\fP, but used to locate the base repository that the pull request targets, which may differ from the head repository (for example, the base in an upstream repository and the head in a fork). +Setting this key is what makes the PR\-reading/\-modifying commands (\fBanno\-prs\fP, \fBcheckout\-prs\fP, \fBretarget\-pr\fP, +\fBrestack\-pr\fP, \fBupdate\-pr\-descriptions\fP) address that base repository rather than the head one. +\fBcreate\-pr\fP does not need it: it infers the base repository from the base branch\(aqs tracking remote, +so it already targets the correct base (even one in a separate fork/upstream repository) even when this key is unset. Defaults to \fBmachete.github.remote\fP when unset. For example, \fBgit config machete.github.baseRemote upstream\fP .TP @@ -1279,6 +1332,25 @@ from the message body of the first unique commit of the branch, even if \fB\&.gi \fBnone\fP \-\-\- prepend no intro to the PR description at all .UNINDENT .UNINDENT +.TP +.B \fBmachete.github.retrieveOnlyMyPullRequests\fP (\fBanno\-prs\fP, \fBcheckout\-prs\fP and \fBupdate\-pr\-descriptions\fP) +When set to \fBtrue\fP, commands that need to list open pull requests in the repository +(such as \fBgithub anno\-prs\fP, \fBgithub checkout\-prs\fP and \fBtraverse\fP with GitHub integration) +will only download open PRs authored by the current user (as determined from the GitHub API token), +instead of all open PRs in the repository. +.sp +This can speed up operations considerably in repositories with hundreds or thousands of open PRs, +at the cost of not being able to discover PRs opened by other users when traversing PR chains +(for example, when checking out an entire stack that includes PRs from multiple authors). +.sp +A valid GitHub API token is required when this key is set. +.sp +The \fB\-\-all\fP flag to \fBgithub checkout\-prs\fP and \fBgithub update\-pr\-descriptions\fP +still downloads all open PRs in the repository, regardless of this setting. +.sp +The \fB\-\-by=\fP flag to \fBgithub checkout\-prs\fP and \fBgithub update\-pr\-descriptions\fP +downloads open PRs authored by the given user directly (rather than filtering the current user\(aqs PRs), +so it keeps working for any author even when this key is set. .UNINDENT .sp \fBEnvironment variables (all subcommands)\fP @@ -1507,7 +1579,8 @@ For example, \fBgit config machete.gitlab.domain git.example.org\fP The name of the git remote (as in \fBgit remote\fP) that git\-machete pushes the source branch to. Unless both \fBmachete.gitlab.namespace\fP and \fBmachete.gitlab.project\fP are set, this remote\(aqs URL is also inspected to derive the GitLab namespace and project that the merge request resides in. -The merge request is operated on through the GitLab API, which addresses that namespace/project rather than a git remote. +Unless the \fBmachete.gitlab.base*\fP keys below point elsewhere, the merge request is operated on through the GitLab API, +which addresses that namespace/project rather than a git remote. By default (when this key is unset), if exactly one remote\(aqs URL corresponds to GitLab, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitLab. For example, \fBgit config machete.gitlab.remote origin\fP @@ -1523,6 +1596,10 @@ For example, \fBgit config machete.gitlab.project hello\-world\fP .B \fBmachete.gitlab.baseRemote\fP Like \fBmachete.gitlab.remote\fP, but used to locate the target project that the merge request targets, which may differ from the source project (for example, the target in an upstream project and the source in a fork). +Setting this key is what makes the MR\-reading/\-modifying commands (\fBanno\-mrs\fP, \fBcheckout\-mrs\fP, \fBretarget\-mr\fP, +\fBrestack\-mr\fP, \fBupdate\-mr\-descriptions\fP) address that target project rather than the source one. +\fBcreate\-mr\fP does not need it: it infers the target project from the target branch\(aqs tracking remote, +so it already targets the correct project (even one in a separate fork/upstream project) even when this key is unset. Defaults to \fBmachete.gitlab.remote\fP when unset. For example, \fBgit config machete.gitlab.baseRemote upstream\fP .TP @@ -1574,6 +1651,25 @@ from the message body of the first unique commit of the branch, even if \fB\&.gi \fBnone\fP \-\-\- prepend no intro to the MR description at all .UNINDENT .UNINDENT +.TP +.B \fBmachete.gitlab.retrieveOnlyMyMergeRequests\fP (\fBanno\-mrs\fP, \fBcheckout\-mrs\fP and \fBupdate\-mr\-descriptions\fP) +When set to \fBtrue\fP, commands that need to list open merge requests in the project +(such as \fBgitlab anno\-mrs\fP, \fBgitlab checkout\-mrs\fP and \fBtraverse\fP with GitLab integration) +will only download open MRs authored by the current user (as determined from the GitLab API token), +instead of all open MRs in the project. +.sp +This can speed up operations considerably in projects with hundreds or thousands of open MRs, +at the cost of not being able to discover MRs opened by other users when traversing MR chains +(for example, when checking out an entire stack that includes MRs from multiple authors). +.sp +A valid GitLab API token is required when this key is set. +.sp +The \fB\-\-all\fP flag to \fBgitlab checkout\-mrs\fP and \fBgitlab update\-mr\-descriptions\fP +still downloads all open MRs in the project, regardless of this setting. +.sp +The \fB\-\-by=\fP flag to \fBgitlab checkout\-mrs\fP and \fBgitlab update\-mr\-descriptions\fP +downloads open MRs authored by the given user directly (rather than filtering the current user\(aqs MRs), +so it keeps working for any author even when this key is set. .UNINDENT .sp \fBEnvironment variables (all subcommands)\fP diff --git a/docs/source/cli/config.rst b/docs/source/cli/config.rst index a3fa905bc..3bf0c990c 100644 --- a/docs/source/cli/config.rst +++ b/docs/source/cli/config.rst @@ -20,6 +20,9 @@ Note: ``config`` is not a command as such, just a help topic (there is no ``git ``machete.github.prDescriptionIntroStyle`` .. include:: git-config-keys/github.prDescriptionIntroStyle.rst +``machete.github.retrieveOnlyMyPullRequests`` + .. include:: git-config-keys/github.retrieveOnlyMyPullRequests.rst + ``machete.gitlab.{domain,remote,namespace,project,baseRemote,baseNamespace,baseProject}`` .. include:: git-config-keys/gitlab.access.rst @@ -32,6 +35,9 @@ Note: ``config`` is not a command as such, just a help topic (there is no ``git ``machete.gitlab.mrDescriptionIntroStyle`` .. include:: git-config-keys/gitlab.mrDescriptionIntroStyle.rst +``machete.gitlab.retrieveOnlyMyMergeRequests`` + .. include:: git-config-keys/gitlab.retrieveOnlyMyMergeRequests.rst + ``machete.overrideForkPoint..to`` Executing ``git machete fork-point --override-to[-parent|-inferred|=] []`` sets up a fork point override for ````. diff --git a/docs/source/cli/github.rst b/docs/source/cli/github.rst index a492621c8..e7eca47f8 100644 --- a/docs/source/cli/github.rst +++ b/docs/source/cli/github.rst @@ -177,6 +177,9 @@ Create, check out and manage GitHub PRs while keeping them reflected in branch l ``machete.github.prDescriptionIntroStyle`` (``create-pr``, ``restack-pr`` and ``retarget-pr``) .. include:: git-config-keys/github.prDescriptionIntroStyle.rst +``machete.github.retrieveOnlyMyPullRequests`` (``anno-prs``, ``checkout-prs`` and ``update-pr-descriptions``) + .. include:: git-config-keys/github.retrieveOnlyMyPullRequests.rst + **Environment variables (all subcommands)** ``GITHUB_TOKEN`` diff --git a/docs/source/cli/gitlab.rst b/docs/source/cli/gitlab.rst index 754d37487..b39242c76 100644 --- a/docs/source/cli/gitlab.rst +++ b/docs/source/cli/gitlab.rst @@ -168,6 +168,9 @@ Create, check out and manage GitLab MRs while keeping them reflected in branch l ``machete.gitlab.mrDescriptionIntroStyle`` (``create-mr``, ``restack-mr`` and ``retarget-mr``) .. include:: git-config-keys/gitlab.mrDescriptionIntroStyle.rst +``machete.gitlab.retrieveOnlyMyMergeRequests`` (``anno-mrs``, ``checkout-mrs`` and ``update-mr-descriptions``) + .. include:: git-config-keys/gitlab.retrieveOnlyMyMergeRequests.rst + **Environment variables (all subcommands)** ``GITLAB_TOKEN`` diff --git a/docs/source/git-config-keys/github.access.rst b/docs/source/git-config-keys/github.access.rst index 33bc17c8c..7116a96b3 100644 --- a/docs/source/git-config-keys/github.access.rst +++ b/docs/source/git-config-keys/github.access.rst @@ -6,7 +6,8 @@ The name of the git remote (as in ``git remote``) that git-machete pushes the head branch to. Unless both ``machete.github.organization`` and ``machete.github.repository`` are set, this remote's URL is also inspected to derive the GitHub organization and repository that the pull request resides in. - The pull request is operated on through the GitHub API, which addresses that organization/repository rather than a git remote. + Unless the ``machete.github.base*`` keys below point elsewhere, the pull request is operated on through the GitHub API, + which addresses that organization/repository rather than a git remote. By default (when this key is unset), if exactly one remote's URL corresponds to GitHub, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitHub. For example, ``git config machete.github.remote origin`` @@ -22,6 +23,10 @@ ``machete.github.baseRemote`` Like ``machete.github.remote``, but used to locate the base repository that the pull request targets, which may differ from the head repository (for example, the base in an upstream repository and the head in a fork). + Setting this key is what makes the PR-reading/-modifying commands (``anno-prs``, ``checkout-prs``, ``retarget-pr``, + ``restack-pr``, ``update-pr-descriptions``) address that base repository rather than the head one. + ``create-pr`` does not need it: it infers the base repository from the base branch's tracking remote, + so it already targets the correct base (even one in a separate fork/upstream repository) even when this key is unset. Defaults to ``machete.github.remote`` when unset. For example, ``git config machete.github.baseRemote upstream`` diff --git a/docs/source/git-config-keys/github.retrieveOnlyMyPullRequests.rst b/docs/source/git-config-keys/github.retrieveOnlyMyPullRequests.rst new file mode 100644 index 000000000..4e6df041f --- /dev/null +++ b/docs/source/git-config-keys/github.retrieveOnlyMyPullRequests.rst @@ -0,0 +1,17 @@ +When set to ``true``, commands that need to list open pull requests in the repository +(such as ``github anno-prs``, ``github checkout-prs`` and ``traverse`` with GitHub integration) +will only download open PRs authored by the current user (as determined from the GitHub API token), +instead of all open PRs in the repository. + +This can speed up operations considerably in repositories with hundreds or thousands of open PRs, +at the cost of not being able to discover PRs opened by other users when traversing PR chains +(for example, when checking out an entire stack that includes PRs from multiple authors). + +A valid GitHub API token is required when this key is set. + +The ``--all`` flag to ``github checkout-prs`` and ``github update-pr-descriptions`` +still downloads all open PRs in the repository, regardless of this setting. + +The ``--by=`` flag to ``github checkout-prs`` and ``github update-pr-descriptions`` +downloads open PRs authored by the given user directly (rather than filtering the current user's PRs), +so it keeps working for any author even when this key is set. diff --git a/docs/source/git-config-keys/gitlab.access.rst b/docs/source/git-config-keys/gitlab.access.rst index 7bf752b62..c6d4c10e9 100644 --- a/docs/source/git-config-keys/gitlab.access.rst +++ b/docs/source/git-config-keys/gitlab.access.rst @@ -6,7 +6,8 @@ The name of the git remote (as in ``git remote``) that git-machete pushes the source branch to. Unless both ``machete.gitlab.namespace`` and ``machete.gitlab.project`` are set, this remote's URL is also inspected to derive the GitLab namespace and project that the merge request resides in. - The merge request is operated on through the GitLab API, which addresses that namespace/project rather than a git remote. + Unless the ``machete.gitlab.base*`` keys below point elsewhere, the merge request is operated on through the GitLab API, + which addresses that namespace/project rather than a git remote. By default (when this key is unset), if exactly one remote's URL corresponds to GitLab, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitLab. For example, ``git config machete.gitlab.remote origin`` @@ -22,6 +23,10 @@ ``machete.gitlab.baseRemote`` Like ``machete.gitlab.remote``, but used to locate the target project that the merge request targets, which may differ from the source project (for example, the target in an upstream project and the source in a fork). + Setting this key is what makes the MR-reading/-modifying commands (``anno-mrs``, ``checkout-mrs``, ``retarget-mr``, + ``restack-mr``, ``update-mr-descriptions``) address that target project rather than the source one. + ``create-mr`` does not need it: it infers the target project from the target branch's tracking remote, + so it already targets the correct project (even one in a separate fork/upstream project) even when this key is unset. Defaults to ``machete.gitlab.remote`` when unset. For example, ``git config machete.gitlab.baseRemote upstream`` diff --git a/docs/source/git-config-keys/gitlab.retrieveOnlyMyMergeRequests.rst b/docs/source/git-config-keys/gitlab.retrieveOnlyMyMergeRequests.rst new file mode 100644 index 000000000..6b623c711 --- /dev/null +++ b/docs/source/git-config-keys/gitlab.retrieveOnlyMyMergeRequests.rst @@ -0,0 +1,17 @@ +When set to ``true``, commands that need to list open merge requests in the project +(such as ``gitlab anno-mrs``, ``gitlab checkout-mrs`` and ``traverse`` with GitLab integration) +will only download open MRs authored by the current user (as determined from the GitLab API token), +instead of all open MRs in the project. + +This can speed up operations considerably in projects with hundreds or thousands of open MRs, +at the cost of not being able to discover MRs opened by other users when traversing MR chains +(for example, when checking out an entire stack that includes MRs from multiple authors). + +A valid GitLab API token is required when this key is set. + +The ``--all`` flag to ``gitlab checkout-mrs`` and ``gitlab update-mr-descriptions`` +still downloads all open MRs in the project, regardless of this setting. + +The ``--by=`` flag to ``gitlab checkout-mrs`` and ``gitlab update-mr-descriptions`` +downloads open MRs authored by the given user directly (rather than filtering the current user's MRs), +so it keeps working for any author even when this key is set. diff --git a/git_machete/__init__.py b/git_machete/__init__.py index 7047f37d0..d5f03c9ac 100644 --- a/git_machete/__init__.py +++ b/git_machete/__init__.py @@ -1 +1 @@ -__version__ = '3.44.2' +__version__ = '3.45.0' diff --git a/git_machete/client/traverse.py b/git_machete/client/traverse.py index 9be49f693..b176be732 100644 --- a/git_machete/client/traverse.py +++ b/git_machete/client/traverse.py @@ -248,7 +248,7 @@ def traverse( needs_retarget_pr = False if opt_sync_github_prs or opt_sync_gitlab_mrs: - prs = list(filter(lambda pr: pr.head == branch, self._get_all_open_prs())) + prs = list(filter(lambda pr: pr.head == branch, self._get_relevant_open_prs())) if len(prs) > 1: spec = self.code_hosting_spec raise MacheteException( @@ -260,7 +260,7 @@ def traverse( needs_create_pr = False if opt_sync_github_prs or opt_sync_gitlab_mrs: if parent: - prs = [_pr for _pr in self._get_all_open_prs() if _pr.head == branch] + prs = [_pr for _pr in self._get_relevant_open_prs() if _pr.head == branch] if not prs: needs_create_pr = True diff --git a/git_machete/client/with_code_hosting.py b/git_machete/client/with_code_hosting.py index 1364d4ad9..f37177b9c 100644 --- a/git_machete/client/with_code_hosting.py +++ b/git_machete/client/with_code_hosting.py @@ -28,6 +28,7 @@ def __init__(self, spec: CodeHostingSpec, *, read_layout_file: bool = True, self.__code_hosting_spec: CodeHostingSpec = spec self.__code_hosting_client: Optional[CodeHostingApi] = None self.__all_open_prs: Optional[List[PullRequest]] = None + self.__open_prs_by_author: Dict[str, List[PullRequest]] = {} @property def code_hosting_spec(self) -> CodeHostingSpec: @@ -43,15 +44,46 @@ def code_hosting_client(self) -> CodeHostingApi: def code_hosting_client(self, value: CodeHostingApi) -> None: self.__code_hosting_client = value + def _get_relevant_open_prs(self) -> List[PullRequest]: + # Honor the `retrieveOnlyMy{PullRequests,MergeRequests}` git config key and narrow the download to the current user's PRs. + # Callers that need every open PR regardless of this key (e.g. the `--all` flag) call `_get_all_open_prs` directly instead; + # once that has happened, reuse the already-downloaded full list here too (e.g. for reconstructing PR chains) + # rather than issuing a second, narrower query for the current user's PRs. + if self.__all_open_prs is not None: + return self.__all_open_prs + keys = self.code_hosting_spec.git_config_keys + if self._config.code_hosting_retrieve_only_my_pull_requests(keys): + return self._get_open_prs_of_current_user() + return self._get_all_open_prs() + def _get_all_open_prs(self) -> List[PullRequest]: if self.__all_open_prs is None: spec = self.code_hosting_spec print_fmt(f'Checking for open {spec.display_name} {spec.pr_short_name}s... ', newline=False) - self.__all_open_prs = self.code_hosting_client.get_open_pull_requests() + self.__all_open_prs = self.code_hosting_client.get_all_open_pull_requests() print_fmt(green_ok()) return self.__all_open_prs + def _get_open_prs_of_current_user(self) -> List[PullRequest]: + spec = self.code_hosting_spec + keys = spec.git_config_keys + author = self.code_hosting_client.get_current_user_login() + if author is None: + raise MacheteException( + f'`{keys.retrieve_only_my_pull_requests}` git config key is set, ' + f'but the current {spec.display_name} user could not be determined.\n' + f'Provide a {spec.display_name} API token via one of the:{spec.token_providers_message}') + return self._get_open_prs_by_author(author) + + def _get_open_prs_by_author(self, author: str) -> List[PullRequest]: + if author not in self.__open_prs_by_author: + spec = self.code_hosting_spec + print_fmt(f'Checking for open {spec.display_name} {spec.pr_short_name}s by {author}... ', newline=False) + self.__open_prs_by_author[author] = self.code_hosting_client.get_open_pull_requests_by_author(author) + print_fmt(green_ok()) + return self.__open_prs_by_author[author] + def _pull_request_annotation(self, pr: PullRequest, current_user: Optional[str], *, include_url: bool = False) -> str: anno = pr.display_text(fmt=False) if current_user != pr.user: @@ -228,7 +260,7 @@ def sync_annotations_to_prs(self, *, include_urls: bool) -> None: self._init_code_hosting_client() current_user: Optional[str] = self.code_hosting_client.get_current_user_login() debug(f'Current {self.code_hosting_spec.display_name} user is {current_user or ""}') - all_open_prs = self._get_all_open_prs() + all_open_prs = self._get_relevant_open_prs() self.__sync_annotations_to_branch_layout_file(all_open_prs, current_user, include_urls=include_urls, verbose=True) def create_pull_request( @@ -433,7 +465,8 @@ def create_pull_request( def restack_pull_request(self, *, opt_update_related_descriptions: bool) -> None: spec = self.code_hosting_spec head = self._git.get_current_branch() - _, org_repo_remote = self._init_code_hosting_client(branch_used_for_tracking_data=head) + _, org_repo_remote = self._init_code_hosting_client( + branch_used_for_tracking_data=head, base_branch_used_for_tracking_data=self.parent_of(head)) pr: Optional[PullRequest] = self.__get_sole_pull_request_for_head(head, ignore_if_missing=False) assert pr is not None @@ -538,7 +571,8 @@ def retarget_pull_request(self, *, opt_branch: Optional[LocalBranchShortName], head: ManagedBranchName = self.expect_in_managed_branches(opt_branch or self._git.get_current_branch()) spec = self.code_hosting_spec if self.__code_hosting_client is None: - self._init_code_hosting_client(branch_used_for_tracking_data=head) + self._init_code_hosting_client( + branch_used_for_tracking_data=head, base_branch_used_for_tracking_data=self.parent_of(head)) pr: Optional[PullRequest] = self.__get_sole_pull_request_for_head( head, ignore_if_missing=opt_ignore_if_missing) @@ -605,7 +639,8 @@ def __derive_org_repo_and_remote( keys = spec.git_config_keys if is_base: remote_key, org_key, repo_key = keys.base_remote, keys.base_organization, keys.base_repository - # The base remote falls back to the (non-base) remote; the base organization/repository have no such fallback. + # The base remote falls back to the (non-base) remote; the base organization/repository have no such fallback + # (`create_pull_request` relies on that to detect a fork base). remote_from_config = self._config.code_hosting_base_remote(keys) or self._config.code_hosting_remote(keys) org_from_config = self._config.code_hosting_base_organization(keys) repo_from_config = self._config.code_hosting_base_repository(keys) @@ -691,16 +726,45 @@ def __derive_org_repo_and_remote( f'{spec.git_config_keys.for_locating_repo_message()}\n') def _init_code_hosting_client(self, - branch_used_for_tracking_data: Optional[LocalBranchShortName] = None + branch_used_for_tracking_data: Optional[LocalBranchShortName] = None, + base_branch_used_for_tracking_data: Optional[LocalBranchShortName] = None ) -> Tuple[str, OrganizationAndRepositoryAndRemote]: if self.__code_hosting_client is not None: raise UnexpectedMacheteException("Code hosting client has already been initialized.") domain = self.__derive_code_hosting_domain() - org_repo_remote = self.__derive_org_repo_and_remote( + # PR-reading/-modifying commands must talk to the repository that *hosts* the PRs, i.e. the base repository. + # In a fork workflow the base (upstream) repository differs from the head (fork) repository that holds the branches, + # so the code hosting client is created against the base repository, while the returned head remote is still what + # callers use to fetch/push branches. + # The base repository is located in two ways: explicit machete..base* config keys take precedence (honored for + # every PR-reading/-modifying command); otherwise, when a base branch is given (retarget/restack), the base repository + # is inferred from that branch's tracking remote, exactly like create_pull_request does. Inference is best-effort: if it + # cannot be resolved unambiguously (e.g. the base branch has no tracking data among several candidate remotes), we fall + # back to the head repository, preserving the pre-inference behavior. When neither applies, the base repository resolves + # to the head one, so this is a no-op for the common (non-fork) case. + head_org_repo_remote = self.__derive_org_repo_and_remote( domain=domain, branch_used_for_tracking_data=branch_used_for_tracking_data) + keys = self.code_hosting_spec.git_config_keys + base_config_present = ( + self._config.code_hosting_base_remote(keys) is not None or + self._config.code_hosting_base_organization(keys) is not None or + self._config.code_hosting_base_repository(keys) is not None) + base_org_repo_remote = head_org_repo_remote + if base_config_present: + base_org_repo_remote = self.__derive_org_repo_and_remote( + domain=domain, + branch_used_for_tracking_data=base_branch_used_for_tracking_data or branch_used_for_tracking_data, + is_base=True) + elif base_branch_used_for_tracking_data is not None: + try: + base_org_repo_remote = self.__derive_org_repo_and_remote( + domain=domain, branch_used_for_tracking_data=base_branch_used_for_tracking_data, is_base=True) + except MacheteException: + # Ambiguous inference with no explicit base* config to honor -> fall back to the head repository. + pass self.code_hosting_client = self.code_hosting_spec.create_client( - domain=domain, organization=org_repo_remote.organization, repository=org_repo_remote.repository) - return domain, org_repo_remote + domain=domain, organization=base_org_repo_remote.organization, repository=base_org_repo_remote.repository) + return domain, head_org_repo_remote START_GIT_MACHETE_GENERATED_COMMENT = '' END_GIT_MACHETE_GENERATED_COMMENT = '' @@ -930,7 +994,7 @@ def __get_downwards_tree_excluding_pr(self, original_pr: PullRequest) -> List[Tu def reverse_pr_dfs(pr: PullRequest, depth: int) -> Iterator[Tuple[PullRequest, int]]: visited_head_branches.add(pr.head) - down_prs = filter(lambda x: x.base == pr.head, self._get_all_open_prs()) + down_prs = filter(lambda x: x.base == pr.head, self._get_relevant_open_prs()) for down_pr in sorted(down_prs, key=lambda x: x.number): if down_pr.head not in visited_head_branches: yield (down_pr, depth + 1) @@ -953,7 +1017,7 @@ def __get_upwards_path_including_pr(self, original_pr: PullRequest) -> List[Pull raise MacheteException(f"There is a cycle between {spec.display_name} {spec.pr_short_name}s: " + " -> ".join(visited_head_branches + [pr_base])) visited_head_branches += [pr_base] - pr = find_or_none(lambda x: x.head == pr_base, self._get_all_open_prs()) + pr = find_or_none(lambda x: x.head == pr_base, self._get_relevant_open_prs()) path = (path + [pr]) if pr else path pr_base = pr.base if pr else None return path @@ -968,12 +1032,12 @@ def _get_applicable_pull_requests( ) -> List[PullRequest]: result: List[PullRequest] = [] spec = self.code_hosting_spec - all_open_prs = self._get_all_open_prs() repo_pretty = ( f"{spec.repository_name} {self.code_hosting_client.organization}/{self.code_hosting_client.repository}") if pr_numbers: + relevant_open_prs = self._get_relevant_open_prs() for pr_number in pr_numbers: - pr: Optional[PullRequest] = find_or_none(lambda x: x.number == pr_number, all_open_prs) + pr: Optional[PullRequest] = find_or_none(lambda x: x.number == pr_number, relevant_open_prs) if pr: result.append(pr) else: @@ -986,12 +1050,21 @@ def _get_applicable_pull_requests( f"{spec.pr_short_name} {spec.pr_ordinal_char}{pr_number} is not found in {repo_pretty}") return result if all: + # The `--all` flag deliberately overrides `retrieveOnlyMy{PullRequests,MergeRequests}` and downloads every open PR. + all_open_prs = self._get_all_open_prs() if not all_open_prs: warn(f"currently there are no {spec.pr_full_name}s opened in {repo_pretty}") return [] return all_open_prs elif by: - result = [pr for pr in all_open_prs if pr.user == by] + keys = spec.git_config_keys + if self._config.code_hosting_retrieve_only_my_pull_requests(keys): + # With this key set we avoid downloading every open PR just to filter locally; + # ask the API for this author's PRs directly. `by` may be a user other than the current one + # (e.g. `--by=`), which is intentionally supported here. + result = self._get_open_prs_by_author(by) + else: + result = [pr for pr in self._get_all_open_prs() if pr.user == by] if not result: warn(f"user {by} has no open {spec.pr_full_name} in {repo_pretty}") return [] diff --git a/git_machete/code_hosting.py b/git_machete/code_hosting.py index 2f2aec0fa..ebba9b856 100644 --- a/git_machete/code_hosting.py +++ b/git_machete/code_hosting.py @@ -160,6 +160,7 @@ class CodeHostingGitConfigKeys(NamedTuple): annotate_with_urls: str force_description_from_commit_message: str pr_description_intro_style: str + retrieve_only_my_pull_requests: str def for_locating_repo_message(self) -> str: return f"`{self.domain}`, `{self.organization}`, `{self.repository}`, `{self.remote}`" @@ -244,11 +245,15 @@ def set_draft_status_of_pull_request(self, number: int, *, target_draft_status: Returns false if PR already had the desired draft status, and hence draft status has NOT been toggled.""" @abstractmethod - def get_open_pull_requests_by_head(self, head: LocalBranchShortName) -> List[PullRequest]: + def get_all_open_pull_requests(self) -> List[PullRequest]: pass @abstractmethod - def get_open_pull_requests(self) -> List[PullRequest]: + def get_open_pull_requests_by_author(self, author: str) -> List[PullRequest]: + pass + + @abstractmethod + def get_open_pull_requests_by_head(self, head: LocalBranchShortName) -> List[PullRequest]: pass @abstractmethod diff --git a/git_machete/config.py b/git_machete/config.py index c232bc19a..4e265e27d 100644 --- a/git_machete/config.py +++ b/git_machete/config.py @@ -148,3 +148,6 @@ def code_hosting_pr_description_intro_style(self, keys: CodeHostingGitConfigKeys return PRDescriptionIntroStyle.from_string( value=value, from_where=f"`{keys.pr_description_intro_style}` git config key") + + def code_hosting_retrieve_only_my_pull_requests(self, keys: CodeHostingGitConfigKeys) -> bool: + return self._git.get_boolean_config_attr(key=keys.retrieve_only_my_pull_requests, default_value=False) diff --git a/git_machete/generated_docs.py b/git_machete/generated_docs.py index 434181d99..a6cf98455 100644 --- a/git_machete/generated_docs.py +++ b/git_machete/generated_docs.py @@ -253,7 +253,8 @@ The name of the git remote (as in `git remote`) that git-machete pushes the head branch to. Unless both `machete.github.organization` and `machete.github.repository` are set, this remote's URL is also inspected to derive the GitHub organization and repository that the pull request resides in. - The pull request is operated on through the GitHub API, which addresses that organization/repository rather than a git remote. + Unless the `machete.github.base*` keys below point elsewhere, the pull request is operated on through the GitHub API, + which addresses that organization/repository rather than a git remote. By default (when this key is unset), if exactly one remote's URL corresponds to GitHub, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitHub. For example, `git config machete.github.remote origin` @@ -269,6 +270,10 @@ `machete.github.baseRemote` Like `machete.github.remote`, but used to locate the base repository that the pull request targets, which may differ from the head repository (for example, the base in an upstream repository and the head in a fork). + Setting this key is what makes the PR-reading/-modifying commands (`anno-prs`, `checkout-prs`, `retarget-pr`, + `restack-pr`, `update-pr-descriptions`) address that base repository rather than the head one. + `create-pr` does not need it: it infers the base repository from the base branch's tracking remote, + so it already targets the correct base (even one in a separate fork/upstream repository) even when this key is unset. Defaults to `machete.github.remote` when unset. For example, `git config machete.github.baseRemote upstream` @@ -310,6 +315,25 @@ * `up-only-no-branches` — same as `up-only`, but no branch names are included (only PR numbers & titles) * `none` — prepend no intro to the PR description at all + `machete.github.retrieveOnlyMyPullRequests` + When set to `true`, commands that need to list open pull requests in the repository + (such as `github anno-prs`, `github checkout-prs` and `traverse` with GitHub integration) + will only download open PRs authored by the current user (as determined from the GitHub API token), + instead of all open PRs in the repository. + + This can speed up operations considerably in repositories with hundreds or thousands of open PRs, + at the cost of not being able to discover PRs opened by other users when traversing PR chains + (for example, when checking out an entire stack that includes PRs from multiple authors). + + A valid GitHub API token is required when this key is set. + + The `--all` flag to `github checkout-prs` and `github update-pr-descriptions` + still downloads all open PRs in the repository, regardless of this setting. + + The `--by=` flag to `github checkout-prs` and `github update-pr-descriptions` + downloads open PRs authored by the given user directly (rather than filtering the current user's PRs), + so it keeps working for any author even when this key is set. + `machete.gitlab.{domain,remote,namespace,project,baseRemote,baseNamespace,baseProject}` `machete.gitlab.domain` The domain of the GitLab API server, for use with a GitLab self-managed instance; otherwise inferred from the remote URL. @@ -319,7 +343,8 @@ The name of the git remote (as in `git remote`) that git-machete pushes the source branch to. Unless both `machete.gitlab.namespace` and `machete.gitlab.project` are set, this remote's URL is also inspected to derive the GitLab namespace and project that the merge request resides in. - The merge request is operated on through the GitLab API, which addresses that namespace/project rather than a git remote. + Unless the `machete.gitlab.base*` keys below point elsewhere, the merge request is operated on through the GitLab API, + which addresses that namespace/project rather than a git remote. By default (when this key is unset), if exactly one remote's URL corresponds to GitLab, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitLab. For example, `git config machete.gitlab.remote origin` @@ -335,6 +360,10 @@ `machete.gitlab.baseRemote` Like `machete.gitlab.remote`, but used to locate the target project that the merge request targets, which may differ from the source project (for example, the target in an upstream project and the source in a fork). + Setting this key is what makes the MR-reading/-modifying commands (`anno-mrs`, `checkout-mrs`, `retarget-mr`, + `restack-mr`, `update-mr-descriptions`) address that target project rather than the source one. + `create-mr` does not need it: it infers the target project from the target branch's tracking remote, + so it already targets the correct project (even one in a separate fork/upstream project) even when this key is unset. Defaults to `machete.gitlab.remote` when unset. For example, `git config machete.gitlab.baseRemote upstream` @@ -376,6 +405,25 @@ * `up-only-no-branches` — same as `up-only`, but no branch names are included (only MR numbers & titles) * `none` — prepend no intro to the MR description at all + `machete.gitlab.retrieveOnlyMyMergeRequests` + When set to `true`, commands that need to list open merge requests in the project + (such as `gitlab anno-mrs`, `gitlab checkout-mrs` and `traverse` with GitLab integration) + will only download open MRs authored by the current user (as determined from the GitLab API token), + instead of all open MRs in the project. + + This can speed up operations considerably in projects with hundreds or thousands of open MRs, + at the cost of not being able to discover MRs opened by other users when traversing MR chains + (for example, when checking out an entire stack that includes MRs from multiple authors). + + A valid GitLab API token is required when this key is set. + + The `--all` flag to `gitlab checkout-mrs` and `gitlab update-mr-descriptions` + still downloads all open MRs in the project, regardless of this setting. + + The `--by=` flag to `gitlab checkout-mrs` and `gitlab update-mr-descriptions` + downloads open MRs authored by the given user directly (rather than filtering the current user's MRs), + so it keeps working for any author even when this key is set. + `machete.overrideForkPoint..to` Executing `git machete fork-point --override-to[-parent|-inferred|=] []` sets up a fork point override for ``. @@ -840,7 +888,8 @@ The name of the git remote (as in `git remote`) that git-machete pushes the head branch to. Unless both `machete.github.organization` and `machete.github.repository` are set, this remote's URL is also inspected to derive the GitHub organization and repository that the pull request resides in. - The pull request is operated on through the GitHub API, which addresses that organization/repository rather than a git remote. + Unless the `machete.github.base*` keys below point elsewhere, the pull request is operated on through the GitHub API, + which addresses that organization/repository rather than a git remote. By default (when this key is unset), if exactly one remote's URL corresponds to GitHub, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitHub. For example, `git config machete.github.remote origin` @@ -856,6 +905,10 @@ `machete.github.baseRemote` Like `machete.github.remote`, but used to locate the base repository that the pull request targets, which may differ from the head repository (for example, the base in an upstream repository and the head in a fork). + Setting this key is what makes the PR-reading/-modifying commands (`anno-prs`, `checkout-prs`, `retarget-pr`, + `restack-pr`, `update-pr-descriptions`) address that base repository rather than the head one. + `create-pr` does not need it: it infers the base repository from the base branch's tracking remote, + so it already targets the correct base (even one in a separate fork/upstream repository) even when this key is unset. Defaults to `machete.github.remote` when unset. For example, `git config machete.github.baseRemote upstream` @@ -897,6 +950,25 @@ * `up-only-no-branches` — same as `up-only`, but no branch names are included (only PR numbers & titles) * `none` — prepend no intro to the PR description at all + `machete.github.retrieveOnlyMyPullRequests` (`anno-prs`, `checkout-prs` and `update-pr-descriptions`) + When set to `true`, commands that need to list open pull requests in the repository + (such as `github anno-prs`, `github checkout-prs` and `traverse` with GitHub integration) + will only download open PRs authored by the current user (as determined from the GitHub API token), + instead of all open PRs in the repository. + + This can speed up operations considerably in repositories with hundreds or thousands of open PRs, + at the cost of not being able to discover PRs opened by other users when traversing PR chains + (for example, when checking out an entire stack that includes PRs from multiple authors). + + A valid GitHub API token is required when this key is set. + + The `--all` flag to `github checkout-prs` and `github update-pr-descriptions` + still downloads all open PRs in the repository, regardless of this setting. + + The `--by=` flag to `github checkout-prs` and `github update-pr-descriptions` + downloads open PRs authored by the given user directly (rather than filtering the current user's PRs), + so it keeps working for any author even when this key is set. + Environment variables (all subcommands) `GITHUB_TOKEN` @@ -1060,7 +1132,8 @@ The name of the git remote (as in `git remote`) that git-machete pushes the source branch to. Unless both `machete.gitlab.namespace` and `machete.gitlab.project` are set, this remote's URL is also inspected to derive the GitLab namespace and project that the merge request resides in. - The merge request is operated on through the GitLab API, which addresses that namespace/project rather than a git remote. + Unless the `machete.gitlab.base*` keys below point elsewhere, the merge request is operated on through the GitLab API, + which addresses that namespace/project rather than a git remote. By default (when this key is unset), if exactly one remote's URL corresponds to GitLab, that remote is selected automatically; set this key to disambiguate when more than one remote points to GitLab. For example, `git config machete.gitlab.remote origin` @@ -1076,6 +1149,10 @@ `machete.gitlab.baseRemote` Like `machete.gitlab.remote`, but used to locate the target project that the merge request targets, which may differ from the source project (for example, the target in an upstream project and the source in a fork). + Setting this key is what makes the MR-reading/-modifying commands (`anno-mrs`, `checkout-mrs`, `retarget-mr`, + `restack-mr`, `update-mr-descriptions`) address that target project rather than the source one. + `create-mr` does not need it: it infers the target project from the target branch's tracking remote, + so it already targets the correct project (even one in a separate fork/upstream project) even when this key is unset. Defaults to `machete.gitlab.remote` when unset. For example, `git config machete.gitlab.baseRemote upstream` @@ -1117,6 +1194,25 @@ * `up-only-no-branches` — same as `up-only`, but no branch names are included (only MR numbers & titles) * `none` — prepend no intro to the MR description at all + `machete.gitlab.retrieveOnlyMyMergeRequests` (`anno-mrs`, `checkout-mrs` and `update-mr-descriptions`) + When set to `true`, commands that need to list open merge requests in the project + (such as `gitlab anno-mrs`, `gitlab checkout-mrs` and `traverse` with GitLab integration) + will only download open MRs authored by the current user (as determined from the GitLab API token), + instead of all open MRs in the project. + + This can speed up operations considerably in projects with hundreds or thousands of open MRs, + at the cost of not being able to discover MRs opened by other users when traversing MR chains + (for example, when checking out an entire stack that includes MRs from multiple authors). + + A valid GitLab API token is required when this key is set. + + The `--all` flag to `gitlab checkout-mrs` and `gitlab update-mr-descriptions` + still downloads all open MRs in the project, regardless of this setting. + + The `--by=` flag to `gitlab checkout-mrs` and `gitlab update-mr-descriptions` + downloads open MRs authored by the given user directly (rather than filtering the current user's MRs), + so it keeps working for any author even when this key is set. + Environment variables (all subcommands) `GITLAB_TOKEN` diff --git a/git_machete/github.py b/git_machete/github.py index f091ef5e6..49ac933b4 100644 --- a/git_machete/github.py +++ b/git_machete/github.py @@ -309,6 +309,69 @@ def __extract_failure_info_from_422(response: Any) -> str: else: return str(response) + def __get_pull_request_from_graphql_search_node(self, pr_node: Dict[str, Any]) -> PullRequest: + head_repository = pr_node.get('headRepository') or {} + return PullRequest( + number=int(pr_node['number']), + display_prefix='PR #', + user=pr_node['author']['login'], + base=pr_node['baseRefName'], + head=pr_node['headRefName'], + head_repo_id=int(head_repository.get('databaseId') or 0), + html_url=pr_node['url'], + state=str(pr_node['state']).lower(), + title=pr_node['title'], + description=pr_node.get('body')) + + # GraphQL (rather than REST) is used here on purpose - there is no REST endpoint that can filter PRs by author server-side + # while still returning the PR-specific fields we need: + # - `/repos/{owner}/{repo}/pulls` returns head/base branches (and head repo id), but supports no author/user filter at all + # (only state/head/base/sort/direction), so it would force downloading every open PR and filtering client-side - + # exactly the pagination cost this whole feature exists to avoid. + # - `/search/issues?q=is:pr+author:...` does filter by author, but returns only the fields common to issues and PRs; + # it omits head/base branch names and head repo id, which we can't do without (and there is no `/search/pulls`). + # The GraphQL `search(type: ISSUE)` connection is the only option that filters by author *and* exposes + # headRefName/baseRefName/headRepository.databaseId in a single query. See https://github.com/VirtusLab/git-machete/issues/1040. + def __get_open_pull_requests_by_author_via_graphql(self, author: str) -> List[PullRequest]: + search_query = f'is:pr is:open author:{author} repo:{self.organization}/{self.repository}' + result: List[PullRequest] = [] + cursor: Optional[str] = None + while True: + after_clause = f', after: "{cursor}"' if cursor else '' + query = f"""{{ + search(query: "{search_query}", type: ISSUE, first: {self.MAX_PULLS_PER_PAGE_COUNT}{after_clause}) {{ + edges {{ + node {{ + ... on PullRequest {{ + number + title + body + state + url + author {{ login }} + baseRefName + headRefName + headRepository {{ databaseId }} + }} + }} + }} + pageInfo {{ + endCursor + hasNextPage + }} + }} + }}""" + response = self.__fire_github_graphql_api_request(query) + search_result = response['data']['search'] + for edge in search_result['edges']: + result.append(self.__get_pull_request_from_graphql_search_node(edge['node'])) + page_info = search_result['pageInfo'] + if page_info['hasNextPage']: + cursor = page_info['endCursor'] + else: + break + return result + def create_pull_request(self, head: str, head_org_repo: OrganizationAndRepository, *, base: str, title: str, description: str, draft: bool) -> PullRequest: request_body: Dict[str, Any] = { @@ -394,12 +457,15 @@ def set_draft_status_of_pull_request(self, number: int, *, target_draft_status: debug(f"mutation response is {response}") return True - def get_open_pull_requests_by_head(self, head: LocalBranchShortName) -> List[PullRequest]: - prs = self.__fire_github_api_repo_request(method='GET', path_suffix=f'/pulls?head={self.organization}:{head}') + def get_all_open_pull_requests(self) -> List[PullRequest]: + prs = self.__fire_github_api_repo_request(method='GET', path_suffix=f'/pulls?per_page={self.MAX_PULLS_PER_PAGE_COUNT}') return [self.__get_pull_request_from_json(pr) for pr in prs] - def get_open_pull_requests(self) -> List[PullRequest]: - prs = self.__fire_github_api_repo_request(method='GET', path_suffix=f'/pulls?per_page={self.MAX_PULLS_PER_PAGE_COUNT}') + def get_open_pull_requests_by_author(self, author: str) -> List[PullRequest]: + return self.__get_open_pull_requests_by_author_via_graphql(author) + + def get_open_pull_requests_by_head(self, head: LocalBranchShortName) -> List[PullRequest]: + prs = self.__fire_github_api_repo_request(method='GET', path_suffix=f'/pulls?head={self.organization}:{head}') return [self.__get_pull_request_from_json(pr) for pr in prs] def get_current_user_login(self) -> Optional[str]: @@ -473,5 +539,6 @@ def get_ref_name_for_pull_request(self, pr_number: int) -> str: annotate_with_urls='machete.github.annotateWithUrls', force_description_from_commit_message='machete.github.forceDescriptionFromCommitMessage', pr_description_intro_style='machete.github.prDescriptionIntroStyle', + retrieve_only_my_pull_requests='machete.github.retrieveOnlyMyPullRequests', ) ) diff --git a/git_machete/gitlab.py b/git_machete/gitlab.py index 2560a7a20..34d177ef1 100644 --- a/git_machete/gitlab.py +++ b/git_machete/gitlab.py @@ -292,15 +292,21 @@ def set_draft_status_of_pull_request(self, number: int, *, target_draft_status: self.__fire_gitlab_api_project_request(method='PUT', path_suffix=f'/merge_requests/{number}', request_body=request_body) return True - def get_open_pull_requests_by_head(self, head: LocalBranchShortName) -> List[PullRequest]: - mrs = self.__fire_gitlab_api_project_request(method='GET', path_suffix=f'/merge_requests?state=opened&source_branch={head}') - return [self.__get_merge_request_from_json(mr) for mr in mrs] - - def get_open_pull_requests(self) -> List[PullRequest]: + def get_all_open_pull_requests(self) -> List[PullRequest]: mrs = self.__fire_gitlab_api_project_request(method='GET', path_suffix=f'/merge_requests?state=opened&per_page={self.MAX_PULLS_PER_PAGE_COUNT}') return [self.__get_merge_request_from_json(mr) for mr in mrs] + def get_open_pull_requests_by_author(self, author: str) -> List[PullRequest]: + path_suffix = (f'/merge_requests?state=opened&per_page={self.MAX_PULLS_PER_PAGE_COUNT}' + f'&author_username={urllib.parse.quote(author, safe="")}') + mrs = self.__fire_gitlab_api_project_request(method='GET', path_suffix=path_suffix) + return [self.__get_merge_request_from_json(mr) for mr in mrs] + + def get_open_pull_requests_by_head(self, head: LocalBranchShortName) -> List[PullRequest]: + mrs = self.__fire_gitlab_api_project_request(method='GET', path_suffix=f'/merge_requests?state=opened&source_branch={head}') + return [self.__get_merge_request_from_json(mr) for mr in mrs] + def get_current_user_login(self) -> Optional[str]: if not self.__token: return None @@ -368,5 +374,6 @@ def get_ref_name_for_pull_request(self, mr_number: int) -> str: annotate_with_urls='machete.gitlab.annotateWithUrls', force_description_from_commit_message='machete.gitlab.forceDescriptionFromCommitMessage', pr_description_intro_style='machete.gitlab.mrDescriptionIntroStyle', + retrieve_only_my_pull_requests='machete.gitlab.retrieveOnlyMyMergeRequests', ) ) diff --git a/tests/mockers_github.py b/tests/mockers_github.py index a161f948a..000539118 100644 --- a/tests/mockers_github.py +++ b/tests/mockers_github.py @@ -14,6 +14,7 @@ def mock_pr_json(head: str, base: str, number: int, repo_id: int = 1, + base_repo_id: Optional[int] = None, user: str = 'some_other_user', html_url: str = 'www.github.com', body: Optional[str] = '# Summary', @@ -23,7 +24,9 @@ def mock_pr_json(head: str, base: str, number: int, return { 'head': {'ref': head, 'repo': {'id': repo_id}}, 'user': {'login': user}, - 'base': {'ref': base}, + # `base.repo.id` records which repository hosts the PR; `None` (the default) means "served for any repository", + # so the vast majority of tests that don't care about fork/base targeting keep working unchanged. + 'base': {'ref': base, 'repo': {'id': base_repo_id}}, 'number': str(number), 'html_url': html_url, 'title': 'PR title', @@ -113,6 +116,12 @@ def url_with_query_params(**new_params: Any) -> str: new_query_string: str = urlencode({**query_params, **new_params}) return parsed_url._replace(query=new_query_string).geturl() + def find_repo_id_by_org_and_repo(org: str, repo: str) -> Optional[int]: + for repo_id, repo_data in github_api_state.repositories.items(): + if repo_data['owner']['login'] == org and repo_data['name'] == repo: + return repo_id + return None + def handle_get() -> "MockAPIResponse": if url_path_matches('/repositories/[0-9]+'): repo_no = int(url_segments[-1]) @@ -120,14 +129,22 @@ def handle_get() -> "MockAPIResponse": return MockAPIResponse(HTTPStatus.OK, github_api_state.repositories[repo_no]) raise error_404() elif url_path_matches('/repos/*/*/pulls'): + # The `/repos/{org}/{repo}/pulls` endpoint lists PRs hosted by the base repository `{org}/{repo}`, + # so filter out PRs whose base repository (if declared) differs from the one being queried. + requested_base_repo_id: Optional[int] = find_repo_id_by_org_and_repo(url_segments[-3], url_segments[-2]) + + def hosted_by_requested_repo(pull: Dict[str, Any]) -> bool: + base_repo_id = pull['base'].get('repo', {}).get('id') + return base_repo_id is None or base_repo_id == requested_base_repo_id + full_head_name: Optional[str] = query_params.get('head') if full_head_name: head: str = full_head_name.split(':')[1] - prs = github_api_state.get_open_pulls_by_head(head) + prs = [pull for pull in github_api_state.get_open_pulls_by_head(head) if hosted_by_requested_repo(pull)] # If no matching PRs are found, the real GitHub returns 200 OK with an empty JSON array - not 404. return MockAPIResponse(HTTPStatus.OK, prs) else: - pulls = github_api_state.get_open_pulls() + pulls = [pull for pull in github_api_state.get_open_pulls() if hosted_by_requested_repo(pull)] page_str = query_params.get('page') page = int(page_str) if page_str else 1 per_page = int(query_params['per_page']) @@ -187,7 +204,9 @@ def handle_post() -> "MockAPIResponse": return MockAPIResponse(HTTPStatus.OK, pull) elif parsed_url.path in ("/api/graphql", "/graphql"): # /api/graphql for Enterprise domains query_or_mutation = json_data['query'] - if 'query {' in query_or_mutation: + if 'search(' in query_or_mutation: + return search_pull_requests_by_author() + elif 'query {' in query_or_mutation: match = re.search(r'pullRequest\(number: ([0-9]+)\)', query_or_mutation) assert match is not None pr_number = int(match.group(1)) @@ -238,6 +257,33 @@ def fill_pull_request_from_json_data(pull: Dict[str, Any]) -> None: else: pull[key] = value + def search_pull_requests_by_author() -> "MockAPIResponse": + query = json_data['query'] + author_match = re.search(r'author:(\S+) repo:', query) + assert author_match is not None + author = author_match.group(1) + matching = [pull for pull in github_api_state.get_open_pulls() if pull['user']['login'] == author] + # Deliberately paginate one PR at a time (ignoring the requested `first:`) so that even a couple of PRs + # exercise the cursor-following loop in `GitHubApi.get_open_pull_requests_by_author`. + after_match = re.search(r'after: "([0-9]+)"', query) + start = int(after_match.group(1)) if after_match else 0 + end = start + 1 + edges = [{'node': { + 'number': int(pull['number']), + 'title': pull['title'], + 'body': pull['body'], + 'state': str(pull['state']).upper(), + 'url': pull['html_url'], + 'author': {'login': pull['user']['login']}, + 'baseRefName': pull['base']['ref'], + 'headRefName': pull['head']['ref'], + 'headRepository': ({'databaseId': pull['head']['repo']['id']} if pull['head']['repo'] is not None else None), + }} for pull in matching[start:end]] + return MockAPIResponse(HTTPStatus.OK, {'data': {'search': { + 'edges': edges, + 'pageInfo': {'endCursor': str(end), 'hasNextPage': end < len(matching)} + }}}) + def redirect_307(location: str) -> HTTPError: return HTTPError(parsed_url.hostname, 307, 'Temporary redirect', {'Location': location}, None) # type: ignore[arg-type] diff --git a/tests/mockers_gitlab.py b/tests/mockers_gitlab.py index 07c0a83c9..6a921f4fc 100644 --- a/tests/mockers_gitlab.py +++ b/tests/mockers_gitlab.py @@ -21,6 +21,7 @@ def mock_mr_json(head: str, base: str, number: int, repo_id: int = 1, + base_repo_id: Optional[int] = None, user: str = 'some_other_user', html_url: str = 'www.gitlab.com', body: Optional[str] = '# Summary', @@ -30,6 +31,9 @@ def mock_mr_json(head: str, base: str, number: int, return { 'source_branch': head, 'source_project_id': repo_id, + # `project_id` records the target project that hosts the MR; `None` (the default) means "served for any project", + # so the vast majority of tests that don't care about fork/target targeting keep working unchanged. + 'project_id': base_repo_id, 'target_branch': base, 'author': {'username': user}, 'iid': str(number), @@ -127,6 +131,15 @@ def url_with_query_params(**new_params: Any) -> str: new_query_string: str = urlencode({**query_params, **new_params}) return parsed_url._replace(query=new_query_string).geturl() + def find_project_id(id_or_path: str) -> Optional[int]: + if id_or_path.isdigit(): + return int(id_or_path) + full_path = urllib.parse.unquote(id_or_path) + for project_id, project in gitlab_api_state.projects.items(): + if project['namespace']['full_path'] == full_path: + return project_id + return None + def handle_get() -> "MockAPIResponse": if url_path_matches('/projects/[0-9]+'): repo_no = int(url_segments[-1]) @@ -140,13 +153,24 @@ def handle_get() -> "MockAPIResponse": return MockAPIResponse(HTTPStatus.OK, project) raise error_404() elif url_path_matches('/projects/*/merge_requests'): + # The `/projects/{id}/merge_requests` endpoint lists MRs hosted by the target project `{id}`, + # so filter out MRs whose target project (if declared) differs from the one being queried. + requested_target_project_id: Optional[int] = find_project_id(url_segments[-2]) + + def hosted_by_requested_project(mr: Dict[str, Any]) -> bool: + target_project_id = mr.get('project_id') + return target_project_id is None or target_project_id == requested_target_project_id + head: Optional[str] = query_params.get('source_branch') if head: - mrs = gitlab_api_state.get_open_mrs_by_head(head) + mrs = [mr for mr in gitlab_api_state.get_open_mrs_by_head(head) if hosted_by_requested_project(mr)] # If no matching MRs are found, the real GitLab returns 200 OK with an empty JSON array - not 404. return MockAPIResponse(HTTPStatus.OK, mrs) else: - mrs = gitlab_api_state.get_open_mrs() + mrs = [mr for mr in gitlab_api_state.get_open_mrs() if hosted_by_requested_project(mr)] + author_username: Optional[str] = query_params.get('author_username') + if author_username: + mrs = [mr for mr in mrs if mr['author']['username'] == author_username] page_str = query_params.get('page') page = int(page_str) if page_str else 1 per_page_str = query_params.get('per_page') diff --git a/tests/test_github_anno_prs.py b/tests/test_github_anno_prs.py index 87883ba6a..eeed25558 100644 --- a/tests/test_github_anno_prs.py +++ b/tests/test_github_anno_prs.py @@ -4,7 +4,9 @@ from tests.cli_runner import assert_failure, assert_success, launch_command, rewrite_branch_layout_file from tests.git_repository import (add_remote, amend_commit, check_out, commit, create_repo, create_repo_with_remote, delete_branch, new_branch, push, remove_remote, reset_to, set_git_config_key, wait_to_bump_commit_timestamp) -from tests.mockers_github import MockGitHubAPIState, mock_github_token_for_domain_fake, mock_pr_json, mock_urlopen +from tests.mockers_code_hosting import mock_from_url +from tests.mockers_github import (MockGitHubAPIState, mock_github_token_for_domain_fake, mock_github_token_for_domain_none, mock_pr_json, + mock_urlopen) class TestGitHubAnnoPRs(BaseTest): @@ -193,6 +195,67 @@ def test_github_anno_prs_no_remotes(self) -> None: remove_remote() assert_failure(["github", "anno-prs"], "No remotes defined for this repository (see git remote)") + def test_github_anno_prs_targets_base_repo(self, mocker: MockerFixture) -> None: + # In a fork workflow the PRs are hosted by the base (upstream) repository, not the head (fork) repository. + # Reading commands must therefore honor `machete.github.baseRemote` and query the base repository for PRs. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + repositories = { + 1: {'owner': {'login': 'example-org'}, 'name': 'example-repo', + 'clone_url': 'https://github.com/example-org/example-repo.git'}, + 2: {'owner': {'login': 'example-org'}, 'name': 'example-repo-1', + 'clone_url': 'https://github.com/example-org/example-repo-1.git'}, + } + github_api_state = MockGitHubAPIState( + repositories, + mock_pr_json(number=1, head='feature', base='develop', repo_id=1, base_repo_id=2, user='github_user'), + mock_pr_json(number=2, head='develop', base='master', repo_id=1, base_repo_id=2, user='github_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(github_api_state)) + + # `origin` (-> example-org/example-repo) is the head/fork remote holding the branches, + # while `upstream` (-> example-org/example-repo-1) is the base remote that hosts the PRs. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push() + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tdevelop\n\t\tfeature") + + # Without any base config the client targets the head repository (origin), which does not host these PRs. + launch_command("github", "anno-prs") + assert_success( + ["status"], + """ + master + | + o-develop + | + o-feature * + """ + ) + + # `machete.github.baseRemote` points reading commands at the base repository that actually hosts the PRs. + set_git_config_key("machete.github.baseRemote", "upstream") + launch_command("github", "anno-prs") + assert_success( + ["status"], + """ + master + | + o-develop PR #2 + | + o-feature * PR #1 + """ + ) + def test_github_anno_prs_multiple_non_origin_github_remotes(self) -> None: create_repo() add_remote("origin-1", "https://github.com/tester/repo_sandbox-1.git") @@ -205,3 +268,82 @@ def test_github_anno_prs_multiple_non_origin_github_remotes(self) -> None: machete.github.domain, machete.github.organization, machete.github.repository, machete.github.remote """ ) + + @staticmethod + def github_api_state_for_test_anno_prs_retrieve_only_mine() -> MockGitHubAPIState: + return MockGitHubAPIState.with_prs( + mock_pr_json(number=7, user='some_other_user', head='allow-ownership-link', base='develop'), + mock_pr_json(number=31, user='github_user', head='call-ws', base='develop'), + mock_pr_json(number=37, user='github_user', head='develop', base='master') + ) + + def __setup_repo_for_retrieve_only_mine(self) -> None: + create_repo_with_remote() + new_branch("master") + commit("master commit") + push() + new_branch("develop") + commit("develop commit") + push() + new_branch("allow-ownership-link") + commit("Allow ownership links") + push() + check_out("develop") + new_branch("call-ws") + commit("Call web service") + push() + check_out("master") + set_git_config_key('machete.github.remote', 'origin') + set_git_config_key('machete.github.organization', 'tester') + set_git_config_key('machete.github.repository', 'repo_sandbox') + body: str = \ + """ + master + develop + allow-ownership-link + call-ws + """ + rewrite_branch_layout_file(body) + + def test_github_anno_prs_retrieve_only_mine(self, mocker: MockerFixture) -> None: + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(self.github_api_state_for_test_anno_prs_retrieve_only_mine())) + + self.__setup_repo_for_retrieve_only_mine() + set_git_config_key('machete.github.retrieveOnlyMyPullRequests', 'true') + + # With `retrieveOnlyMyPullRequests` set, only PRs authored by the current user (github_user) are downloaded and annotated; + # allow-ownership-link (PR #7 by some_other_user) is deliberately left unannotated. + launch_command('github', 'anno-prs') + assert_success( + ["status"], + """ + master * + | + o-develop PR #37 + | + o-allow-ownership-link + | + o-call-ws PR #31 + """ + ) + + def test_github_anno_prs_retrieve_only_mine_no_current_user(self, mocker: MockerFixture) -> None: + # No token -> the current user cannot be determined, so the fetch fails before any HTTP request is made + # (hence `urllib.request.urlopen` is deliberately left unpatched). + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_none) + + self.__setup_repo_for_retrieve_only_mine() + set_git_config_key('machete.github.retrieveOnlyMyPullRequests', 'true') + + assert_failure( + ["github", "anno-prs"], + """ + machete.github.retrieveOnlyMyPullRequests git config key is set, but the current GitHub user could not be determined. + Provide a GitHub API token via one of the: + 1. GITHUB_TOKEN environment variable + 2. Content of the ~/.github-token file + 3. Current auth token from the gh GitHub CLI + 4. Current auth token from the hub GitHub CLI + """ + ) diff --git a/tests/test_github_checkout_prs.py b/tests/test_github_checkout_prs.py index d36b03e58..9f2e1b9b9 100644 --- a/tests/test_github_checkout_prs.py +++ b/tests/test_github_checkout_prs.py @@ -422,6 +422,112 @@ def test_github_checkout_prs_of_current_user_and_other_users(self, mocker: Mocke """ ) + @staticmethod + def github_api_state_for_test_checkout_prs_retrieve_only_mine() -> MockGitHubAPIState: + return MockGitHubAPIState.with_prs( + mock_pr_json(head='feature/mine', base='develop', number=1, user='github_user'), + mock_pr_json(head='feature/theirs', base='develop', number=2, user='some_other_user'), + ) + + def __setup_repo_for_checkout_prs_retrieve_only_mine(self, mocker: MockerFixture) -> None: + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + self.patch_symbol(mocker, 'urllib.request.urlopen', + mock_urlopen(self.github_api_state_for_test_checkout_prs_retrieve_only_mine())) + create_repo_with_remote() + new_branch("develop") + commit("develop commit") + push() + new_branch("feature/mine") + commit("mine commit") + push() + check_out("develop") + new_branch("feature/theirs") + commit("theirs commit") + push() + check_out("develop") + rewrite_branch_layout_file("develop") + set_git_config_key('machete.github.retrieveOnlyMyPullRequests', 'true') + + def test_github_checkout_prs_retrieve_only_mine_all(self, mocker: MockerFixture) -> None: + # `--all` overrides `retrieveOnlyMyPullRequests`: every open PR is downloaded and checked out, + # including feature/theirs (PR #2) which belongs to another user. + self.__setup_repo_for_checkout_prs_retrieve_only_mine(mocker) + + assert_success( + ['github', 'checkout-prs', '--all'], + """ + Checking for open GitHub PRs... OK + PR #1 checked out at local branch feature/mine + PR #2 checked out at local branch feature/theirs + """ + ) + assert_success( + ["status"], + """ + develop * + | + o-feature/mine PR #1 + | + o-feature/theirs PR #2 (some_other_user) rebase=no push=no + """ + ) + + def test_github_checkout_prs_retrieve_only_mine_by_other_user(self, mocker: MockerFixture) -> None: + # `--by=` asks the API for that user's PRs directly, so feature/theirs (PR #2) is still + # reachable even though `retrieveOnlyMyPullRequests` is set and PR #2 is not authored by the current user. + self.__setup_repo_for_checkout_prs_retrieve_only_mine(mocker) + + # The first fetch is some_other_user's PRs (the `--by` selection); + # the second is the chain reconstruction walking upstream over the current user's PRs. + assert_success( + ['github', 'checkout-prs', '--by', 'some_other_user'], + """ + Checking for open GitHub PRs by some_other_user... OK + Checking for open GitHub PRs by github_user... OK + PR #2 checked out at local branch feature/theirs + """ + ) + assert_success( + ["status"], + """ + develop + | + o-feature/theirs * PR #2 (some_other_user) rebase=no push=no + """ + ) + + def test_github_checkout_prs_retrieve_only_mine_regular(self, mocker: MockerFixture) -> None: + self.__setup_repo_for_checkout_prs_retrieve_only_mine(mocker) + + # `--mine` downloads only the current user's PRs, so only feature/mine (PR #1) is checked out. + assert_success( + ['github', 'checkout-prs', '--mine'], + """ + Checking for open GitHub PRs by github_user... OK + PR #1 checked out at local branch feature/mine + """ + ) + # Checking out another user's PR by explicit number still works (it falls back to a by-number fetch) + # even though that PR is absent from the current user's PR list. + assert_success( + ['github', 'checkout-prs', '2'], + """ + Checking for open GitHub PRs by github_user... OK + PR #2 checked out at local branch feature/theirs + """ + ) + assert_success( + ["status"], + """ + develop + | + o-feature/mine PR #1 + | + o-feature/theirs * PR #2 (some_other_user) rebase=no push=no + """ + ) + def test_github_checkout_prs_misc_failures_and_warns(self, mocker: MockerFixture) -> None: create_repo_with_remote() self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) @@ -602,3 +708,62 @@ def test_github_checkout_prs_main_to_main_pr(self, mocker: MockerFixture) -> Non 'Checking for open GitHub PRs... OK\n' 'PR #2 checked out at local branch fix-10341\n' ) + + def test_github_checkout_prs_targets_base_repo(self, mocker: MockerFixture) -> None: + # In a fork workflow the PRs are hosted by the base (upstream) repository, not the head (fork) repository. + # checkout-prs must therefore honor `machete.github.baseRemote` and query the base repository for PRs. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + repositories = { + 1: {'owner': {'login': 'example-org'}, 'name': 'example-repo', + 'clone_url': 'https://github.com/example-org/example-repo.git'}, + 2: {'owner': {'login': 'example-org'}, 'name': 'example-repo-1', + 'clone_url': 'https://github.com/example-org/example-repo-1.git'}, + } + github_api_state = MockGitHubAPIState( + repositories, + mock_pr_json(number=1, head='feature', base='develop', repo_id=1, base_repo_id=2, user='github_user'), + mock_pr_json(number=2, head='develop', base='master', repo_id=1, base_repo_id=2, user='github_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(github_api_state)) + + # `origin` (-> example-org/example-repo) is the head/fork remote holding the branches, + # while `upstream` (-> example-org/example-repo-1) is the base remote that hosts the PRs. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push() + new_branch("feature") + commit() + push() + + # Without any base config the client targets the head repository (origin), which does not host these PRs. + assert_success( + ['github', 'checkout-prs', '--all'], + 'Checking for open GitHub PRs... OK\n' + 'Warn: currently there are no pull requests opened in repository example-org/example-repo\n' + ) + + # `machete.github.baseRemote` points checkout-prs at the base repository that actually hosts the PRs. + set_git_config_key("machete.github.baseRemote", "upstream") + assert_success( + ['github', 'checkout-prs', '--all'], + 'Checking for open GitHub PRs... OK\n' + 'PR #2 checked out at local branch develop\n' + 'PR #1 checked out at local branch feature\n' + ) + assert_success( + ["status"], + """ + master + | + o-develop PR #2 + | + o-feature * PR #1 + """ + ) diff --git a/tests/test_github_restack_pr.py b/tests/test_github_restack_pr.py index aea59c2ab..970a42151 100644 --- a/tests/test_github_restack_pr.py +++ b/tests/test_github_restack_pr.py @@ -4,7 +4,8 @@ from tests.base_test import BaseTest from tests.cli_runner import assert_failure, assert_success, rewrite_branch_layout_file -from tests.git_repository import amend_commit, commit, create_repo_with_remote, new_branch, push, reset_to, set_git_config_key +from tests.git_repository import (add_remote, amend_commit, commit, create_repo, create_repo_with_remote, new_branch, push, reset_to, + set_git_config_key) from tests.mockers import fixed_author_and_committer_date_in_past from tests.mockers_code_hosting import mock_from_url from tests.mockers_github import MockGitHubAPIState, mock_github_token_for_domain_fake, mock_pr_json, mock_urlopen @@ -331,3 +332,46 @@ def test_github_restack_pr_branch_diverged_and_older(self, mocker: MockerFixture assert pr is not None assert pr['draft'] is False assert pr['base']['ref'] == 'master' + + def test_github_restack_pr_infers_base_repo_from_parent_tracking(self, mocker: MockerFixture) -> None: + # In a fork workflow the PR is hosted by the base (upstream) repository, not the head (fork) repository. + # Even with no base* config keys, restack-pr infers the base repository from the parent branch's tracking remote + # (just like create-pr does), so it queries the repository that actually hosts the PR. + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + repositories = { + 1: {'owner': {'login': 'example-org'}, 'name': 'example-repo', + 'clone_url': 'https://github.com/example-org/example-repo.git'}, + 2: {'owner': {'login': 'example-org'}, 'name': 'example-repo-1', + 'clone_url': 'https://github.com/example-org/example-repo-1.git'}, + } + github_api_state = MockGitHubAPIState( + repositories, + mock_pr_json(number=1, head='feature', base='master', repo_id=1, base_repo_id=2, user='github_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(github_api_state)) + + # `origin` (-> example-org/example-repo) is the head/fork remote holding the branches, while + # `upstream` (-> example-org/example-repo-1) is the base remote that hosts the PR. The parent branch (develop) + # tracks `upstream`, so the base repository is inferred from it - no machete.github.base* key is set. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push(remote="upstream") + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tdevelop\n\t\tfeature") + + assert_success( + ['github', 'restack-pr'], + "Switching base branch of PR #1 to develop... OK\n" + ) + pr = github_api_state.get_pull_by_number(1) + assert pr is not None + assert pr['base']['ref'] == 'develop' diff --git a/tests/test_github_retarget_pr.py b/tests/test_github_retarget_pr.py index bb61d75ee..70b74c4a7 100644 --- a/tests/test_github_retarget_pr.py +++ b/tests/test_github_retarget_pr.py @@ -487,3 +487,115 @@ def test_github_retarget_pr_root_branch(self, mocker: MockerFixture) -> None: "Branch master does not have a parent branch (it is a root) even though there is an open PR #15 to root.\n" "Consider modifying the branch layout file (git machete edit) so that master is a child of root." ) + + def test_github_retarget_pr_infers_base_repo_from_parent_tracking(self, mocker: MockerFixture) -> None: + # In a fork workflow the PR is hosted by the base (upstream) repository, not the head (fork) repository. + # Even with no base* config keys, retarget-pr infers the base repository from the parent branch's tracking remote + # (just like create-pr does), so it queries the repository that actually hosts the PR. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + repositories = { + 1: {'owner': {'login': 'example-org'}, 'name': 'example-repo', + 'clone_url': 'https://github.com/example-org/example-repo.git'}, + 2: {'owner': {'login': 'example-org'}, 'name': 'example-repo-1', + 'clone_url': 'https://github.com/example-org/example-repo-1.git'}, + } + github_api_state = MockGitHubAPIState( + repositories, + mock_pr_json(number=1, head='feature', base='master', repo_id=1, base_repo_id=2, user='github_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(github_api_state)) + + # `origin` (-> example-org/example-repo) is the head/fork remote holding the branches, while + # `upstream` (-> example-org/example-repo-1) is the base remote that hosts the PR. The parent branch (develop) + # tracks `upstream`, so the base repository is inferred from it - no machete.github.base* key is set. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push(remote="upstream") + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tdevelop\n\t\tfeature") + + assert_success( + ['github', 'retarget-pr'], + "Switching base branch of PR #1 to develop... OK\n" + ) + pr1 = github_api_state.get_pull_by_number(1) + assert pr1 is not None + assert pr1['base']['ref'] == 'develop' + + def test_github_retarget_pr_surfaces_base_config_error(self, mocker: MockerFixture) -> None: + # When a base* config key is set but cannot be resolved (here it points at a nonexistent remote), retarget-pr must + # surface the misconfiguration rather than silently falling back to the head repository. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + + create_repo_with_remote() + new_branch("master") + commit() + push() + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tfeature") + + set_git_config_key("machete.github.baseRemote", "nonexistent") + assert_failure( + ['github', 'retarget-pr'], + "machete.github.baseRemote git config key points to nonexistent remote, but such remote does not exist") + + def test_github_retarget_pr_targets_base_repo(self, mocker: MockerFixture) -> None: + # In a fork workflow the PR is hosted by the base (upstream) repository, not the head (fork) repository. + # retarget-pr must therefore honor `machete.github.baseRemote` and query the base repository for the PR. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + repositories = { + 1: {'owner': {'login': 'example-org'}, 'name': 'example-repo', + 'clone_url': 'https://github.com/example-org/example-repo.git'}, + 2: {'owner': {'login': 'example-org'}, 'name': 'example-repo-1', + 'clone_url': 'https://github.com/example-org/example-repo-1.git'}, + } + github_api_state = MockGitHubAPIState( + repositories, + mock_pr_json(number=1, head='feature', base='master', repo_id=1, base_repo_id=2, user='github_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(github_api_state)) + + # `origin` (-> example-org/example-repo) is the head/fork remote holding the branches, + # while `upstream` (-> example-org/example-repo-1) is the base remote that hosts the PR. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push() + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tdevelop\n\t\tfeature") + + # Without any base config the client targets the head repository (origin), which does not host the PR. + assert_failure( + ['github', 'retarget-pr'], + "No PRs in example-org/example-repo have feature as its head branch" + ) + + # `machete.github.baseRemote` points retarget-pr at the base repository that actually hosts the PR; + # the PR's stale base (master) is then retargeted to feature's actual parent (develop). + set_git_config_key("machete.github.baseRemote", "upstream") + assert_success( + ['github', 'retarget-pr'], + "Switching base branch of PR #1 to develop... OK\n" + ) + pr1 = github_api_state.get_pull_by_number(1) + assert pr1 is not None + assert pr1['base']['ref'] == 'develop' diff --git a/tests/test_github_update_pr_descriptions.py b/tests/test_github_update_pr_descriptions.py index 01579538e..e26423a0c 100644 --- a/tests/test_github_update_pr_descriptions.py +++ b/tests/test_github_update_pr_descriptions.py @@ -84,6 +84,96 @@ def test_github_update_pr_descriptions_related_updates_entire_stack(self, mocker """ ) + def test_github_update_pr_descriptions_related_retrieve_only_mine(self, mocker: MockerFixture) -> None: + """ + With `machete.github.retrieveOnlyMyPullRequests` set, the stack traversal underlying `--related` + downloads only the current user's PRs (repeatedly, exercising the per-author PR cache) rather than every open PR. + """ + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + prs = [ + mock_pr_json(head='branch1', base='root', number=1, user='github_user', body='# Summary\n'), + mock_pr_json(head='branch2', base='branch1', number=2, user='github_user', body='# Summary\n'), + mock_pr_json(head='branch3', base='branch2', number=3, user='github_user', body='# Summary\n') + ] + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(MockGitHubAPIState.with_prs(*prs))) + self.patch_symbol(mocker, 'git_machete.utils.date.get_current_date', lambda: '2023-12-31') + + create_repo_with_remote() + new_branch("root") + commit("initial commit") + push() + new_branch("branch1") + commit("branch1 commit") + push() + new_branch("branch2") + commit("branch2 commit") + push() + new_branch("branch3") + commit("branch3 commit") + push() + + rewrite_branch_layout_file(""" + root + branch1 + branch2 + branch3 + """) + check_out('branch2') + set_git_config_key('machete.github.retrieveOnlyMyPullRequests', 'true') + + assert_success( + ['github', 'update-pr-descriptions', '--related'], + """ + Checking for open GitHub PRs by github_user... OK + Updating description of PR #2 (branch2 -> branch1)... OK + Updating description of PR #3 (branch3 -> branch2)... OK + """ + ) + + def test_github_update_pr_descriptions_retrieve_only_mine_by_other_user(self, mocker: MockerFixture) -> None: + """ + `--by=` combined with `machete.github.retrieveOnlyMyPullRequests` asks the API for that user's PRs directly + (rather than filtering the current user's PRs), so PRs authored by someone else are still reachable when the key is set. + """ + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.github.GitHubToken.for_domain', mock_github_token_for_domain_fake) + prs = [ + mock_pr_json(head='branch1', base='root', number=1, user='github_user', body='# Summary\n'), + mock_pr_json(head='branch2', base='branch1', number=2, user='some_other_user', body='# Summary\n'), + ] + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(MockGitHubAPIState.with_prs(*prs))) + self.patch_symbol(mocker, 'git_machete.utils.date.get_current_date', lambda: '2023-12-31') + + create_repo_with_remote() + new_branch("root") + commit("initial commit") + push() + new_branch("branch1") + commit("branch1 commit") + push() + new_branch("branch2") + commit("branch2 commit") + push() + + rewrite_branch_layout_file(""" + root + branch1 + branch2 + """) + set_git_config_key('machete.github.retrieveOnlyMyPullRequests', 'true') + + # The first fetch is some_other_user's PRs (the `--by` selection); + # the second is the up-only intro generation walking the stack over the current user's PRs. + assert_success( + ['github', 'update-pr-descriptions', '--by', 'some_other_user'], + """ + Checking for open GitHub PRs by some_other_user... OK + Checking for open GitHub PRs by github_user... OK + Updating description of PR #2 (branch2 -> branch1)... OK + """ + ) + def test_github_update_pr_descriptions_no_flag_defaults_to_related(self, mocker: MockerFixture) -> None: """ With no `--all`/`--by`/`--mine`/`--related` flag, `update-pr-descriptions` defaults to `--related`: diff --git a/tests/test_gitlab_anno_mrs.py b/tests/test_gitlab_anno_mrs.py index a128ca872..3ac5d883d 100644 --- a/tests/test_gitlab_anno_mrs.py +++ b/tests/test_gitlab_anno_mrs.py @@ -4,6 +4,7 @@ from tests.cli_runner import assert_failure, assert_success, launch_command, rewrite_branch_layout_file from tests.git_repository import (add_remote, amend_commit, check_out, commit, create_repo, create_repo_with_remote, delete_branch, new_branch, push, remove_remote, reset_to, set_git_config_key, wait_to_bump_commit_timestamp) +from tests.mockers_code_hosting import mock_from_url from tests.mockers_gitlab import MockGitLabAPIState, mock_gitlab_token_for_domain_fake, mock_mr_json, mock_urlopen @@ -193,6 +194,62 @@ def test_gitlab_anno_mrs_no_remotes(self) -> None: remove_remote() assert_failure(["gitlab", "anno-mrs"], "No remotes defined for this repository (see git remote)") + def test_gitlab_anno_mrs_targets_base_project(self, mocker: MockerFixture) -> None: + # In a fork workflow the MRs are hosted by the target (upstream) project, not the source (fork) project. + # Reading commands must therefore honor `machete.gitlab.baseRemote` and query the target project for MRs. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.gitlab.GitLabToken.for_domain', mock_gitlab_token_for_domain_fake) + # Project id 2 (-> example-org/example-repo) is the source/fork project holding the branches, + # while project id 3 (-> example-org/example-repo-1) is the target project that hosts the MRs. + gitlab_api_state = MockGitLabAPIState.with_mrs( + mock_mr_json(number=1, head='feature', base='develop', repo_id=2, base_repo_id=3, user='gitlab_user'), + mock_mr_json(number=2, head='develop', base='master', repo_id=2, base_repo_id=3, user='gitlab_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(gitlab_api_state)) + + # `origin` (-> example-org/example-repo) is the source/fork remote holding the branches, + # while `upstream` (-> example-org/example-repo-1) is the base remote that hosts the MRs. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push() + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tdevelop\n\t\tfeature") + + # Without any base config the client targets the source project (origin), which does not host these MRs. + launch_command("gitlab", "anno-mrs") + assert_success( + ["status"], + """ + master + | + o-develop + | + o-feature * + """ + ) + + # `machete.gitlab.baseRemote` points reading commands at the target project that actually hosts the MRs. + set_git_config_key("machete.gitlab.baseRemote", "upstream") + launch_command("gitlab", "anno-mrs") + assert_success( + ["status"], + """ + master + | + o-develop MR !2 + | + o-feature * MR !1 + """ + ) + def test_gitlab_anno_mrs_multiple_non_origin_gitlab_remotes(self) -> None: create_repo() add_remote("origin-1", "https://gitlab.com/tester/repo_sandbox-1.git") @@ -206,3 +263,59 @@ def test_gitlab_anno_mrs_multiple_non_origin_gitlab_remotes(self) -> None: machete.gitlab.domain, machete.gitlab.namespace, machete.gitlab.project, machete.gitlab.remote """ # noqa: E501 ) + + @staticmethod + def gitlab_api_state_for_test_anno_mrs_retrieve_only_mine() -> MockGitLabAPIState: + return MockGitLabAPIState.with_mrs( + mock_mr_json(number=7, user='some_other_user', head='allow-ownership-link', base='develop'), + mock_mr_json(number=31, user='gitlab_user', head='call-ws', base='develop'), + mock_mr_json(number=37, user='gitlab_user', head='develop', base='master') + ) + + def test_gitlab_anno_mrs_retrieve_only_mine(self, mocker: MockerFixture) -> None: + self.patch_symbol(mocker, 'git_machete.gitlab.GitLabToken.for_domain', mock_gitlab_token_for_domain_fake) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(self.gitlab_api_state_for_test_anno_mrs_retrieve_only_mine())) + + create_repo_with_remote() + new_branch("master") + commit("master commit") + push() + new_branch("develop") + commit("develop commit") + push() + new_branch("allow-ownership-link") + commit("Allow ownership links") + push() + check_out("develop") + new_branch("call-ws") + commit("Call web service") + push() + check_out("master") + set_git_config_key('machete.gitlab.remote', 'origin') + set_git_config_key('machete.gitlab.namespace', 'tester') + set_git_config_key('machete.gitlab.project', 'repo_sandbox') + set_git_config_key('machete.gitlab.retrieveOnlyMyMergeRequests', 'true') + body: str = \ + """ + master + develop + allow-ownership-link + call-ws + """ + rewrite_branch_layout_file(body) + + # With `retrieveOnlyMyMergeRequests` set, only MRs authored by the current user (gitlab_user) are downloaded and annotated; + # allow-ownership-link (MR !7 by some_other_user) is deliberately left unannotated. + launch_command('gitlab', 'anno-mrs') + assert_success( + ["status"], + """ + master * + | + o-develop MR !37 + | + o-allow-ownership-link + | + o-call-ws MR !31 + """ + ) diff --git a/tests/test_gitlab_retarget_mr.py b/tests/test_gitlab_retarget_mr.py index 17d56e451..cb9030c24 100644 --- a/tests/test_gitlab_retarget_mr.py +++ b/tests/test_gitlab_retarget_mr.py @@ -505,3 +505,46 @@ def test_gitlab_retarget_mr_root_branch(self, mocker: MockerFixture) -> None: "Branch master does not have a parent branch (it is a root) even though there is an open MR !15 to root.\n" "Consider modifying the branch layout file (git machete edit) so that master is a child of root." ) + + def test_gitlab_retarget_mr_infers_target_project_from_parent_tracking(self, mocker: MockerFixture) -> None: + # In a fork workflow the MR is hosted by the target (upstream) project, not the source (fork) project. + # Even with no base* config keys, retarget-mr infers the target project from the parent branch's tracking remote + # (just like create-mr does), so it queries the project that actually hosts the MR. + self.patch_symbol(mocker, 'git_machete.code_hosting.OrganizationAndRepository.from_url', mock_from_url) + self.patch_symbol(mocker, 'git_machete.gitlab.GitLabToken.for_domain', mock_gitlab_token_for_domain_fake) + projects = { + 2: {'id': 2, 'namespace': {'full_path': 'example-org/example-repo'}, 'name': 'example-repo', + 'http_url_to_repo': 'https://gitlab.com/example-org/example-repo.git'}, + 3: {'id': 3, 'namespace': {'full_path': 'example-org/example-repo-1'}, 'name': 'example-repo-1', + 'http_url_to_repo': 'https://gitlab.com/example-org/example-repo-1.git'}, + } + gitlab_api_state = MockGitLabAPIState( + projects, + mock_mr_json(number=1, head='feature', base='master', repo_id=2, base_repo_id=3, user='gitlab_user')) + self.patch_symbol(mocker, 'urllib.request.urlopen', mock_urlopen(gitlab_api_state)) + + # `origin` (-> example-org/example-repo) is the source/fork remote holding the branches, while + # `upstream` (-> example-org/example-repo-1) is the base remote that hosts the MR. The parent branch (develop) + # tracks `upstream`, so the target project is inferred from it - no machete.gitlab.base* key is set. + create_repo_with_remote() + upstream_path = create_repo("remote-1", bare=True, switch_dir_to_new_repo=False) + add_remote("upstream", upstream_path) + + new_branch("master") + commit() + push() + new_branch("develop") + commit() + push(remote="upstream") + new_branch("feature") + commit() + push() + rewrite_branch_layout_file("master\n\tdevelop\n\t\tfeature") + + assert_success( + ['gitlab', 'retarget-mr'], + "Switching target branch of MR !1 to develop... OK\n" + ) + mr = gitlab_api_state.get_mr_by_number(1) + assert mr is not None + assert mr['target_branch'] == 'develop'