[JENKINS-75902] Always save a timestamp of the latest GitLab project configurations into the config, signalling this plugin has done some work

[Pitxyoki]

This change continues the work done in jenkinsci#495.

We found that this plugins' GitLabSCMSource#retrieve() methods are called when the job-dsl-plugin's jobDsl() method is called.
If at those points the code in this plugin fails to communicate with GitLab, the config.xml file corresponding to the pipeline being parsed is saved without any information that would come from this project's GitLab project (i.e.: sshRemote, httpRemote,
projectId will not exist). The contents of that file will be the same as if this plugin hadn't executed.

When the jobDsl() method is called again at a later time, it compares the saved config.xml to the config.xml it expects to save at that point. See https://github.com/jenkinsci/job-dsl-plugin/blob/master/job-dsl-plugin/src/main/java/javaposse/jobdsl/plugin/JenkinsJobManagement.java#L453.
Note that at that point, the new expected config.xml that the job-dsl-plugin uses for that diff does not have any information that should come from the gitlab-branch-source-plugin.
As such, at that point both the base config.xml as well as the expected-to-save config.xml are equal.

This change adds a new field to the pipeline's configuration, lastRetrieveTimestamp, that is updated at every attempt to update it, regardless of the GitLab communication having succeeded or not. In this way, the attempted diff by the job-dsl-plugin will always trigger an update and corresponding reattempt at communicating with GitLab, preventing previous failures from avoiding that.

Testing done

Similarly to jenkinsci#495, we deployed a patched gitlab-branch-source-plugin with both that and this change into our running Jenkins. We have not seen the issue reoccurring and will keep on observing it in the next weeks. We will notify if we see this wasn't effective.

Submitter checklist

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests that demonstrate the feature works or the issue is fixed

[jetersen]

do you want to point this branch onto the jenkinsci project or merge it into your existing PR? 😅

[Pitxyoki]

do you want to point this branch onto the jenkinsci project or merge it into your existing PR? 😅

Hello @jetersen!

This PR was an initial experimentation that should be merged only after jenkinsci#495 was merged. Can you please take a look at that, by the way? 🙂

I will be moving this to target the jenkinsci project shortly.

[jglick]

This change adds a new field to the pipeline's configuration, lastRetrieveTimestamp, that is updated at every attempt to update it

I am not sure I follow the issue with job-dsl, but in general this sounds wrong. An SCMSource should never attempt to update its own configuration.

the config.xml file corresponding to the pipeline being parsed is saved without any information that would come from this project's GitLab project (i.e.: sshRemote, httpRemote, projectId will not exist)

This also sounds suspicious. Running retrieve should not have modified SCMSource configuration. If there is some distinction between actual configuration—what the user enters in the GUI or JCasC or job-dsl or POSTing to config.xml or using the update-job CLI command etc. etc.—and some information that the plugin would like to retain from one branch indexing to the next, it must be kept elsewhere.

[Pitxyoki]

This also sounds suspicious. Running retrieve should not have modified SCMSource configuration. If there is some distinction between actual configuration—what the user enters in the GUI or JCasC or job-dsl or POSTing to config.xml or using the update-job CLI command etc. etc.—and some information that the plugin would like to retain from one branch indexing to the next, it must be kept elsewhere.

Hello @jglick, thanks for your input!

If I recall correctly, this plugin does modify the SCMSource configuration on retrieve. See the current code in master here: https://github.com/jenkinsci/gitlab-branch-source-plugin/blob/50978dcaefb9dc89aa410aa869b744574a7e0ddd/src/main/java/io/jenkins/plugins/gitlabbranchsource/GitLabSCMSource.java#L625

This finally block is always called, including when calls inside the retrieve method fail due to connection errors to GitLab. When that is the case, this GitLabSCMSource's class members are all set to the default values (null / 0), and the config.xml file is saved according to that.

I was very wary of changing that behaviour, as I am not aware of its side-effects. Do you think a better fix should be to address that instead? How?

[jglick]

It is hard to follow the history without being very familiar with this plugin, but it looks like jenkinsci#53 may have been the first time that retrieve set a non-transient field (jenkinsci#65 was only cementing the mistake). It looks like this PR would make matters worse.

Again, I am not very familiar with job-dsl or what it is doing comparing config.xml diffs, but the advice is simple: a Describable should not edit its own configuration (i.e., non-transient fields) at runtime.