Improve project validation and add project cross checks

[gpetretto]

Following #483, this PR adds a check to prevent having the same jobs_handle_dir for two batch workers on the same host.
As a general improvement, also added a jf project check-conflicts that will verify this and a few other incompatibilities among different projects. This is not executed when the projects are read, so it needs to be called explicitly. I don't think it would be good to enfornce it and having the need to perform all the checks every time.

Also, basically unrelated, I have realized that the case OnMissing.PASS from jobflow was not handled at all in jobflow-remote. This should fix it.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 89.09091% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.13%. Comparing base (e3c85bc) to head (fc485a5).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/jobflow_remote/config/helper.py	87.09%	4 Missing and 4 partials ⚠️
src/jobflow_remote/cli/project.py	89.65%	1 Missing and 2 partials ⚠️
src/jobflow_remote/remote/host/separated.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #484      +/-   ##
===========================================
+ Coverage    75.92%   76.13%   +0.20%     
===========================================
  Files           52       52              
  Lines         8162     8271     +109     
  Branches      1345     1373      +28     
===========================================
+ Hits          6197     6297     +100     
- Misses        1525     1528       +3     
- Partials       440      446       +6     

Flag	Coverage Δ
all_local_tests	75.94% <89.09%> (+0.21%)	⬆️
all_tests	76.13% <89.09%> (+0.20%)	⬆️
db_tests	71.70% <85.45%> (+0.21%)	⬆️
integration_local_tests	45.70% <22.72%> (-0.31%)	⬇️
integration_remote_tests	26.05% <14.54%> (-0.16%)	⬇️
integration_tests	46.56% <22.72%> (-0.32%)	⬇️
unit_tests	28.17% <64.54%> (+0.82%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/jobflow_remote/config/base.py	87.33% <100.00%> (+0.82%)	⬆️
src/jobflow_remote/jobs/jobcontroller.py	81.63% <100.00%> (ø)
src/jobflow_remote/remote/host/local.py	81.81% <100.00%> (+0.42%)	⬆️
src/jobflow_remote/remote/host/remote.py	69.39% <100.00%> (+0.40%)	⬆️
src/jobflow_remote/remote/host/separated.py	93.10% <50.00%> (-1.54%)	⬇️
src/jobflow_remote/cli/project.py	85.09% <89.65%> (+0.58%)	⬆️
src/jobflow_remote/config/helper.py	79.18% <87.09%> (+3.63%)	⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[davidwaroquiers]

All good to me, you can merge as is or change the small thing about 1 project / No project.
Do you think the check-conflicts should be added to the documentation maybe somewhere ? (one question also, maybe when doing jf project check, it should check conflicts with other projects no ?)

[davidwaroquiers]

This would either be "Only 1 project parsed" or "No project parsed". Not super important but maybe change that ?

[gpetretto]

good point. Will do.

[gpetretto]

All good to me, you can merge as is or change the small thing about 1 project / No project. Do you think the check-conflicts should be added to the documentation maybe somewhere ? (one question also, maybe when doing jf project check, it should check conflicts with other projects no ?)

I see the point here. I will add it to the docs. I am not fully convinced of adding this to add it to the jf project check. Maybe a warning that one might consider running jf project check-conflicts to check potential conflicts with other projects?

[davidwaroquiers]

All good to me, you can merge as is or change the small thing about 1 project / No project. Do you think the check-conflicts should be added to the documentation maybe somewhere ? (one question also, maybe when doing jf project check, it should check conflicts with other projects no ?)

I see the point here. I will add it to the docs. I am not fully convinced of adding this to add it to the jf project check. Maybe a warning that one might consider running jf project check-conflicts to check potential conflicts with other projects?

For the addition to the project check, I don't know either, it's just that it probably does not take much time to do. Maybe it's done when you do a full project check (using jf project check --full) ? Why are you not convinced ? (I can't say exactly myself why I'm not :D)

[gpetretto]

mainly because it is not obvious that one wants that check to be done and may be somehow confusing. The description says

Check that the connection to the different elements of the projects are working.

not that it will make a validation. That is in principle more for jf project list. But I am not really convinced of having this additional message there either. Mainly bacause if you have many projects the output can be already a bit messy on its own. Maybe changing this entirely and have it as an option for jf project list and remove jf project check-conflicts?

[davidwaroquiers]

mainly because it is not obvious that one wants that check to be done and may be somehow confusing. The description says

Check that the connection to the different elements of the projects are working.

not that it will make a validation. That is in principle more for jf project list. But I am not really convinced of having this additional message there either. Mainly bacause if you have many projects the output can be already a bit messy on its own. Maybe changing this entirely and have it as an option for jf project list and remove jf project check-conflicts?

Yes true, that might be misleading or we should change the description at least. I let you decide what you think is best. Maybe I would change the description to something like : "Check that the connection to the different elements of the project are working and checks potential conflicts with other projects". But if you prefer to keep it just as such it's fine too. It's just that in that case we are not adding an additional cli just for that (but again I let you decide)