New structure for documentation

[TeresiaOlsson]

We have previously discussed to restructure the documentation. I have promised to take the task but it would be useful with input so I have a better idea of what new structure to build.

So far we have discussed:

• Move all documentation which requires several packages into it's own repository. This would be examples, tutorials etc. Currently it is all inside the pyaml repository.
• Keep only documentation for a specific package in it's own repository and provide a common entry point to it, for example from the https://python-accelerator-middle-layer.github.io/ webpage.

I have looked at some other projects to get ideas for how they structure their documentation and would like input on what you like/don't like:

• PreCICE: https://precice.org/
• Bluesky project: https://blueskyproject.io/

In PreCICE they have one common webpage for all documentation. They have the menu options Quickstart, Docs and Tutorials.
In the Bluesky project they have separated the documentation for different packages. They have the options Documentation where you can click on the package you are interested in and Tutorial for ecosystem wide (I guess...) tutorials.

Currently I'm leaning towards doing it the PreCICE way. My experience of the Bluesky documentation is that you go to a specific package, find a package specific tutorial in there and after a while you are confused about how the ecosystem fits together because there is no joint documentation. The idea would then be to still have package specific documentation in each repository but the main purpose of it is to link all the pieces together into one joint documentation.

[TeresiaOlsson]

I found this documentation for how PreCICE manage their documentation: https://precice.org/docs-meta-overview. It looks like they have the documentation as part of their main webpage but then import content from other repositories using git submodules.

[TeresiaOlsson]

What I don't like about that approach is that they have the ecosystem level documentation directly inside the repository for the webpage. That seems a bit messy to me because you want to rebuild the website when there is some community news and rebuild the docs when there is a new code version. And also probably not use the same template. The reponsibilites for it is also different. The steering committee is responsible for the webpage and the maintainers for the docs.

Maybe better to make a separate repository for the ecosystem docs which will publish it's own page? That page can then be linked on the organisation webpage.

[TeresiaOlsson]

For the content I think it was from @simoneliuzzo I learned about diataxis. That feels like a good approach to follow. We already tried to do it for the last workshop.

[TeresiaOlsson]

@gubaidulinvadim @GamelinAl @kparasch @JeanLucPons Do you have some opinions on this? If not, I will go ahead based on what I think and make a first implementation for people to look at.

[kparasch]

I like diataxis. A separate single repository (without submodules) for the documentation I think is the most convenient option for the "officially supported" parts of pyaml.

[JeanLucPons]

I agree diataxis is good.
It would be nice to keep in pyaml-core only the API reference.
And to move example / tutorials / getting_started to an other repo.
IMHO, it is more urgent to freeze pyaml-core (mainly CS and pyaml devices abstraction) in order to avoid big refurbishment each time otherwise the progression will be slower and slower.

[TeresiaOlsson]

I agree diataxis is good. It would be nice to keep in pyaml-core only the API reference. And to move example / tutorials / getting_started to an other repo. IMHO, it is more urgent to freeze pyaml-core (mainly CS and pyaml devices abstraction) in order to avoid big refurbishment each time otherwise the progression will be slower and slower.

According to the roadmap the hardware abstraction layer doesn't have to be frozen until the end of the year (which I think includes the CS and the pyaml devices abstraction?) but the configuration layer should be done before the summer vacations and part of that is user documentation for how to write the configuration. My thinking was to refactor the documentation now so we can add the configuration part, tell the steering committee that milestone is done and then focus on the hardware abstraction layer.

[JeanLucPons]

According to the roadmap the hardware abstraction layer doesn't have to be frozen until the end of the year (which I think includes the CS and the pyaml devices abstraction?) but the configuration layer should be done before the summer vacations and part of that is user documentation for how to write the configuration. My thinking was to refactor the documentation now so we can add the configuration part, tell the steering committee that milestone is done and then focus on the hardware abstraction layer.

The CS part of the configuration layer is dynamic. As i already said we cannot consider all use cases of all control systems and try to build a definitive schema for all. The important thing here is to fix the CS interface and keep it as simple as possible. I would say that all the configuration part under the control: field belongs to various CS backends.

Concerning the pyaml level:

• A BPM configuration simplification (for offset) is required. We have to discuss if we want to keep a model (to handle electrode to position conversion for instance). Today it is not possible to implement BBA on internal simulator.
• Configuration of additional corrector that affect a same multipole (2 power supply for 1 strength on 1 element) , i already proposed few solutions but here at esrf we are in favor of having a correction strength file (It is not my choice) we would prefer an additional field in the lattice file. We have to take a decision.
• Configuration for multi combined function magnet (n power supplies , m strengths, k elements) A very tricky point for pole face strip.

[TeresiaOlsson]

According to the roadmap the hardware abstraction layer doesn't have to be frozen until the end of the year (which I think includes the CS and the pyaml devices abstraction?) but the configuration layer should be done before the summer vacations and part of that is user documentation for how to write the configuration. My thinking was to refactor the documentation now so we can add the configuration part, tell the steering committee that milestone is done and then focus on the hardware abstraction layer.

The CS part of the configuration layer is dynamic. As i already said we cannot consider all use cases of all control systems and try to build a definitive schema for all. The important thing here is to fix the CS interface and keep it as simple as possible. I would say that all the configuration part under the control: field belongs to various CS backends.

Concerning the pyaml level:

* A BPM configuration simplification (for offset) is required. We have to discuss if we want to keep a model (to handle electrode to position conversion for instance). Today it is not possible to implement BBA on internal simulator.

* Configuration of additional corrector that affect a same multipole (2 power supply for 1 strength on 1 element) , i already proposed few solutions but here at esrf we are in favor of having a correction strength file (It is not my choice) we would prefer an additional field in the lattice file. We have to take a decision.

* Configuration for multi combined function magnet (n power supplies , m strengths, k elements) A very tricky point for pole face strip.

I agree. For me those things are part of the hardware abstraction layer and not the configuration layer. For me the configuration layer is limited to the architecture for it, how the user would use the different parts and make it sufficiently flexible to read in schemas from external packages so objects from external packages can be validated and built within the same framework as the standard pyAML implementation. I think those parts can be frozen and the configuration layer considered done while still working on settling the things on your list. Maybe @gubaidulinvadim would disagree since he needs to rewrite the configuration file all the time but at least for me the exact fields in the configuration file isn't part of the configuration layer since they depend on how the CS, devices, models etc are implemented and which attributes they need.

[TeresiaOlsson]

Then I will create a new repository called pyaml-documentation, copy the existing docs folder from pyaml there and make the necessary changes so only the API reference is built in pyaml, tango-pyaml and pyaml-cs-oa and linked into the main documentation.

[TeresiaOlsson]

We now have this Github page: https://python-accelerator-middle-layer.github.io/documentation/ which is built from https://github.com/python-accelerator-middle-layer/documentation.

I will now start moving things from the pyaml repository and replace the existing readthedocs site with another one built from this repository so we can have the similar structure as for pyAT where the latest version is available on the Github page and versions for different releases can be found on readthedocs.

[TeresiaOlsson]

I haven't yet made an explicit link on https://python-accelerator-middle-layer.github.io/. Will wait with that until the page is ready for users to read.

[TeresiaOlsson]

The new page is finished. You can see what you think and come with feedback: https://python-accelerator-middle-layer.github.io/documentation/index.html.

I concluded that probably what we want is not to pull all the API documentation from the other repositories into this with intersphinx. It's possible but felt more messy than just putting links to the latest version of their respective documentation. My plan now is to not publish this docs on readthedocs since it feels difficult to make it versioned (which package should we base the version on since in the future there might be a long list of packages which update versions somewhat independently of each other and this should be the docs for how they interact with each other?). Instead I plan that the docs for the individual packages should be published on readthedocs with specific versions and then via this page you can find earlier versions if you need.

Left to do:

• Remove the duplicate docs from the pyaml repository. The plan is to make a template for generating only API documentation which we can apply to all the packages in the same way.
• Add tutorials. I will make a separate discussion for this since it feels like we need to discuss the content of that.

[GamelinAl]

I like it a lot. The interface is simple, it's very clear on where to go depending on what you look for.
Great job, now we'll need to fill with the good content.