Sync: update portability page

[josephjclark]

This PR adds a v4 Portability spec

This spec is built around a few key ideas:

• The project spec file is a data structure which defines workflows. It is usually serialized to yaml, but we must not think of it as a yaml structure. That's just a serialisation format.
  • The unit of interopability is a project spec file. If you want to share or move a project, a spec file is what you need
  • Llghtning can take a project spec and import it into a Project
  • The CLI can take a spec and expand it to the file system, and execute it
• The CLI deploy, execute, compile and project commands must all use exactly the same project spec file as Lighting's importer and exporter
• A "workflow yaml" file is just the workflow part of the spec. So CLI and Lightning must understand the same structure of workflow.yaml files
• State files are NOT part of the portability spec. They are a proprietary artefact of openfn sync.
• And so the provisioner API, which uses state.json as an interopability mechanism, is entirely unaffected by this.
• But by the way, state files are just a spec file PLUS some extra metadata (and I'm pretty sure this is what's implemented right now, although I might want to harden that concept)

It assumes that work will be done in Lightning and Kit to guarantee this interopability.

Things I'm thinking about

• Rename project spec to project source

Still to do:

• Take another pass and finish things off
• Add some notes about execution and compilation (but refer to other pages as much as possible)
• Find a better, more canonical example of project.yaml
• Move the legacy docs in to a v3 section on the history page

Notes

• project.yaml is the unit of portability. I don't think that's really true now? kinda true. A good goal.

• portability and cli should link to a seperate doc which describes the project.yaml structure

• include an exampl project yaml

• the workflow.yaml spec is not the same

• think a little but more about how we link to workflow.yaml

• in v2 the project.yaml file is the state

• and in v2 the dir structure is the project spec

• so the portability spec in v2 isa folder and a zip. that file structure is the spec

• in v3 the unit of portability was the project.yaml file

• in v4 the unit of portability is the folder. It's git.

• portability spec is core to merging sandboxes, deploying projects, storing on git, socialising ans sharing, using the cli. It's core to the definition of a project. github sync uses the portability specification. deploy uses the portability spec. Downloading a project locally to debug uses the portability spec.

• target audience is a buyer: openfn is not locked in. Portability is instrumental to design.

• link to a page which explains how to run a project with vanilla node.js

Do go off and think about workflow.yaml. It is super uncomfortable thats ints incompatible with lightning, How do we fix it? DO I just give up and use the lightning format? But it's so hard to reason about in yaml...

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to
know!):

• I have used Claude Code
• I have used another model
• I have not used AI

You can read more details in our
Responsible AI Policy