WIT should have the same expressivity power as the component model

[fitzgen]

My mental model has been that WIT is "just" syntax sugar¹ on top of the component model: anything expressible in a raw component model interface should also be expressible in WIT.

But this is not actually the case today. I strongly believe it should be.

The biggest counterexample I am aware of is #287.

My hopes in filing this issue are two-fold:

1. Build consensus that "anything expressible in a raw component model interface should also be expressible in WIT" is a highly-desirable property. Does anyone disagree? Can we formalize this into one of the design docs in this repo or something?

2. We can do a collective survey/audit of CM features and exhaustively enumerate WIT's expressivity gaps. Is anyone aware of any other counterexamples?

Footnotes

1. Curly brackets instead of S-expressions, stuff that helps bindings generators but don't affect semantics like doc comments, a package system to spread definitions across files, etc... ↩

[lukewagner]

I totally agree that full co-expressivity is a goal for WIT and think it does belong on the "todo list" for 1.0. Because this is a purely WIT-level feature, unless someone is motivated to work on it sooner (which would be great!), it would naturally go near the end, after other runtime-impacting features.

FWIW, I don't believe #287 is an example of a feature gap: raw CM types have the same constraints as WIT (name uniqueness means only one import/export with the same interfacename).

There are however a number of other gaps to fill:

• expressing non-default resource-type-sharing behavior (Split 'use' inside worlds into 'use import' and 'use export' #308) (this is the big one, and felt)
• expressing nested plain-named instance types (Nested interfaces #372)
• exposing "dependency" and "URL" names directly in WIT (Unlocked deps #393)
• importing/exporting types directly (not wrapped in an interface)
• use of plainnames, not just interface names #287

I expect there are others too. An easy way to find missing cases is to harvest every component type we can find in the WAST tests and roundtrip them through WIT.

[fitzgen]

FWIW, I don't believe #287 is an example of a feature gap: raw CM types have the same constraints as WIT (name uniqueness means only one import/export with the same interfacename).

Maybe I linked the slightly-wrong issue, there seem to be a few semi-related/similar issues there. Perhaps #526 was a better issue to link.

In raw component model WAT, I can first define and name an instance type and then import that instance type twice under two different names. For example:

(component
  (type $wasi-fs (instance (* ... *)))
  (import "local-fs" (instance (type $wasi-fs)))
  (import "network-fs" (instance (type $wasi-fs)))
  ;; ...
)

In WIT, however, IIUC, it seems like import names and import types are conflated, and the closest approximation of doing this same thing is to inline the type definition multiple times:

component {
  import local-fs: interface { /* ...copy of wasi-fs interface... */ };
  import network-fs: interface { /* ...copy of wasi-fs interface... */ };
  // ...
}

But AIUI, this workaround doesn't work with WIT's module/package system, because you can't name the instance type (rather than a particular import of that instance type with a particular import string, unlike raw CM wat) which in turn means you can't use/include/whatever just that instnance type so that you can import multiple instances of that type.

FWIW, I do think the proposed solution in #526 (comment) will resolve this use case, but I think we could avoid this kind of impedance mismatch by construction if we had the following layering/principles:

• There is a one-to-one translation from every raw component model WAT sexpr to the sugary WIT with curly braces
• WIT additionally allows splitting definitions across files/packages and allows you to use/include definitions from other files and packages (and this is not the same as an import at the Wasm level, ofc)
• WIT allows defining conventions of common import and export interfaces as worlds
• etc...

Basically, by making WIT purely syntax sugar + some more stuff layered on top of CM WAT, we will never get into a situation where CM WAT can express something but WIT cannot.

[alexcrichton]

One thing I can note is that @fitzgen the two examples you have, the text format component and the WIT, actually mean the exact same thing. The text-format component is quite subtle but the true meaning is that local-fs and network-fs have entirely independent and distinct types. For example if (type $wasi-fs ...) had a resource then the resource exported by local-fs would be distinct from network-fs. This has to do with resource type substitution and such. Put another way, the wasm-text-format you listed is effectively syntax sugar for what the WIT is saying.

Another thing I can say though is that I don't think there's any disagreement that the component model and WIT should end up being 1:1 with their expressivity. It's not like everything is intentionally mismatched today. What this requires, however, is quite a lot of work and thought on behalf of someone to basically pull together the expressivity of components to meet the real-world expectations and constraints of WIT (e.g. language bindings, bindings generators, human readable-ness, WASI requirements, etc). From my experience this is an extremely difficult role to fulfill and requires effectively total knowledge of absolutely everything involved. Any small changes in WIT can have massive impacts on how bindings generators are architected, the implementation of tooling, etc.

I want to emphasize that I agree that the end state of the world will be simpler, but I also want to emphasize that we can't just say "let's have these be 1:1" and then assume the work will get done. There's a massive amount of work (in my opinion) to bring everything together and this has to all be balanced with all the other priorities we're faced with in terms of component model features, designs, use cases, etc. It's a classic problem of we can spend a ton of time now to save ourselves even more pain in the future, but if we spend so much time fixing these problems we may not have that future in store because it delays so many other things.

From a personal note I want to point out that I tried really, really, really hard to get WIT to get as close to the component model when it was first being written/designed. I'll be the first to acknowledge it didn't land on the component model as-is, and that was a very intentional design decision (had to ship something...). The very real constraint of effort required to implement changes like "make WIT and component model coexpressive" I feel needs to be acknowledged as it's inevitably part of the calculus in making decisions about how to evolve WIT.

[fitzgen]

Another thing I can say though is that I don't think there's any disagreement that the component model and WIT should end up being 1:1 with their expressivity. It's not like everything is intentionally mismatched today.

For sure, I just wanted to get some more-formal consensus on this and eventually get it written down as a design principle in the design docs or wherever, so that in the future whenever we find ourselves adding something at the core CM WAT level, we won't forget / will know we need to also add associated syntax at the WIT level.

What this requires, however, is quite a lot of work and thought on behalf of someone to basically pull together the expressivity of components to meet the real-world expectations and constraints of WIT (e.g. language bindings, bindings generators, human readable-ness, WASI requirements, etc).

I am not expecting other people to magically do the work for me, but I do think the first step of doing that work is building consensus that it is desirable, which is a large part of why I filed this issue.

Furthermore, if we stick to that simple "when you add a raw CM feature, also expose it in its full power in WIT" rule I am proposing, then I don't think that is actually that much additional work on top of the current state of things. When particular languages want to use those features, then that will require building out e.g. the wit-bindgen tooling, but those code paths can remain return Err("not yet implemented") until a language/toolchain wants to take advantage of those new features.

The backlog of divergences that already exist before this proposed rule comes into place is the larger engineering effort/resources question AFAICT. But again, the first step is getting consensus that we want to tackle that stuff, the second step is gathering the list of everything that needs to be fixed, and only after that can we actually start to do the engineering to address it.