Why is it `(dtor (func n))` instead of `(dtor (core func n))`?

[bvisness]

The text explainer for resource types says the following:

resourcetype  ::= (resource (rep i32) (dtor <funcidx>)?)

And the binary explainer says the following:

resourcetype  ::= 0x3f 0x7f f?:<funcidx>?      => (resource (rep i32) (dtor f)?)

A signature of i32 => [] implies that the destructor should be a core func, not a component func. If so, I would expect both grammars to use core:funcidx, which is what you see for e.g. realloc: (realloc <core:funcidx>), 0x04 f:<core:funcidx> => (realloc f).

That said, I am then further confused because the realloc examples in the explainer all use (realloc (func $libc "realloc")). I assume this is an inlinealias, but is there really no distinction between a func and core func when performing an inline alias? If I understand correctly, an explicit alias would look like (alias core export $libc "realloc" (core func)), while the implicit alias apparently is (func $libc "realloc"). I don't understand this inconsistency.

Regardless of (func vs. (core func in the text format, though, should the formal grammars for resourcetype indeed be changed to core:funcidx?

[bvisness]

To further hammer on the inconsistency:

(component
  (core module $m
    (func (export "dtor") (param i32))
  )
  (core instance $i (instantiate 0))
  (alias core export $i "dtor" (core func $dtor))

  (type $node (resource (rep i32) (dtor (func $dtor))))    ;; why is this (func $dtor)...
  (type $nodefunc (func (param "n" (borrow $node))))

  (func (type $nodefunc) (canon lift (core func $dtor)))   ;; when this is (core func $dtor)?
)

[lukewagner]

Good catches! Filed #649 for the simpler core:funcidx grammar change.

As for the text format inconsistencies, I think you're right:

1. Given the grammar rule (dtor <core:funcidx>), I think by usual WAT rules you should be able to write (dtor $d) instead of (dtor (func $d)) -- the latter would only be necessary/allowed if the rule was (dtor <core:sortidx>) where we don't know if it's a func or something else. Agreeing with this, (callback <core:funcidx>) seems to already allow (callback $cb) in wasm-tools (but also (callback (func $cb)) too...).

2. For canon lift, the (core func ...) comes from the grammar rule saying (canon lift core-prefix(<core:funcidx>) ...), where core-prefix forces a (core prefix. That being said, if <core:funcidx> can be parsed as an $identifier, core-prefix doesn't even really make sense to use here b/c there's no func token to prefix with (core. So I wonder if we should change the grammar rule to (canon lift <core:funcidx> ...) so that you can just write (canon lift $foo ...).

(Incidentally, the above is independent ofsortidx and inlinealias, both of which always requires an explicit (sort ...).)

WDYT @alexcrichton? Obviously, we don't want to disable existing syntax (without some slow deprecation process), but does additionally accepting (dtor $f) and (canon lift $f ...) make sense?

[bvisness]

Thanks!

(Incidentally, the above is independent ofsortidx and inlinealias, both of which always requires an explicit (sort ...).)

How does this apply to the realloc example, where things currently are (realloc (func $libc "realloc"))? The explicit sort makes sense, but not the fact that it accepts func instead of core func.

[alexcrichton]

Personally I'm fine changing the text format whenever insofar as breakage isn't the end of the world, it primarily only affects tests and that's pretty isolated to just us-as-developers-of-the-component-model-itself.

Otherwise though I sort of forget at this point where we were going with the text format. IIRC there was a small push to change all $foo to (kind $foo) to enable auto-aliasing via (kind $instance "the-export-name") and that was some of the rationale here. IIRC we also wanted to default everything to (core thing ...) unless it was already within a (core other-thing ...) or "obviously in a core context" or... something like that. I don't think that ended up with everything being perfectly aligned though (as seen here).

It's kinda nice to be able to inline-alias, so would accepting $foo, (func $foo), and (func $instance "foo") all at the same time be reasonable?

As for core func vs func ... I sort of like the rule of "if you're in (core ...) don't need to use core again", which would mean that (realloc ...) would need to switch to using (core func ...) which is a bit redundant, but perhaps easier to remember? TBH I don't feel too strongly here either way.

[lukewagner]

@bvisness The (realloc (func $libc "realloc")) follows from the "if you're in a core context (because you're already parsing core stuff or the grammar says you're parsing a <core:Xidx>), you don't have to say core" rule that Alex mentioned, which I also prefer. Similarly, this would allow you to write (canon lift (func $instance "foo") ...) and (dtor (func $instance "foo")) without (core func ...), b/c each of those targets a <core:funcidx>.

@alexcrichton My understanding is that if we're following Core Wasm text format rules, if we're parsing a <funcidx>, the parser would accept $foo, but not (func $foo). And if we're parsing a <sortidx>, the parser would accept (func $foo), but not $foo. And then in both cases, the inlinealias rule lets us parse (func $instance "foo"), because inlinealias wants an explicit sort for unambiguous parsing. If we wanted to, we could also say that (func $foo) is also allowed for <funcidx>, even though the func is totally superfluous, just to minimize annoyance, but that'd be independent of the Core Wasm rules and the inlinealias sugar.

[alexcrichton]

That makes sense yeah, and I mostly just get everything mixed up and forget what's what. For example <sortidx> makes sense to be required in some places, such as (export "foo" (kind $thing)) where there's no other way to know. On the other hand for canon lift and realloc, as described here, there's in theory no need to support (func $foo) at all (or core func) as it's unambiguously always a core function.

In some sense on one hand it does seem a bit odd to allow all three forms "just because", but on the other hand it means you never have to remember what form is accepted where. Allowing just $foo and (kind $instance "thing") but not (kind $foo) feels a bit odd too insofar as the experience of remembering what-goes-where is kind of weird. Just allowing $foo feels like a mild regression from the status quo today as well where it's pretty nice to be able to quickly alias something in the text format (which is already quite verbose).

I realize that I'm a bit afield of the original question here. The answer to the original question in my head is "we should probably not require core func in canon lift" since it fits the pattern of "enclosed in a thing that requires core things". The broader question of "why require (func $foo) over $foo is something I've sort of injected here...

[lukewagner]

Having written a bunch of WAST recently, I also often forget whether I write $foo or (func $foo) or (core func $foo) in various cases, so I think I agree with allowing the superfluous (X $id) for Xidx.

So if that sounds good, to check, is the goal state:

• Xidx is parsed as $identifer or (X $identifier) or (X $instance "export")
• core:Xidx is parsed as $identifer or (X $identifier) or (X $instance "export") (the core is implied)
• sortidx is parsed as (X $id) or (core X $id) or (X $instance "export") or (core X $instance "export")
• core:sortidx is parsed as (X $id) or (X $instance "export") (the core is implied)
• remove core-prefix(...) from canon lift, leaving just <core:funcidx>; the rules above apply as usual

[bvisness]

• core:Xidx is parsed as $identifer or (X $identifier) or (X $instance "export") (the core is implied)
• core:sortidx is parsed as (X $id) or (X $instance "export") (the core is implied)

I don't like this, because it's not always clear to me when I am or am not in a "core context". I'm sure it is when you have lived and breathed the component model for years, but this whole issue was prompted by me not knowing whether I was in a core context or not! The examples (which omitted "core") only muddied the waters.

Also, I know it's not 100% reliable, but I find myself wanting to do things like grepping for (func or (core func to find what is in a particular index space. If you're going to go to the effort of typing the redundant (core? sort ...) around your identifier anyway, how much effort are you really saving by making core optional? I would prefer core to always be required whenever a core sort is required.

I think the other three rules sound good.

[alexcrichton]

One possibility: we define "core context" to be "something containing this is (core ...). For example a (core module ...) naturally only uses (func ...) internally, but we'd also say that (core instance ...) wouldn't need (core func ...) for the arguments it references. Everywhere else, though, would require (core ...). For example canon lift would require (core func ...) and (realloc ...) would also require (core func ...) since they're not otherwise contained in a (core ...).

As I say this though I realize the waters are a bit muddied with (core func $thing (canon lower ... (realloc (func $thing2)))) where you're "in a core thing" but realloc doesn't need core func, so maybe this isn't a good idea after all.

At minimum though the rules you outline @lukewagner all sound like a good starting point, and for the question about requiring core on core:sortidx and core:Xidx I'm personally ambivalent about

[lukewagner]

Ok, yes, I suppose it does seem less surprising to always see (core func ...) unless you're inside a (core ...) expression already. For the corner case of (core func $thing (canon lower ... (realloc ...))), since this is just an abbreviation (syntactic rewriting into) (canon lower ... (realloc ...) (core func ...)), I think we can interpret it as the latter and thus realloc needs a (core func ...).

So then iiuc the revised goal state would be:

• when inside (core module|instance|type ...):
  • core:Xidx is parsed as $id or (X $id) or (X $instance "export")
  • core:sortidx is parsed as (X $id) or (X $instance "export")
• otherwise:
  • Xidx is parsed as $id or (X $id) or (X $instance "export")
  • core:Xidx is parsed as $id or (core X $id) or (core X $instance "export")
  • sortidx is parsed as (core? X $id) or (core? X $instance "export") (core required for core sorts)
  • core:sortidx is parsed as (core X $id) or (core X $instance "export")
• remove core-prefix(...) from canon lift, leaving <core:funcidx>, which accepts $id, (core func $id) or (core func $instance "export")

[alexcrichton]

It's going to cause a lot of churn, but yeah that seems ok to me. I'll probably do something like:

• Implement support for the new form
• Add an env var to accept the old form, defaulting to "accepted"
• Eventually switch the env var to "stop accepting"
• Eventually delete the env var and support for old code

Basically this affects so many *.wat/*.wast files by default I don't want to cause too too much churn and force it to all happen at once.