visibility design and discussion - #187
Conversation
|
This looks very solid to me and would definitely bring a big usability gain to libraries like lygia. The one thing that doesn't seem be discussed in much detail is
It seems like the docs here don't explain why this approach is chosen over an
Personally I don't have a strong opinion either way, but there is lots of prior art for the second option (most JS package systems, Python, Rust, Lua) while I can't think of anything that has this strict root/non-root separation off the top of my head. |
|
I believe that we should be able to dig up the Rust recommendations for that. IIRC, Rust discourages the For example, if I have 3 tabs open in VSCode and they all say |
|
Also quick and very inconvenient note: |
|
Without wildcards, resolving an import means going to the right module and finding the item. With wildcards, resolving an import means looking through a set of modules to find the right item. With wildcard re-exports, resolving an import means looking through a set of modules recursively to find the right item. And since modules can have cycles, one now needs to keep track of which modules have been visited during import resolution. In the language server, this is a fair amount of extra complexity. This particular feature also, I believe, interacts really badly with macros or other compile time code generation. Rust's import system ends up being a very complex fixed point finding algorithm which may sometimes simply fail. (I think it also interacts with parameterized modules. Since wildcard reexports is the missing piece to make import statements recursive. If imports can cause any sort of computations, which compile time generated code does, and which parameterized modules do, then one gets the halting problem. Or so the story goes.) Usually I wouldn't object to complexity too much. |
Not sure, always interesting to look around tho! Here I was only filling the gap where our current scheme doesn't allow something at the root level. I think small libraries especially might like to enable their users to It'd be a separate issue, but your comments about consistency... I suppose we could revise the existing rules so that |
wat!? good catch. Perhaps |
Worth thinking about. I like the idea of implementing the needed features w/o gilding the lily too much. The cycles/complexity concerns seem heavy for a feature that isn't must-have. And foreclosing options for parameterized modules would be unfortunate. Also I realize that the PR doesn't address some related issues:
I'm thinking best would be to pull wildcard re-exports from this PR. Side note, if we go with the publish tool route as planned with #183, I suppose we could also expand wildcard re-exports like we're planning for wildcard imports. That would help for the cross package cases. |
|
pushed a draft
|
|
We could also go with
Also, what does private mean? Rust goes all in on the hierarchical module system and says
|
I like this option too, a bit more cryptic but if perhaps best if we prefer keywords to attributes.
I like I suppose we could do
just the module, no hierarchy. we could expand hierarchically later, it's a nice idea. But I think of most WESL programs as smallish, so perhaps unnecessary complexity, better to defer. |
|
For the
|
Ah, that was my first instinct too. I like modularity. But I've come to think that package by default for WESL is better. See |
|
Also interesting tidbit: The other style is the global style, where I use 3 access modifiers. Public, internal (protected) and private. And apparently some of the very prolific Rust contributors, including epage who maintains a bunch of extremely well written libraries, vastly prefer the global style. So we are on the right track! https://www.reddit.com/r/rust/comments/1k5yv6p/two_ways_of_interpreting_visibility_in_rust/ |
|
More from Rust land:
They also include statistics which says that |
|
And from Kotlin lang: They went with public by default, only to realize that it might have been a mistake. So defaulting to a more restrictive choice like "internal" seems wise |
|
Note from Java: |
|
We really should ask the WGSL committee regarding the |
|
I dug into it and asked the WebGPU commitee gpuweb/gpuweb#6264 I couldn't quite figure out where it got lost though. It seems to have happened quite a while ago, judging from Naga's source code |
|
I talked to a friend regarding this. |
Fixes to wesl.toml
replace the left-to-right segment walk with direct resolution:
the last path segment names the declared item, the preceding segments name the module.
declarations and modules naturally live in separate namespaces now
- `fn foo() {}` and `foo::bar();` do not conflict
lazy vs eager resolution is an implementation choice
clarify terms a bit
Conflict resolution per 193-into-187 merge checklist: - keep the public import re-export paragraph in the grammar section; take direct-imports' module attribute wording - replace the old import walk algorithm with the new 'Resolving a declaration path' model from direct-imports - reattach public import re-exports to item lookup in the new resolution section - carry over the resolution-vs-visibility paragraph and the wildcard visibility-skip rule after the resolution rules - point Filesystem Resolution at the new resolution section - keep the Visibility.md re-exports link in the wildcardable recommendations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add wildcard imports
- describe super:: flattening early, so resolution later can be clearer - write package module generically w/o being fs specific - drop wesl.toml mini-section (we have a whole page on it now) - Cleanup fs resolution section logic for direct imports - Add brief section on non-fs resolution
- demote the "tools can enumerate resolutions" paragraph to a [!NOTE] - clarify intermediate paths don't need to be modules
direct import resolution (more editorial fixes coming with #187)
- tighten visibility terminology and scope the summary table - clarify root selection, static access, and defaulted override behavior - explaine rationale for root module determines pipeline visible API - correct WGSL terminology and the @!wildcardable example - sharpen the rationale for visibility levels, syntax, and deferred extensions
|
I am happy with this direction. A proper review will come later. So my preferred option would be reviewing and merging this. Afterwards we create a draft proposal for private-by-default, to explore what that would imply. For example, a truly good private-by-default proposal would likely involve extending the And the other thing that I'd like to explore is: What is the most common visibility in any given project. Edit: We talked about this today, the strongest signal will simply be observing what our users do. For example, if people start writing |
We should feel good about the direction, knowing that we can course correct while the language and community are young.
Right that's a hurdle, e.g. for potential upstreaming into WGSL.
Yeah, having something for Bevy was the original motivation for this. In general visibility control is important for libraries. Also presuming that we don't end up with public by default, we'll either break compatibility or commit to dealing with maintaining migration capability in the tools. So good to get something in soon while there's little to break. More recently, I've realized I would like to use visibility soon for the wgsl-play stuff too (e.g., to make clean root modules via re-exports).
Just noting we want to look at a breadth of projects / use cases. The design goal I think about most often is how to keep things simple for the common / new user, while enabling power features for the sophisticated. |
k2d222
left a comment
There was a problem hiding this comment.
Looking good! a bunch of editorial and nitpicks comments, otherwise there is just one thing I want to defend: re-exporting package-visible declarations, see the comment about Lygia.
Also, make it clear to readers that app users need not to worry about public it's a good selling point of the design!
| import my_lib::prelude::*; // wildcard pulls all of the above | ||
| ``` | ||
|
|
||
| A bare `import` brings names into local scope without re-exporting them. |
There was a problem hiding this comment.
editorial: with a private visibility. i.e. it behaves like a private local declaration.
There was a problem hiding this comment.
tried a fix in the latest draft (tho different wording w/o the private analogy)
| declarations in the root module and follows references transitively. | ||
|
|
||
| An `import` statement is not itself a static access. An imported item is | ||
| statically accessed when an identifier referring to it appears in a body, type, |
There was a problem hiding this comment.
editorial: ...appears in a statically accessed body... or attribute of a statically accessed declaration.
There was a problem hiding this comment.
revised in latest draft
| initializer, or attribute. | ||
|
|
||
| Pipeline-visible items keep their exposed names in the linked WGSL output. A | ||
| root-declared item keeps its declared name. An item re-exported by `public |
There was a problem hiding this comment.
editorial: line jump messes up formatting
There was a problem hiding this comment.
revised in latest draft
| it. A wildcard import omits items that are not visible to the importer. A name | ||
| collision between two wildcard imports is therefore an error only when both | ||
| items are visible (see [Wildcard imports](Imports.md#wildcard-imports)). | ||
|
|
There was a problem hiding this comment.
we should also reference that fact in wildcard-imports
| Naming a less-visible type in a `public` signature or field is permitted with no | ||
| diagnostic. | ||
|
|
||
| ## Re-exports |
There was a problem hiding this comment.
can you re-export from "parent" modules? sounds like an anti-pattern, but I'm not sure.
There was a problem hiding this comment.
Nothing stops it, agree seems likely bad style, good for a linter rule.
|
Cf. #202 (comment), We should treat this case in the spec: private struct foo { ... }
public alias bar = foo; // is bar public? can I access its fields?I think i this case |
Good one. I agree we'll want something so that APIs can return structs with hidden internal details. I think it deserves its own issue. Filed #204.
I'll update to make the alias an error in this PR while we consider what we want in #204. WGSL alias semantics say that aliases don't create new types. Should they create new visibility? Maybe, but there are other approaches to opaqueness that I suspect we'll like better.. |
| An alias whose target resolves through any chain of aliases to a declared | ||
| structure type must not be more visible than the structure declaration. A WGSL | ||
| alias introduces another name for the same type and preserves its value | ||
| constructors and members. A more-visible alias would therefore make the type | ||
| nameable and constructible outside its visibility boundary (see | ||
| [A re-export cannot widen visibility](#a-re-export-cannot-widen-visibility)). | ||
| The visibility of an intermediate alias does not constrain a later alias. For | ||
| example, `public alias Shared = Local;` is permitted when `Local` is a | ||
| *package*-visible alias for a `public` structure type. |
There was a problem hiding this comment.
I'm not 100% sure this is the best solution here, but we can keep it this way for now. In -rs, the 1st implementation iteration will probably allow public aliases to private structs, because checking the spec rule is harder.
There was a problem hiding this comment.
stefnotch
left a comment
There was a problem hiding this comment.
I found an edge case that we'll have to test.
// in foo.wesl
public import package::utils as GetUtils; // a module
// in package.wesl
import package::foo::GetUtils;
const a = GetUtils::SOME_CONST;
If I do it correctly, I first expand to
const a = package::foo::GetUtils::SOME_CONST;
and then I resolve.
If I do it as a two step process, I might do the incorrect
// resolves to package::utils
import package::foo::GetUtils;
// so this resolves to
const a = package::utils::SOME_CONST;
We should add that to the testsuite. And I'll have to be careful in wgsl-analyzer. W-a currently purposefully tries to do it as a two step process so that we can detect errors earlier.
| // sun_intensity not in the pipeline-visible API: host cannot set it; uses its default of 1.0 | ||
| ``` | ||
|
|
||
| ### Aggregating entry points |
There was a problem hiding this comment.
I wonder if we could design an even neater world if we start messing with the host APIs. Like "any entrypoint in any module that is public can be used, the host API just has to say which one it wants"
|
I approve of shipping this and implementing it. There's no reason to block further progress on small comments. If we come up with significantly better solutions, we can always ship a new iteration of the spec. |
TLDR
public(visible to any package),private(declaring module only), and package (same package, the unmarkeddefault).
public importre-exports the imported names under thecurrent module's path;
public import path::*does the same for a wildcard(re-uses
@wildcardablediagnostics).API: entry points, resource variables, and pipeline-overridable constants
reach the host only if the root module declares them or
public imports them.package.wesl. The file backing a package's top-level module, so itsitems are reachable as
<package>::item(e.g.,import wgsl_test::*). Oneplace a library can put a re-export prelude.
Why these four things are covered:
import wgsl_test::*;(Note that VisibilityLanguages.md is discussion background, it'll be removed before merging.)
Would resolve:
(largely covered already; this PR fills in the re-export piece)
Related