API: We should throw an exception when a hook is used outside of a component context

[bryphe]

We don't do any checking or handling when a hook is called outside of a render function. This is simple to reproduce - just by calling useState outside of a component.

We should add a test case that exercises this, and validates an appropriate, actionable exception is thrown.

[jchavarri]

I have been talking a bit with @cristianoc about hooks and how their rules could be encoded in the type system, and he shared with me some time ago an exploration that involved using linear types to track usages of hooks inside the render function: https://gist.github.com/cristianoc/cef37bcfc0446da482da4723dc3319a8

@bryphe I believe Cristiano's solution was targeting the detection of usages of hooks inside conditional statements. But I think it would solve the problem you mention above (usages outside render).

One downside of this approach is in "API costs": the value that contains the linearity of hooks needs to be carried over between hooks function calls, and also passed to the render function.

@bryphe I wonder what you think about this, and if it would be an approach worth exploring? An implementation in the type system has some implications, but I think it could bring a better dev experience. The implementation also would be more idiomatic than having these checks implemented with tests + runtime exceptions.

I also wonder if, considering the current design of reason-reactify there could be a way to hide the hooks value from final users, so the linearity is checked internally, kind of like how we passed the context to every dispatch without end users knowing.

[bryphe]

Cool, thanks for thinking about this @jchavarri & @cristianoc . The current model we have in this project doesn't sit right with me, just because it's too easy to get wrong and crash. We should be 'using the language' for this.

@bryphe I wonder what you think about this, and if it would be an approach worth exploring?

I really like @cristianoc 's proposal. Preserving the hooks between function calls seems reasonable to me, because it provides a clear benefit - we can leverage the type checker.

One of the major benefits of hooks, imo, is the ability to compose & reuse them - so it was helpful to see the example in the gist of how that would work.

An implementation in the type system has some implications, but I think it could bring a better dev experience.

Agreed!

So if we used a model like @cristianoc 's - we'd get better type checking in general, and it would solve this problem (actually, completely sidestep the problem) - with the minor cost of passing a hooks around (which can be easily justified). I'm on board!

[jchavarri]

I played a bit with the "hooks carrying" solution, and one small downside is that it doesn't prevent users from not passing the hooks value around, in which case the checks would not apply.

For example, in this gist the last function calls a different number of times the useState hook depending on the branch, and it compiles just fine.

This makes me wonder if we should explore an approach that combines the ideas from the linear types solution by @cristianoc but changing the API so the hooks return the values inside a callback, in a continuation passing style. Essentially, the idea expressed in https://paulgray.net/an-alternative-design-for-hooks/.

This, combined with something like https://github.com/jaredly/let-anything/ could lead to a result like:

let render = () => {
  let.Hook (count, dispatch) = useReducer(reducer, 0);

  <view>
    <button title="Decrement" onPress={() => dispatch(Decrement)} />
    <text> {"Counter: " ++ str(count)} </text>
    <button title="Increment" onPress={() => dispatch(Increment)} />
  </view>;
};

This approach would remove the need of passing the hooks value to the render function, and passing it around. The syntax sugar allows to avoid all the nesting, and the result is as expressive as the JS implementation I believe. 🤔

[bryphe]

I played a bit with the "hooks carrying" solution, and one small downside is that it doesn't prevent users from not passing the hooks value around, in which case the checks would not apply.

Ah, good catch! That would make it easy to lose the safety net we constructed around this concept.

This makes me wonder if we should explore an approach that combines the ideas from the linear types solution by @cristianoc but changing the API so the hooks return the values inside a callback, in a continuation passing style. Essentially, the idea expressed in https://paulgray.net/an-alternative-design-for-hooks/.

Great idea! The issue I had with the proposal in the article was the syntax being more cumbersome to work with (for a JS dev coming from React), and I wasn't sure how to address that. But your idea to use a PPX solves that!

I think the the let.Hook ... is very reasonable, to get the benefit of the type safety. We'd essentially just add one rule to our Rules of Hooks - use let.Hook for hooks. Benefit - you get type safety, and you might not have to care as much about the other hook rules, because we'd help enforce it for you with the type system. It's a very small tradeoff for all the value you'd get from the type system!

Do you see any downsides with the let.Hook approach, @jchavarri ? It seems like custom hooks would be supported - we'd just use let.Hook inside those too, correct?

It's pretty amazing to me that we could validate the situation you mentioned - calling useState multiple times - in such a seamless way. Really cool stuff 😄

[cristianoc]

This looks like a very interesting direction. @jchavarri would you flesh out what let.Hook would desugar to?

[jchavarri]

@bryphe Glad you're on board with the idea! 😄

We'd essentially just add one rule to our Rules of Hooks - use let.Hook for hooks.

I think with a type-based solution, there would be no more need for explicit rules anymore... one would just have to follow the type system guidance (but any additional documentation to help understand what the type system is doing always helps 🙂 ).

Do you see any downsides with the let.Hook approach, @jchavarri ? It seems like custom hooks would be supported - we'd just use let.Hook inside those too, correct?

One downside imo is that it's a heavier approach than the current API. So to alleviate the heavier API we introduce a dependency on the PPX, which is also not ideal as ppxs bring some inherent complexity. But with native support for the continuation passing style ppx in Reason in the horizon (reasonml/reason#2140) I see the risk of that dependency would be kind of mitigated.

The other downside is that I think the more complex types make the API less accessible. While this kind of abstractions are quite idiomatic in OCaml and other typed functional languages like Haskell, I think it takes some time for someone coming from JavaScript to understand them because there's no referents to rely upon in the language we come from. Or at least that's been my experience. 😅 It took me some time for these monadic solutions to "click" for me, and I still struggle when I have to design or work with monadic-like types. Using ppxs also obscure the dev experience, as errors can be more cryptic etc. However, I think these challenges can be counter-balanced with great documentation and lots of good examples. I believe that the unlocked power that the type system offers makes it worth it!

@jchavarri would you flesh out what let.Hook would desugar to?

@cristianoc Sure! The example above would desugar to:

let render = () =>
  useReducer(reducer, 0, (count, dispatch) =>
    <view>
      <button title="Decrement" onPress={() => dispatch(Decrement)} />
      <text> {"Counter: " ++ str(count)} </text>
      <button title="Increment" onPress={() => dispatch(Increment)} />
    </view>
  );

In terms of types, I'm not sure yet about all the details and would appreciate any insights you might have, but it seems the return value of the components render function would go from simple type element to an abstract type element('t) where 't "accumulates" the different types of all the core hooks that are called inside:

module Hooks = {
  type state('a);
  type element('t);

  let primitive: string => element(unit) = Obj.magic;

  let addState:
    (~state: 'state, element('t)) => element(('t, state('state))) =
    (~state as _, x) => Obj.magic(x);
};

let useState = (state, f) => {
  let setState = x => ignore(x == state);
  Hooks.addState(~state, f(state, setState));
};

let helloComponent = (~message) =>
  useState("Harry", (name, _setName) => {
    Hooks.primitive(name ++ ", you have a message: " ++ message);
  });

In that case, helloComponent has type:

(~message: string) => Hooks.element( (unit, Hooks.state(string) ) )

The custom hooks, because they're made from core hooks, would also compose the core types accordingly. I think / hope 😄

[jchavarri]

@bryphe Another downside: there is an extra function call that has to be made to call the callback passed to the hook to get the results back. I mention this not only for perf (the cost of a function call can probably be omitted) but mainly for debugging purposes. From Dan Abramov's blog post:

Another way to impose more ceremony is by making Hooks monadic or adding a first-class concept like React.createHook(). Aside from the runtime overhead, any solution that adds wrappers loses a huge benefit of using plain functions: they are as easy to debug as it gets.

Plain functions let you step in and out with a debugger without any library code in the middle, and see exactly how values flow inside your component body. Indirections make this difficult. Solutions similar in spirit to either higher-order components (“decorator” Hooks) or render props (e.g. adopt proposal or yielding from generators) suffer from the same problem. Indirections also complicate static typing.

Except for the last sentence –I don't understand exactly what he meant with "indirections" and "complicate"– I can see the points he makes.

There was a very interesting convo on Twitter a few months ago about a monadic solution for React hooks that mentioned some of the downsides and upsides of a monadic solution: https://twitter.com/rauchg/status/1057662611641196544

[jchavarri]

I started something in jchavarri@02b8643.

I'm struggling with some typing issues, because the callback in useState returns a type hook(state('a)). This goes against all the current type assumptions, where all render functions return a component, the children are a list(component) etc.

So, right now, the compiler fails to type check.

I wonder if there's a way to keep the simple type component around, but still have the guarantees that the abstract type hooks('a) provides. I was thinking on adding some conversion in this line:

https://github.com/jchavarri/reason-reactify/blob/02b8643d0a10d1367a57a60410a336e9a7db889c/lib/Reactify.re#L141

But I can't come up with anything that works. Maybe we actually need to propagate the hooks('a) type all around? 🤔

[bryphe]

Thanks for the thorough and detailed proposal @jchavarri ! Those twitter conversations on hooks / type safety were really interesting. I suspect that perhaps the monadic / continuation models in JavaScript have a different trade-off profile vs here in Reason - since there are lots of JS environments that couldn't give you type-safety (and no ubiquitous tooling like ppx), it's not worth it. However, since we do have those tools with reason, and the ppx transform lessens the cost of the continuation approach - having the type-safety seems totally worth it! It sounds like @jordwalke was on board with a plan like this from the twitter thread.

I'm struggling with some typing issues, because the callback in useState returns a type hook(state('a)). This goes against all the current type assumptions, where all render functions return a component, the children are a list(component) etc.

Yes, this seems tricky! It seems like the important place for this is the place we actually call the component's rendering function, the c() here:

      Component(
        id,
        () => {
          Effects.resetEffects(__globalEffects);
          let children: list(component) = [c()];
          let effects = Effects.getEffects(__globalEffects);
          let renderResult: elementWithChildren = (
            children,
            effects,
            __globalContext^,
          );
          renderResult;
        },
      );

I wonder if there is a composite type we could return here, but then discard the hook(state('a))? I don't believe we have a need for it in the internals, as it is primarily for verifying the semantics of the hooks usage in the component's rendering function.

Thinking about a desugared useState example:

let render = () =>
  useState(0, (state, setState) =>
    <view>
      <text> {"State: " ++ str(count)} </text>
    </view>
  );

Perhaps it would make sense for useState to return a tuple of (list(component), hook(state('a)). Then, we'd update our children as something like:

let childrenAndHookState = c();
let (children, _) = childrenAndHookState;

(Something like that). I believe this would keep us from needing to propagate the hook(state('a)) everywhere. We'd basically use the type accumulation in the render method to ensure consistency, but then scrap it internally once we get the list of children.

I'm not an expert on managing the type system though; not 100% sure this works! I'm optimistic there is a way to express this though.

Maybe we actually need to propagate the hooks('a) type all around

Hmm, what would this look like? Would we need to add hooks('a) to the component type?

[jchavarri]

Thanks for the help and guidance @bryphe ! I really appreciate it. 🙂

I'm documenting here the progress so it serves to arrange my thoughts, and maybe also helps in case you, @cristianoc or maybe others have ideas on how to keep moving forward, as I'm not sure I have enough knowledge about OCaml and type systems to implement this 😄 .

I started following the path you proposed: make useState return a tuple. However, that change causes a cascading effect in the types design:

• It forces to consolidate components returning the result of useState with components that keep returning regular elements under the same type
• Because of that, I change the primitives to also return a tuple. In the primitives case: (component, hook(unit).
• That change causes the compiler to fail for the children that appear in the render function
• We can tweak the code to deal with children in the form of a tuple too. Instead of the type childComponents = list(component) we can have a type childComponentsAndHooks('t) = list((component, hook('t)))

But in that last step is where I find the main show-stopper. We don't want to consolidate all children under the same type, as we want to keep allowing components to render one child with useState, another without any hooks, another with useState and useReducer, etc.

So it seems we might want an approach without tuples.

Then I thought about a different strategy which would involve adding an extra variant to the component type:

type component =
    | Primitive(primitives, render)
    | Component(ComponentId.t, render)
    | Provider(render)
    | Empty(render)
    | Hook(hookChain, component)

With hookChain being the type keeping track of the hooks chaining. However, it would have the same problem: hookChain would need to be made abstract (hookChain('t)), so that type variable would "propagate" upstream to all functions using the component type.

So... maybe making hookChain a GADT, so the type variable doesn't escape to the wrapping component type could be a potential path to follow? I'm not 100% sure how to do this 😅

To summarize, I think my issue is I don't know how to create a type for hooks that is composable / abstract so we can keep track of the different hooks that are applied in the component render function, and at the same time without leaking that type variable across other types that would be forced to carry that variable around.

I hope this makes some sense! 😄

[bryphe]

Thanks for the help and guidance @bryphe ! I really appreciate it. 🙂

Not sure I'm actually helping at all, I'm learning right along with you 😄

Great write-up and summary as always!

I was reading through this blog post again - Diff Lists - trying to get some ideas. I'm still trying to learn GADTs so I hope that everytime I read it maybe more will sink in 😄 It describes an interesting way to construct type-safe, heterogenous lists.

So... maybe making hookChain a GADT, so the type variable doesn't escape to the wrapping component type could be a potential path to follow? I'm not 100% sure how to do this 😅

At the top of that blog post - there is an example of creating a heterogenous list which has a hidden type (using an existential). The syntax is like this (in reason):

type ex_list =
  | Nil: ex_list
  | /** 'a is hidden! */ Cons('a, ex_list): ex_list;

I suspect we could use a similiar strategy to drop the type variable for the hookChain, making it abstract. That might be enough for us to get away with making this work, since, we could use the existing scaffolding today w/o reading from the hookChain. That perhaps may be enough to allow us to get to the next step of prototyping this approach. If I understand correctly, it gives us a way to 'hide' that type variable so it doesn't escape or propagate.

If it does - I wonder then if we could extend that 'diff list' idea in the blog post, and instead of a tuple of (hookChain, component) we could create a heterogeneous linked-list of hook 'effects' to model the rendering?

My thinking is we could have a hook type:

type component =
    | Primitive(primitives, render)
    | Component(ComponentId.t, render)
    | Provider(render)
    | Empty(render) 
and hook
/* | UseEffect(...) */
| SetState(state)
| Render(component);

We'd be generalizing the actual component-render as a hook too here. In other words, components would always return a list of hooks - but in the normal-no-hook-case, it would just be a single node of Render(component). As you chain hooks, it'd create longer lists, up to the final Render(component). The diff list concept in the blog could help us by maintaining the typing across that linked list.

I was playing with this idea and it would certainly have downstream impacts (didn't quite get it compiling, yet) - https://github.com/revery-ui/reason-reactify/tree/bryphe/wip/ideas-for-hooks - it would unfortunately involve cross-cutting changes - every component would need to now render a 'hook', even if it is just wrapping their existing behavior.

Then, there'd be a couple cases:

• Render a component without a hook - we'd return Cons(Render(component))
• Render a component with a single hook - we'd return append(Cons(SetState(a)), Cons(Render(component))
• Render a component with multiple hooks - we'd return append(Cons(SetState(b)), append(Cons(SetState(a)), Cons(Render(component)))

We'd always return this typed, heterogenous linked list. The empty case would just be a single-entry, render component case. The hooks cases would just be a chain of hooks, prior to the ultimate render, like:

SetState -> SetState -> SetState -> Render

The chaining would happen naturally as a result of your proposed continuation model - we'd append a node to the list at each level of the continuation, until we returned the full diff-list of hooks + render.

One nice property of this is that (I think) we could get out of the business of having to track global state for effects / hooks. The render function would return this linked list / diff list of 'hooks' from the render function, which conveniently stores the same things that we currently put in our global state management / effect management variables.

We could then unpack the resulting linked-list it to get the child components (it'd be a Render(component) entry at either the head or tail of the list, depending on how we order it), for reconciliation, as well as store the linked list on the instance to preserve the state. Since the order is always the same - rehydrating the state for an instance would be just traversing the list in the right direction.

Sorry this is sort of a brain-dump and nothing concrete... just some ideas I had trying out the diff list idea.

[jchavarri]

Thanks a ton @bryphe !!

Your message is very encouraging because I was doing some work in parallel: https://github.com/revery-ui/reason-reactify/compare/master...jchavarri:starting-hooks?expand=1

And it seems both approaches are quite similar 🎉 as they're both based in GADTs. I'm also learning GADTs, but from what I understand, we can leverage them to keep the abstract types coming from the different hooks (like (reducer(string), state(int)) hidden temporarily as they get "packed" into the GADT, which makes life easier for intermediate parts that don't necessarily need to know about the internals of these abstract types.

In the branch starting-hooks, I'm creating a new type wrappedElement:

and wrappedElement =
    | Hook(component, hook('t)): wrappedElement
    | Regular(component): wrappedElement

which is very similar to the hook type in your branch:

and hook =
    | SetState('a): hook
    | Render(component): hook

(Side note: I think we should rename the type component to element for clarity, as its meaning is closer to the second, but I'm curious what your thoughts are...)

One of the things I'd like to try is to keep the useState and other hooks calls as merely abstract types, nested into each other through tuple types. The benefit of this approach (as opposed to represent them with a list of variant types) is that it has zero runtime overhead, as all these types gets erased once one "unpacks" the GADT and discards the part that represents the hooks nesting. I'm not sure if it's totally possible but from Cristiano's example it seemed a possibility.

So, a component with hooks would always return Hook(component, hooks('t)) with the hooks('t) never becoming a value that is realized at runtime, but just existing at compilation type with types like hook((state(int), hook(reducer(int))) etc.

@bryphe What do you think about the approach above? Maybe it's over complicating things a bit? I'm not sure where these roads will take us but in general using a GADT seems like a promising path 😄 Will keep investigating!

[jchavarri]

Hm... so, after more attempts in the starting-hooks branch, I'm not sure I know how to make this approach work with the first class modules.

The challenge is that we want the calls to useState and other hooks to return a different type depending on the hooks used by each component. In particular, depending on the type of hook + type of the state. This unfortunately doesn't go well with the GADT approach to "hide" the type we were discussing above: if we hide the hooks types, then all hooks would return the same type and thus no errors would be shown by the compiler if some component uses different hooks in the render function depending on some condition.

So, let's assume, supposing the above is true, that the component module contains a type t('t) instead of a simple t to reflect the dynamic type returned by the render function. Then I'm afraid we can't keep using first class modules as they don't allow to abstract over parametric types. We might need to use functors for this (see https://discuss.ocaml.org/t/functors-vs-first-class-modules/1722)

@bryphe @cristianoc Does the above make sense? I might keep exploring the functor path, unless you know if there are ways to implement this behavior using first class modules?