intermittent `TypeError: NodeClass is not a constructor` when getting tree nodes

[verhovsky]

To reproduce this:

cd /tmp
mkdir tree-sitter-nodeclass-repro
cd tree-sitter-nodeclass-repro/
npm init -y
npm install tree-sitter tree-sitter-bash tree-sitter-python

and then create this main.js

const Parser = require('tree-sitter');
const Bash = require('tree-sitter-bash');
const Python = require('tree-sitter-python');

const parser = new Parser();

// parser.setLanguage(Bash);
parser.setLanguage(Python);
const sourceCode = `0`;

const tree = parser.parse(sourceCode);
let i = 0;
try {
  for (;; i++) {
    for (const n of tree.rootNode.namedChildren) {}
  }
} catch (e) {
  console.log(i + " iterations");
  console.log(e);
  process.exit();
}

Then run it

while true; do
  timeout 1 node main.js
done

If the error doesn't happen after about a second, it doesn't seem to happen ever (I left node main.js running for 10 minutes). It can take a few tries, but eventually you will see this error

769 iterations
TypeError: NodeClass is not a constructor
    at unmarshalNode (/private/tmp/tree-sitter-nodeclass-repro/node_modules/tree-sitter/index.js:783:18)
    at Tree.get (/private/tmp/tree-sitter-nodeclass-repro/node_modules/tree-sitter/index.js:25:14)
    at Object.<anonymous> (/private/tmp/tree-sitter-nodeclass-repro/main.js:15:26)
    at Module._compile (node:internal/modules/cjs/loader:1368:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1426:10)
    at Module.load (node:internal/modules/cjs/loader:1205:32)
    at Module._load (node:internal/modules/cjs/loader:1021:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12)
    at node:internal/main/run_main_module:28:49

It happens when accessing .children as well.

[verhovsky]

I've also just seen this error

Error: Invalid argument
    at get namedChildren [as namedChildren] (/Users/space/code/curlconverter/node_modules/tree-sitter/index.js:166:39)
    at extractCommandNodes (/Users/space/code/curlconverter/src/shell/tokenizer.ts:580:32)
    at tokenize (/Users/space/code/curlconverter/src/shell/tokenizer.ts:722:24)
    at findCommands (/Users/space/code/curlconverter/src/parse.ts:31:12)
    at parse (/Users/space/code/curlconverter/src/parse.ts:58:24)
    at toClojureWarn (/Users/space/code/curlconverter/src/generators/clojure.ts:515:20)
    at Object.toClojure (/Users/space/code/curlconverter/src/generators/clojure.ts:520:10)
    at <anonymous> (/Users/space/code/curlconverter/test/test.ts:95:28)

I re-ran the tests without any changes and got the TypeError: NodeClass is not a constructor error. Re-ran without changes a third time and they passed.

[verhovsky]

It's fairly common actually, it happened in the CI as well

https://github.com/curlconverter/curlconverter/actions/runs/8385221889/job/22963783256?pr=645#step:5:31

[verhovsky]

@segevfiner could this be related to the NAPI changes or something in the upstream tree-sitter C code?

[segevfiner]

It could be... But I'm not really sure what could possibly cause this. Guess some good old fashioned debugging is in order... 😛

[verhovsky]

Turns out #190 (comment) is not related to Jest.

[segevfiner]

@verhovsky The bug seems to be that we are holding a weak reference in cached_nodes_, yet the finalizer hasn't been called yet, as per Node.js docs:

Finalizer callbacks may be deferred, leaving a window where the object has been garbage collected (and the weak reference is invalid) but the finalizer hasn't been called yet. When using napi_get_reference_value() on weak references returned by napi_wrap(), you should still handle an empty result.

[verhovsky]

Fixed by #197