Define LLM policy

[faisal]

Codify our guidelines for acceptable use of LLMs in code contributions.

[faisal]

Here's a strawman proposal for an acceptable use policy when creating new code or changing what's there.

1. It's OK to use an LLM but you must disclose its use when filing your PR. We may wish to add something to the checklist.
2. If using an LLM you should review the result thoroughly as if it were produced by a precocious intern.
3. We discourage the use of LLMs for large projects unless broken into multiple discrete tasks about which one can reason.

If this sort of thing makes sense, then can we add a checklist item to cover 1&2 somehow? Like you either confirm you did not use an LLM to write the code, or you confirm that you reviewed it.

Although that raises another point: sometimes it's useful to use an LLM to throw together a proof of concept for discussion but that you don't mean to integrate. Is that OK for this project?

[pnomolos]

A few thoughts seeing as I submitted the PR that triggered this 🙂

I think that points 1 and 2 are good. I would probably reword 2 to be a little bit more general - "If I did not handwrite this code then I have carefully reviewed it and confirmed it is of high quality and is as bug free as I am capable of discerning"

For 3 - I feel like that just goes into general contribution hygiene. Even non-LLM code shouldn't be submitted in giant PRs 🤷

Maybe for the final bit, you have that as part of the contributing guidelines and call out that an issue (or discussion, if that feature was enabled for this repo) needs to be raised first, rather than just dropping a PoC PR without prior context.