AI generated frameworks..

[MDA2AV]

I am thinking on no longer accepting these, too much time reviewing them and not sure if it brings any relevant data to the platform.. what do you think?

I see new framework entry.. check repo and 200k lines of code in 1-2 weeks, no human review in sight, anyone can just create this with a few prompts so does it make sense to benchmark these..?

Last week I spent over 12 hours total just digging at some of these source codes, everything is the same everywhere, clones that do the same mistakes all over with zero new concepts or any out of the box solution, like reading the same book over and over.

[MarvinDrude]

I agree, digging through hundreds of thousands of lines of prompt-generated code that just replicates the same patterns and flaws adds zero value to the overall benchmarks. Personally i like to think the most value comes by comparing frameworks that have atleast one of the following attributes:

• widely used by many people and/or is a standard framework
• it's humanly designed with genuine thought behind it and maintenance put into it
• it brings in some novel concepts that are new or rare etc.

Me personally i am also pretty exhausted by a lot of clanker code, not only in open source but also at work. And I am not saying, no AI at all, no one bats an eye if you generate some tests or a very scoped method / class with the necessary steps to iterate until its in harmony with the rest of your project.

[lalinsky]

I agree, but if you want a softer rule, add some project maturity metric as a precondition. Most of the vibe coded projects are alive for a few months at best, and then the author moves on. There is probably not much benefit in benchmarking frameworks that won't be maintained in the following year.

[floatdrop]

It is a good rule to make a 90 days cooldown for repository (and optional minimal stars count). Many projects are pushed and abandoned.