@samir I just took time to properly read https://functional.computer/blog/llms-cant-program
It’s very good. Thanks for writing up your experiences.
I’m intrigued that you are having/have had success with getting LLMs to fill in the gaps or identify weaknesses.
In my experience LLMs are very good at summarising code – they work well when they have all the information they need to respond.
I’ve found them very hit and miss when asking them to be inventive. Sometimes they are amazing, but it’s unreliable.