It’s easy to believe: the model keeps up the conversation, explains terms, remembers the context of the message and writes code with comments. In conversation this looks like understanding.
But coherent speech isn’t the same as a stable picture of the task. AI can correctly describe a payment service and then suggest removing the idempotency check “to simplify things”. It can name the cause of an error and immediately build a solution that contradicts it.
The model is good at continuing familiar patterns. It isn’t responsible for the consequences and doesn’t notice a contradiction on its own — especially if it’s hidden across several files, business rules and infrastructure constraints.
So I treat an AI answer as the version of a colleague who quickly read the task but hasn’t worked on the project.
It’s more useful to ask not “did you understand?” but:
— which constraints did you use;
— what might be wrong in this solution;
— which scenarios will break it;
— what’s missing from the context.
AI doesn’t have to understand the task to be useful. But a human still needs to understand what exactly they’re accepting into the codebase.