A generative AI tool such as a large language model produces text by predicting the next most likely word, over and over, from patterns in the data it was trained on. It is a very sophisticated pattern matcher. It does not look anything up, does not reason about truth, and has no understanding of what it writes.

This matters for teaching because it explains the behavior you see. The tool sounds confident whether or not it is correct. It can produce fluent, well-organized prose that is also wrong. And because it predicts from patterns, it reflects whatever was common in its training data, including errors and bias. Knowing this, you can treat its output as a draft to verify, never as an authority.