You Are the Judge

More than a month ago I was looking for a particular feature on a new car.  I was surprised by an answer provided by AI.  It said that it was on a make and model I thought I knew well.

I checked the maker’s website.  The feature was not available as an option.

Interestingly, I looked again recently.  It now states that it was an option on the highest trim of a previous generation of that model.  I tried rephrasing my query, and that affected whether this fact was clearly stated in the results or buried in the details.  It makes me wonder if I missed it the first time.  Either way, it serves as another example of why we should not take AI results, often prominently displayed and seeming so straightforward in answering our question, as gospel.

The experience reminded me of when I taught middle school in the early 1990s and I was asked to coach the academic bowl team.  Given that I had been on such a team in high school, I understood why I was tagged for the role, but my opinion of my own knowledge had been tempered by experience.  Being an avid reader as a child, I had sponged up all sorts of random facts, but I had concluded that they were of little use.  As a science teacher, I taught students to focus on asking big questions and pushed them to reconsider their preconceptions of the world around them.  I openly derided my own abilities in quick recall as “Jeopardy knowledge” that wasn’t worth much, unless it allowed one to win big money on the game show of the same name.

Little did I know then that my outlook would evolve again.  As my experience in curriculum development deepened, I realized that my knowledge had a much greater value.  The facts that I had learned and the understandings that I had gained allowed me to judge the quality of ideas that others proffered.  I wasn’t wrong about the importance of questions, but I had completely undervalued the need for knowledge in making sound professional judgments.  The idea of “standing on the shoulders of giants” who had come before us, as Isaac Newton said, suddenly had much more meaning.  My ability to critique poorly constructed arguments was not a product of logic or reasoning alone, but the confidence I had in facts and ideas that had become part of how I viewed the world because I, as well as the giants before me, had found them to have predictive value.  They provided a foundation for future understandings.  They helped me make sense of what was happening around me.

My mind has not changed about this, but I remind myself that my thinking may evolve again.  In the philosophy of science, the fundamental understandings that underpin a field of study are referred to as paradigms.  These are the theories that serve as a worldview, but even they undergo revision as knowledge accumulates.  Change usually comes slowly before there is what American philosopher Thomas Kuhn terms a “scientific revolution” that produces a new consensus.  The key is that it arises from evidence and the repeated testing of new explanations by a community of practitioners.

Answers provided by AI come from large language models based on the corpus of human knowledge, and thus they are often solid.  But do not blindly follow.  Be skeptical and check—and double-check—even what may sound true.

Our future depends on everyone’s careful consideration.  Remember, you are the judge.