Sean- I really appreciate your willingness to revisit a reasoning—AI and otherwise—that is not your own. I think this is a testament to greatness. Something everyone included, needs to be reminded of. A great food for thought.
Agree with the need for a good dose of critical thinking as we are exploring interactions with AI in its primitive form, the ubiquitous chatbot interface.
I just discussed the need for human evaluation with one of my colleagues. A related and interesting tidbit, as we were trying to explain to ourselves how many people are non-critically using ChatGPT or Perplexity, even though it is easy to show the errors and hallucinations that show up: the Gell-Mann Amnesia effect.
As a result, it is important to realize when we are interacting with AI in the form of LLMs mostly, in fields that we are knowledgeable about, and we are not.
Hi, Nico! Thanks for bringing up the Gell-Mann Amnesia effect - it perfectly captures something I've been struggling to articulate! It's fascinating how we can spot AI's mistakes in our own field, then turn around and trust it completely in other areas. I catch myself doing this sometimes - rolling my eyes at an AI's attempt to explain historical events (especially when it cites non-existent sources!), then nodding along with its takes on physics.
Your point about these being "primitive" chatbot interfaces is spot-on too. I've found some real improvements in platforms that provide clear sourcing and walk through their reasoning step by step - it helps me apply that historian's skepticism more consistently. But we're still in the early days of figuring out how to interact with these systems thoughtfully. Really appreciate you adding this perspective to the conversation - it's exactly the kind of critical thinking I hope we can foster here.
How do you stay aware of this tendency in your own AI interactions?
Sean- I really appreciate your willingness to revisit a reasoning—AI and otherwise—that is not your own. I think this is a testament to greatness. Something everyone included, needs to be reminded of. A great food for thought.
Agree with the need for a good dose of critical thinking as we are exploring interactions with AI in its primitive form, the ubiquitous chatbot interface.
I just discussed the need for human evaluation with one of my colleagues. A related and interesting tidbit, as we were trying to explain to ourselves how many people are non-critically using ChatGPT or Perplexity, even though it is easy to show the errors and hallucinations that show up: the Gell-Mann Amnesia effect.
As a result, it is important to realize when we are interacting with AI in the form of LLMs mostly, in fields that we are knowledgeable about, and we are not.
Hi, Nico! Thanks for bringing up the Gell-Mann Amnesia effect - it perfectly captures something I've been struggling to articulate! It's fascinating how we can spot AI's mistakes in our own field, then turn around and trust it completely in other areas. I catch myself doing this sometimes - rolling my eyes at an AI's attempt to explain historical events (especially when it cites non-existent sources!), then nodding along with its takes on physics.
Your point about these being "primitive" chatbot interfaces is spot-on too. I've found some real improvements in platforms that provide clear sourcing and walk through their reasoning step by step - it helps me apply that historian's skepticism more consistently. But we're still in the early days of figuring out how to interact with these systems thoughtfully. Really appreciate you adding this perspective to the conversation - it's exactly the kind of critical thinking I hope we can foster here.
How do you stay aware of this tendency in your own AI interactions?