Completely agree. The use of LLMs as a source is laughable. You don’t have to work with them very much to realize that they only partially reliable even for simple tasks. Just had both ChatGPT and Claude fail to identify a brand change on a medical device due to a merger which took place on 2021.
To use a tool effectively or to operate productively, you need to understand the system. Systems are complex so we automatically model them in our head. My biggest takeaway from this article are the perspectives, the pieces of the model puzzle that I'm still fitting into place. I liked your book. It laid a good foundation for my understanding. But there are many facets to using AI effectively and all of your writing, including your use of agents, has helped me understand the overall AI model and use it myself for software development that I couldn't afford without AI. Thank you, I appreciate it.
The comparison to Wikipedia I think is interesting.
The usual objection is that Wikipedia is not a “primary” source and is instead a summary of primary sources by design. This applies to LLMs as well as they are inherently a kind of “summary” of their training data.
One thing a citation is SUPPOSED to provide is a reference which can be checked and followed up by the reader. Wikipedia cite does provide this if you include both the page and date of citation (even if it gets changed later history is preserved/recorded). The LLMs fail this because no one includes the full prompt in the citation or even if they did, LLMs are non deterministic by nature and may not say the same information consistently.
Completely agree. The use of LLMs as a source is laughable. You don’t have to work with them very much to realize that they only partially reliable even for simple tasks. Just had both ChatGPT and Claude fail to identify a brand change on a medical device due to a merger which took place on 2021.
To use a tool effectively or to operate productively, you need to understand the system. Systems are complex so we automatically model them in our head. My biggest takeaway from this article are the perspectives, the pieces of the model puzzle that I'm still fitting into place. I liked your book. It laid a good foundation for my understanding. But there are many facets to using AI effectively and all of your writing, including your use of agents, has helped me understand the overall AI model and use it myself for software development that I couldn't afford without AI. Thank you, I appreciate it.
The comparison to Wikipedia I think is interesting.
The usual objection is that Wikipedia is not a “primary” source and is instead a summary of primary sources by design. This applies to LLMs as well as they are inherently a kind of “summary” of their training data.
One thing a citation is SUPPOSED to provide is a reference which can be checked and followed up by the reader. Wikipedia cite does provide this if you include both the page and date of citation (even if it gets changed later history is preserved/recorded). The LLMs fail this because no one includes the full prompt in the citation or even if they did, LLMs are non deterministic by nature and may not say the same information consistently.