Great piece! I have noticed anecdotally that Perplexity.ai hallucinates *much* less (if at all) than the latest models from Claude or ChatGPT (the one with internet search access). It has become my go-to replacement for Google searches that long ago became low yield. Do you know if Perplexity uses RAG or another approach to tune down the problem of fake citations and made-up results?
I use Perplexity.ai all the time too. From the beginning Perplexity has injected web search context into their model's responses. They were really the only ones doing it until lately. So, it is a form of RAG. They are really good at it and I think are very strict with making sure that the LLM's responses are grounded in the context provided. I don't know anything for sure, but it seems like Claude and ChatGPT with web search still give the LLM itself a lot of room to use their trained-in data more than Perplexity and that leads to much more hallucination
Thanks, that makes a lot of sense! I actually don't think I ever caught a noticeable hallucination with Perplexity (which is why I use and recommend it). For anything remotely high stakes (not just a curiosity search) I make sure to verify claims by following the attached links, which they also did before OpenAI and Google copied the trend. I tell people that for LLMs, I used them for very different tasks:
- Perplexity: Factual searches / research
- ChatGPT: Image generation, brainstorming / workshopping ideas, data analysis
- Claude: The superior writing/editing tool, IMO
I currently only pay for ChatGPT, but as Anthropic adds more capabilities to Claude I will probably switch over time
Great piece! I have noticed anecdotally that Perplexity.ai hallucinates *much* less (if at all) than the latest models from Claude or ChatGPT (the one with internet search access). It has become my go-to replacement for Google searches that long ago became low yield. Do you know if Perplexity uses RAG or another approach to tune down the problem of fake citations and made-up results?
I use Perplexity.ai all the time too. From the beginning Perplexity has injected web search context into their model's responses. They were really the only ones doing it until lately. So, it is a form of RAG. They are really good at it and I think are very strict with making sure that the LLM's responses are grounded in the context provided. I don't know anything for sure, but it seems like Claude and ChatGPT with web search still give the LLM itself a lot of room to use their trained-in data more than Perplexity and that leads to much more hallucination
Thanks, that makes a lot of sense! I actually don't think I ever caught a noticeable hallucination with Perplexity (which is why I use and recommend it). For anything remotely high stakes (not just a curiosity search) I make sure to verify claims by following the attached links, which they also did before OpenAI and Google copied the trend. I tell people that for LLMs, I used them for very different tasks:
- Perplexity: Factual searches / research
- ChatGPT: Image generation, brainstorming / workshopping ideas, data analysis
- Claude: The superior writing/editing tool, IMO
I currently only pay for ChatGPT, but as Anthropic adds more capabilities to Claude I will probably switch over time
Dave, what happens if you make databases for LLMs have real-time data and have automatic external verification systems in place?
That should help to reduce the hallucinations. But the external verification system has to be tuned and validated that it's working correctly.