Alice has two sisters, how many sisters does her brother have?

This is a question a colleague shared with me recently as his favourite question to AI.

This questions directly relates to this interesting post from futurism.com pointing out the flaws of LLMs identified by this paper from AI research nonprofit LAION.

For the past two years, I've been using AI tools, particularly LLMs, and I've noticed many flaws. However, it's generally difficult to articulate or categorize them. Test with this straightforward question, it is possible to quickly grasp how AI "works" and especially how, as the paper highlights very well:

"models also express strong overconfidence in their wrong solutions, while providing often non-sensical 'reasoning'-like explanations akin to confabulations to justify and backup the validity of their clearly failed responses, making them sound plausible,".

So here are a few answers and the models behind them 🥁:

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

I hope this was insightful! 💡