“Everybody says that you … are crazy.” … are wrong here.” Or worse, “Nobody else does that.”
Think about why you use this phrase or tactic. Are you trying to bully the other person?
If you are upset about something, talk about it with respect – without bringing in other people into the conversation.
(And, btw, how do you know that everybody thinks or says that?)