A small matter that is a big issue for listeners

Just kidding. No big deal?
You really think the listener would say it’s not a big deal? Some words are meant to be jokes for the speakers, but the listeners don’t think so.
These words cause unimaginable pain to the listeners.