Hcfyk

I'm currently reading "the body keeps the score", and this passage appears in the first page of chapter 6.

(Warning: passage is about emotional trauma, do not read if you're not ready for that)

"I know I wasn't wanted", she told me, "I'm not sure when I first realized that, but I've thought about things that my mother said to me, and the signs were always there. She'd tell me, 'You know, I don't think you belong in this family. I think they gave us the wrong baby'. And she'd say it with a smile on her face. "But of course, people often pretend to joke when they say something serious".

This is a beautiful observation, one of the kind that's worthy of having its own word.

My question for you, is there a word, or phrase that more eloquently captures the idea of "pretending to joke when you say something serious"?

There is the saying many a true word is spoken in jest. As Cambridge Dictionaries explains, this is

said about humorous remarks that contain serious or true statements.

I've seen this used in two different kinds of situations: The first is similar to that in your quote, where someone is trying to cover up the truth of their (usually insulting) statement by pretending it's a joke. The second is when a truly humorous story or statement also contains some important truth, such as a comedic movie that helps illuminate some part of the human condition that might be too painful to approach in a more serious manner.

The origins of this exact phrasing are murky, but similar sentiments have been expressed by many authors, including Chaucer and Shakespeare (see linked article by The Phrase Finder). It's also sometimes shortened to there's truth in jest, perhaps modeled after the similar English translation of the Latin aphorism in vino veritas (in wine, there is truth/there's truth in wine).

It's not one word but back in the day, we used to say a person was "crackin' but factin'" meaning that they were cracking a joke but they were really stating a fact.

But of course there's a grain of truth in every joke

The Deccan Chronicle has an article that says:

There is a grain of truth in every joke, a joke is either truth wrapped in a “don’t take me seriously” smile or it subtly exposes one’s deeply hidden strong beliefs. Many jokes are hostile and sarcastic jokes in particular are laced with an uncomfortable truth and indicate an underlying anger or hostility that seek escape.

I hate using Wikipedia as an "authoritative" source, but I'll say that these types of sayings are examples of caustic humour:

a type of humour which relies on witty insults. As is implied by the name (which literally means humour which is designed to burn or to corrode), it involves the clever use of language to convey biting, insulting, or sometimes even cruel remarks.

This kind of humour is often attributed to such comedians and comedic writers as Lenny Bruce, Richard Pryor, Don Rickles, George Carlin, Bill Hicks, Bea Arthur, Andrew Dice Clay, Ben "Yahtzee" Croshaw, Simon Amstell, philosophers Voltaire and Ludwig Wittgenstein and politicians Sir Michael Cullen, Paul Keating, Derryn Hinch, and Dennis Skinner.

The liberals can understand everything but people who don't understand them.

The only truly anonymous donor is the guy who knocks up your daughter.

Your example seems to be more subtly phrased than some of the more obvious sarcasm.

There's also the phrase covert put downs, which I found used in this rather aggressively written article:

Covert put downs and caustic humor are always telltale warning signs of an Abuser. Whether it’s your snarky mom or Narcopath dad who makes the cut, a jealous or competitive sibling, or a toxic peer bullying friends and playmates on the schoolyard, it’s never fun. Except for the Sadist, that is — never the victim(s).

I think the phrase you're looking for is called "hostile humor". It's when you say something that you're really upset about, but shroud it in a joke, a laugh, or a smile. These lines are often delivered in group settings so the person delivering them has the security of the group. They can say something that will come off as humorous to the group, but that they know their target will get the meaning of, and the targeted person will come off looking hostile if they respond to the underlying hostility that belies the joke, smile, or laugh.