Well that minus 20 degree tolerance depends on the individual, age, health and they way they are dressed. If you fall into 29 degree seawater without a survival suit, you have only minutes to live before succumbing to hypothermia.
You can die of hypothermia in 50F water, or even air - that doesn't make 50F some kind of scary emergency temperature for outside air to be.
Neither is -2F - that's not cold enough to freeze a guy from the inside out just by breathing it, or cause his internal organs to go haywire. -20 can be a problem if your car breaks down or your house catches fire and you aren't ready, but it's a temperature I used to go outside and play in as a child. The biggest problem was that the ice and snow got more abrasive and your skates, sleds, etc, didn't work as well. Also, you can't make a good snowball. But your boots and mitts tended to stay dryer - which meant you actually had an easier time, in some ways, staying warm, than on technically warmer days. Close the schools in Minnesota?
To be fair, the northern temps did get down there - near -40F&C, which is dangerous and has to be respected - but still nobody died of thymus swelling. I'll bet people went ice fishing, to the rink, took the kids - got to do something with them if school's closed, they go stir crazy penned up in the house too long, and this was after a weekend.
I've met a guy who got hit by lightning. I've met a guy who was kicked in the balls by a loon. I went hunting with a guy who stepped on a skunk in broad daylight. I've known people to get killed (and eaten) by their pigs, assaulted and gored by white-tail deer, hit by falling trees, knocked out of their boats by jumping fish (that was a new one, just last year), die by getting stuck in a badger hole (that was ugly), drown in the corn in their own silo, get shot by their dog, get blown up by their car battery, cut their own heads off changing a tire on their truck. I have never, in all my years in some of the coldest but nevertheless well populated areas of the planet, heard of anyone killed or lived having their thymus swell from breathing cold air - even really cold air, never mind some -2F normal winter morning. Know what's more llikely? They woke up a beehive in a wall and got stung somehow, went into anaphylactic shock. Sure it's unlikely - but not as unlikely as thymuses gone wild for no apparent reason.
Come to think of it, I don't believe a thymus could crush a trachea if it did swell - that's a soft organ, there's other places for it to push into, and the force necessary to compress a trachea is significant