I often walk around the house in my undies or even totally nude.
At my age, I don't really give a stuff what people think of my body.
In public, I suggest that being clothed is firstly a result of the weather, and also as a bloke, walking in public in the raw could cause some problems, particularly if a Miss Universe type walked by equally naked, and I was to experience some form of "inappropriate excitement" if you know what I mean!
Plus of course, perhaps we [we being both sexes] have over the eons, that baring it all, and letting it all hang out, and the "inappropriate excitement" I spoke about, may cause problems, particularly for married couples security.
Being nude at work will also depend on one's working environment and also the health and safety aspect of any particular situation.
I was a maintenance Fitter, working in confined spaces, between rollers, in freezers, next to furnaces, in close proximity to work mates swinging 14lb hammers, others with pliers, screwdrivers and wrenches and such....Ouch!
Am I making myself clear!
We wear clothes out of necessity in the main, and not because of any shame or embarrassment imo.