Double standard or is there something else. Should I take my bosses comments more seriously? Or what? What do you guys think?
I am not sure it is a "double standard" in the classic sense, but rather there are materially different psychological forces involved. The truth is, sexual harassment in this sort of context is an issue of *unwanted* sexual or romantic activities between co-workers.
Imagine I ask a woman I work with out on a date, and she feels very weird about having been asked (which of course means she's completely uninterested). I however press the case, and ask her out a few more time (on dates and to various other non-work related "group" activities, where I nonetheless have the opportunity to try to show her that I'd be a good guy to go out with. She may feel a tension in our future work interactions because she now perceives me as someone interested in her and someone willing to "chase" when she's already shot me down. To HER, it seems she's being harassed by a would be suitor, and it affects her job.
Imagine however, that I did convince her that I was a good enough prospect to date. Either because she accepted when I first asked her, or she warmed to me in subsequent encounters (or maybe she wanted to be "chased" a bit). Since my advances were not unwelcome, she's not going to see them as "harassment." (Hence the number one rule to avoiding sexual harassment claims: "Be handsome.")
The law doesn't forbid dating co-workers, or even underlings, but the line is very fuzzy when it comes to unwanted advances.
In the case of a female manger, it's unlikely that the men feel harassed. First, they almost certainly do not feel "put down" as males by the terms of endearments, because the self image of a man qua "maleness" is generally pretty strong. If someone calls me "baby" because they see males as weak and inferior, I'd be inclined to laugh at them. More likely I would take the term--mostly on a subconscious level--as a sign that I was desirable and that she was acknowledging that, not as a sign that she things men are inferior.
The power dynamic and history between men and women is such that different inferences are appropriate when a manager uses such terms depending on the sex of the manager and the subordinate employee. SO in a sense there is a "double standard" but a very reasonable one.