But I only wondered if then γ and m aren't actually defined quite right.
m is defined to be the rest mass (i.e. a frame-invariant mass). I'm not aware of any other obvious frame-invariant quantities with dimensions of mass.
γ is derived from the postulates of relativity. If the theory is right, so is the derivation of gamma. Redefine gamma and you have a new theory.