This can only happen if every observer in the hole entire universe could agree on a single reference frame from which they can all calculate their relative velocities (such as our Sun). However, this is not possible. Some observers will be to far away to be able to observe that single reference point because of our individual Hubble Volumes. The only "universal reference point" that I know of is the moment of the Big Bang (BB). If you were to observe the temperature of the microwave background radiation as X and yet I observed it as Y then we could deduce our relative velocities from each other according to the BB. Everything is moving away from the BB at a constant rate eventhough this rate can be different between each observer immersed inside our space-time continuum. Logically speaking, the closer any object is to the BB the younger the universe will appear to them. From what I've worked out, the faster any object is traveling relative to another object the longer it takes to move away from the BB according to the other observer. The same thing applies to strong gravitational fields. The stronger the gravitational field you find yourself in the longer it takes to move away from the BB according to any observer immersed inside a weaker gravitational field.