Enmos
OK, lets break this down
so you're obliged to act according to your prescribed duty
fruits meaning enjoyment.
IOW it is not recommended that one perform prescribed duty with the view of being the self same beneficiary, at least if one is after a peaceful existence
further details
here on why
BG 3.27 The spirit soul bewildered by the influence of false ego thinks himself the doer of activities that are in actuality carried out by the three modes of material nature.
I guess a simple short hand of this is to state that the "three modes of nature" (sattva, rajas and tamas) is the substance that generates what we might coin "fate".
IOW action through the three modes is simply the action of fate (and due to false ego, or illusion, we think that we are actually at the helm)
kind of like the other end of the scale of simply doing duty for the fruits. IOW being predisposed to issues of praise ("so therefore let me be the enjoyer of prescribed duty") and blame ("so therefore let me not perform my prescribed duty") make for the inferior performance of prescribed duty.
Hence
BG 14.22 The Supreme Personality of Godhead said: O son of Päëòu, he who does not hate illumination, attachment and delusion when they are present or long for them when they disappear; who is unwavering and undisturbed through all these reactions of the material qualities, remaining neutral and transcendental, knowing that the modes alone are active; who is situated in the self and regards alike happiness and distress; who looks upon a lump of earth, a stone and a piece of gold with an equal eye; who is equal toward the desirable and the undesirable; who is steady, situated equally well in praise and blame, honor and dishonor; who treats alike both friend and enemy; and who has renounced all material activities—such a person is said to have transcended the modes of nature.
IOW the whole point of the gita is aimed at action that transcends the modes (or action that can break the bonds of fate).
So inferior action is deemed as that which works within the modes.
So its not so much an issue of passing the buck ("hey its my duty and I screwed up but don't blame me because I am not the doer") but a means of being able to perform duty in an equipoised manner ("If I fail, I continue, if I succeed, I continue).
Basically if one acts alternatively to this suggestion, one will be working with in the consciousness that they are the proprietor, which is the crux of all conflict in this world.
In short, we can neither renounce this world, or claim to be the proprietor, so karma yoga aims to show the means of action in a temporal existence, culminating at the point of performing action in the service of god