A neurobiologist would probably say that "It's not mine" does indeed come down to deterministic physics and chemistry taking place within the person's brain.
He would only say it if he didn't mind making unsupportable claims.
Oh, chemistry is not exactly deterministic. Instead it is a matter of equilibriums.