As sickening as I find their behavior, I have to admit that if I were to trade places with one of these men, atom for atom, I would be him: There is no extra part of me that could decide to see the world differently or to resist the impulse to victimize other people. Even if you believe that every human being harbors an immortal soul, the problem of responsibility remains: I cannot take credit for the fact that I do not have the soul of a psychopath.I agree with Harris halfway: given the truth of causal determinism, were he to trade places "atom for atom," he too would be a depraved criminal. But then, like the criminals, he too would be properly blameable and punishable for his crimes.
What could Harris say to this? If Harris means to dismiss or rule out this response (and he does), then an independent argument is needed in support of the substantive philosophical position that causal determinism categorically rules out moral responsibility. So far as I can tell from the above excerpt, Harris has only assumed this critical principle, not argued for it.
To be fair, I haven't read his book. Maybe he has the right sort of philosophical argument for the 'causal determinism rules out free will' background principle there. But I doubt it. From what I have read, he gives the impression that he thinks the issue is simply one of showing causal determinism to be true. But it is not enough simply to assume that if causal determinism is true, then moral responsibility is impossible. That premise of the argument is substantive and controversial and not at all obvious. Moreover, this philosophically substantive premise could not be established on empirical grounds; moral responsibility as such, if it exists at all, wouldn't be the sort of thing that exists as an object of empirical investigation or scientific study. If moral responsibility exists in some possible world, then, in that world, try as you might, you're not going to get moral responsibility into a test tube, or into an fMRI.
So then, a challenge for Sam Harris, and for anyone else tempted to his position: why in the first place should we think that if casual determinism is true, then moral responsibility is impossible?