In a nutshell can someone explain to me why caffeine affects some people but not others?

I drink regular coffee - appx. 16 oz a day. I experimented in the past where I drank decaff (blech) only for a few weeks to see if weight loss would get faster/better. I saw no changes. During this time I did not "cheat" and pretty much ate the same things.

yet others, swear up and down that once they stopped caffeine they started to lose better.

Can some of us just be immune to caffeine or is it the amount that can be the problem?? Another thing is that caffeine is "hidden" in many things...many herbal diet remedies, vitamin formulas, pain relievers, etc... so for those that are caffeine sensitive....they should not take any of the other caffeine-containing products as well.