I've had trouble with dairy my entire life. As a baby my mother gave me goat milk as I did better with it. However, as I grew up I really avoided dairy due to the digestive upset it caused, and it is still a problem if I have much of it (I'm now 74).
About 15 years ago I tested positive for casein sensitivity and was told to avoid it, so I don't buy whey powder as it has casein in it. I eventually found Jay Robb's egg white protein powder and have stuck with that for the last 10 years or so. I use it mainly after a weight workout, but not really at other times so I do well enough with it. The taste is good.