I'm not someplace where I can dig the article up right away, but I thought I read recently that agriculture has been mostly ruled out as a probable cause of brain size decrease in homo, due to the shrinkage being a) evident prior to the introduction of agriculture, and b) visible in populations that developed agriculture much later. I believe another possibly confounding factor was mentioned as well, but I don't remember what it was.

It was a pretty good article, covering five or six possible theories about why this has occurred.

ETA: It was in Discover magazine, September 2010 issue: “The Incredible Shrinking Brain,” by Kathleen McAuliffe (gated)