I too use Sh'ra's strategy: "I no longer eat sugar or flour" No one ever complains about this, though once someone said, "hmm, that must be hard considering the way everything is served." Rare is the time that someone will bring a pan of brown rice or quinoa into work and you'd have to make up a lie to cover for not eating that. Instead of getting horror, I get "wow, great idea, I should do that!"

But I also like the reponse others have used, "I can't eat that. I break out in fat."