Instead of leaving the baked potato or rice on your plate, why not ask your wait staff to hold the potato or rice? Normally when this is requested they will ask if you want a substitute. I ask for either a salad or steamed veggies.

So far anyone I have dined out with doesn't ask or say a word about me making the request. Some people just don't care for potatoes or rice with their dinner.