You've obviously been talking to the wrong "experts".
There are plenty of doctors who prescribe a low carb diet for diabetics, PCOS, and a variety of other ailments (Dr. Atkins, to name one prominent one - but you can't ask him any longer).

Listen to your body and your health, and eat how you know works for you. Isn't your health and welfare the most important? They are not in your skin.