I would say the occasional higher fat day will do you no damage. I would suggest monitoring your calories on the higher fat days. If you do experieince a stall or gain, you'll be in a better position knowing whether the fat, or more likely overdoing calories were the culprit.