Most people are not terribly observant. And those first pounds, I think, tend to come off from all over, rather than anywhere specific. So the average person around you may not really see a significant impact at that point. It happened to me; 30 pounds and those who see me frequently couldn't tell. However, those people who see me only occasionally immediately remarked the loss.

So don't worry. The cerebrum is the most underused organ in our bodies. Most people spend 90% of their lives in a subconscious state, and wouldn't see a train until it was on them.
Keep up the good work!

Good Luck!