Back years ago, I think I got down to 125 on my 5'8" frame and the smallest pants I ever fit into were 10's. Of course, sizes have inflated since then! Hmm... another thing is I weigh more than you and I'm just beginning to fit into 14's. Maybe its a height thing.

I have always been thicker in the hips/thighs than the waist though.