I have gotten rid of just about everything as it became too big. My closet is bare now! I did keep a few t-shirts for sleeping, same with an older pair of pj pants.

When I had a lot of clothes, I organized by season and then by what I wore the most - keeping them easily accessible.