I have two sun hats, one is a Rohan bush hat type that they don't appear to make any more, this is my general purpose sun hat for gardening, walking to the shops, days out, holidays and summer walks when I'm carrying very little. It has an adjustable chin strap that stops it blowing off into the sea when on ferries, but I tuck this into the back of the hat most of the time.
The other one is a discontinued Craghoppers hat, similar to the above but much smaller. It is reversible, I can have olive or fawn on the outside, and it is extremely light (57g). I carry this solely for backpacking or on most walks.
Both of the hats can be stuffed in a pocket and come out none the worse for it, but the most important thing is that they are both made of synthetic materials, one polyamide, the other polycotton. This means they dry quickly if they get wet, and they have not shrunk to the size of a pimple on the head like cotton ones do.