If I rode a bicycle to work, I wouldn't wear the uniform associated with that either. It would seem that millions of people who ride a bicycle through necessity (don't own a car), don't feel the need to be kitted-out in Lycra. And yet bicycle daily unhindered and without complaint. It's usually cyclists who have other modes of transport available to them, but chose to bicycle to work are generally the first to wear Lycra. If I were to bicycle anywhere, I'd likely just wear appropriate clothing, not go overboard and look like a complete nerk.
And it's pretty much the same with walking. The longer the walk, the more I'd probably look like a walker. But on day walks, I will just have appropriate footwear, a jacket (just in case) and that's it. And I've rarely set off on a day walk after a hearty breakfast. It's a day walk. I can survive without eating for a couple of hours.
To answer you question, "why would I not want to look like a walker?" Think for a moment about a schoolboy on his first day of school, new shoes, new trousers, new blazer, packed lunch, pencil case loaded with unnecessary items that won't even be used until at least Year 10. And he gets to school to find everybody else has skirts rolled up, skinny school ties, make-up, and much cooler school shoes. Well, that kid is every Nouveau walker that turns up for a walk dressed head to toe in Gore-tex, and a backpack fit to burst with unnecessary items that are not required unless walking the GR10 or similar. I don't need the walker uniform on a short walk to Catbells