On a recent walk the trail I was following began to diverge sharply from my GPS route. I assumed I had taken a wrong turning and left the trail, cutting across rough ground to reach the path indicated by the GPS. When I got there I found there was no path. Then I realised that the trail I had been following was the right one and the GPS route was simply a straight line between two waypoints that disregarded the curvature of the trail between those waypoints. My GPS (not the unit itself, but the route I had downloaded) was wrong. But you can't follow a GPS blindly, any more than you can follow a map blindly.
On the same walk the day after, I had to make a diversion from the route I was following because of a collapsed bridge. The diversion was a couple of miles long and I found that my GPS with its tiny screen was all but useless for plotting a route over that distance. I had to navigate by map, and I enjoyed it.
I agree that navigating by GPS frees you up to enjoy the scenery more. But it does take something away too. When you navigate by map you have more of a sense of where you are in relation to your wider surroundings. You can look at a distant hill or river and identify it from the map, and you can say "my route took me within 2 miles west of x". You can't do that with a GPS. Perhaps it's not an issue if you're on familiar territory, but it does emerge when you're walking somewhere new.
Then again, I've been on hills in cloud and just carried on walking thanks to my GPS, whereas I would certainly have found navigating by map harder and I might easily have gone astray. Each method has its advantages and disadvantages.