OK, just got back from having a weather window slammed in my face and have translated your file to a trace on the OS map. The fence line at Hugh Seat is an easy to locate landmark. The going by the fence line as you approach it is boggy but normally passable. If you are coming from the east, you may need to scale the fence, I do not recall a stile. Lady's Pillar is downslope from the angle in the fence but is easy to see once you set off from the fence so it makes a good final fix before you start your off piste run.
I have been as far as Lady's Pillar a few times, I wasn't looking for down slope path but I do not remember one so if you have a gps, this wold be a good time to use it as I expect you will be forced off line from time to time on the way down and you do not want to end up on the wrong side of a gully. On the plus side, it is a downslope route so you should be able to see the geography ahead of you.
Edit to add: Looking at the map, if it was me I would follow the line of Red Gill a bit further to pick up the cairn at 795987. This would line you up with the reentrant contours to take you WNW to clear the heads of the watercourses above the track.