What a shame about the weather, I am with you on the awful path on the loose stones down from Thornthwaite Crag, torture in descent. I quite like the route up Caudale Moor but perhaps not as much fun in the damp and clag.
There is a Karl route that goes directly down to Brotherswater from Caudale Moor. You are wise to have descended the more usual way