I'll second Werrington Walkers' suggestion. The Church in the Sea is lovely. There was a wedding on when I was there so did not get a chance to look inside, but what a fantastic location to get married at and it was good to see it being used.
My favourite parts of the coast path where south and west from Holyhead over Holyhead Mountain and around South Stack and also and the north west tip of the island, but this is the remotest part.