What is adequate very much depends upon other factors, particularly how people are affected by the cold - some are very tolerant and others more susceptible.
Also, the mat is just part of the system needed to keep you warm. The sleeping bag used, any extra clothing worn, how draughty the tent is, and (often overlooked), the surface that you have pitched upon - short heather, springy grass, crowberry will trap air and have some insulation value, as well as being comfortable!
Personally, I have always found a simple closed cell foam mat to be perfectly adequate for wild camping/bivvying. As I get older, the thought of more comfort is tempting though....