It sounds like Koh Samui is ideal for your sensitivities. Pattaya shocked me on my first time there also. It is in yer face sex industry and the nickname for it is Bangkok on speed! However, once you've found your way around and learn the ropes it's one of the most pleasant places globally.
The restaurant scene is international, the place encompasses Cosmopolitan and just outside has some of the best housing and small places to enjoy all the pleasures in a more small town way.
Everything you could ever want is a motorbike ride away.