Just out of curiosity, have you ever been down there?
I go to Phuket every December for the holidays but rarely go any further South. About 20 years ago I visited Hat Yai and then on to Malaysia traveling for about 1 month. Beautiful Country!
I was in Phuket briefly 4 or 5 years ago during the bike week. Uhhh, a little different than bike weeks here. A million scooters.
I have visited Hat Yai. A few days before flying to Bangkok during a month round trip in Thailand.
A cheap hotel where I did not want to take my sandals off inside room except for when entering the bed

Carpet that might have had all kinds of dirts and maybe sharp objects hidden in it.
Even worse than another cheap hotel very close to Patpong area a few years earlier

We had been in Phuket 2 weeks before. Took the local bus to Krabi. It was cheaper but took most of the day. We saw a lot though during that trip. From Krabi anorher bus, quicker this time to Hat Yai.
People in Hat Yai were not as friendly as further north. A little bit more reserved as I saw when visiting Malaysia a few years earlier, Penang.
I understand those that never stop to visit Thailand a month or 2 every year. Or even stay there.