You're right about the heat effecting it all, but I've found one of my exhaust valve at .005 after running it up to red line....had no problem. That's with 836 and mild high lift long duration cam. This and the GL1000 engines are famous for being very tough engines with tollerance to spare. I've been running both for about....the '76 SS since 2001 and '77 GL since 2003. I can attest they're both very tough engines.
I'm not saying you should let them get out spec, especially valves too tight, but a bit too loose hasn't shown me any trouble, beside a bit of clatter. Of course they're both at about 69k miles. They'll last me the rest of my life.
Really every bike will be different. I use to set mine at .002 and .003, but I found that a couple of my intakes would always close up a bit.....after spending a good deal of time checking them every day and logging their clearance changes what I found was that two of my intakes valves, if you set them under .003 they'd tighten down to about .0015 in 4-5 days, and if I set them about .003 they'd end up at .004 anyway, so I started leaving them at .004 and they don't change. They stay there without adjustment for a year or more. It was about the same deal with the exhaust, and every single valve is different in that reguard. I literally logged the changes in valve clearance every day for nearly 4 months.
So what I've done is just find a place where the valves on my ol' bike like to stay....longer than any other setting I could put them at. I've run the heck out the bike that way for several years.
...in San Antonio from 25 degrees to 110 (on the freeway) degrees.