Funny thing is that I have always read from sons regretting the time lost without their fathers because of arguments, and never from dads for time lost with their sons for the same arguments.
Shouldn't fathers be the first ones to regret it? Being older, they should start to realize the clock is ticking before their sons do, and therefore build bridges before their sons do.
Maybe he doesn't like you? And, if you can't compromise with a person as stubborn as you, then what's the point? In fact, he may already regret it. But, believes you don't want to have anything to do with him. Would you force yourself on him?
If you don't even talk, how would you know? If each is waiting for the other to communicate first, when will that happen? When some external event forces you together?
Certainly, there has to be desire from both parties to share experiences. If that's not there, then you are both better off with separate lives.
Just a clinical observation....