I rode mini bikes and trail 50's when I was a kid, at 13 a guy let me ride his Triumph 500 Tiger...I went down the driveway, crossed the street, up the neighbors driveway and right through there closed front door. I got a Honda 175 Scrambler after that and learned to ride and have been riding ever since.
NOW I ride like I'm a ghost and know one can see me, cagers are stupid and they will prove it all the time. Take a MSF coarse if you can it will help, as for the fear...it will slowly be replaced by a big smile as time goes by.