Yep, that's what's happened, and even if you turn your fuel off, if the washers under the petcock mounting screws are boogered, fuel will still pass thru to your carbs. If you have a sticky float valve the fuel will run into one or more cylinders, seep past your piston rings, and fill your sump, thinning your oil, and potentially ruining your engine, not to mention the potential for fire. Fix your fuel leak, then dump your oil and filter, and replace them with fresh oil and a new filter.