Carb size is determined by both the displacement and the maximum rpm's that the engine will be turning.
The correct jet size is determined by a comination of things, as stated previously. I was taught to think of an engine as an over glorified air pump. The jetting is determined by how much air is being pumped through the engine. Each engine and each situation will be different.
Before changing jets, start from the beginning and make sure that the engine is tuned properly. (valve lash, point gap, timing, cam chain adjustment, spark advance, new spark plugs, fresh gas, clean carbs) Then synchronize the carbs before doing the plug chops. I wouldn't want you to waste a bunch of of your time and money by trying to rejet an improperly tuned engine.