"...not spend a fortune..." There is really very little on these bikes that you can just get rid of and not miss their absence. Sure, many have and many will learn to live without, but most of the gross weight offenders have to be replaced with something, and something lighter is always going to cost more.
So it's a process. the length of which is determined by how much money you are willing to spend. That lead/acid battery is working good now, so no reason to throw it away. But it will not last forever. Spend a more on a lighter new tech version when it is time to replace, and it will not feel so expensive. Perhaps your exhaust condition is useable but deteriorating and it finally gets further damaged in a garage mishap?...replace it with something lighter.
And you have to learn how to build parts yourself or be able to persuade others, with more tools, equipment, and knowledge to help as often the best performing light weight part is just not available for your bike.