the adjuster should have a lock nut on it, to hold the adjustment you've made, but this will not hold the axle tight on the swingarm.
if your axle stack/bearing retainers are not installed properly, your axle castle nut will work loose, and this puts pressure back on your adjuster, kicking it out of adjustment.
check your bearings, retainers, sprocket carrier....etc. make sure everything is right.
you should be able to adequately torque down the axle nut without effecting the rear brake once your adjuster is set.