Are you sure the bolt doesn't thread into the arm itself (no slotted nut)? You can often remove/install chainring bolts with just the hex wrench, letting friction hold the nut. If there is a nut and it's spinning, you might be able to hold it with a screwdriver.