You definitely need to find out what's causing that voltage drop. You've got no chance of the logic working at 3V and I'd be a bit worried that you're hurting something with a potential short on the board.
Personally, I'd disconnect the board and attach a 12V light bulb to the +5VDC and GND lines on the PSU so it has some load on it, then measure what voltage is being produced. It could be that the PSU is crapping out so when a load is applied it drops down to 3V, or it could be that the board has a partial short somewhere which could explain the 3V reading.
I'd also check the battery in your multimeter - a number of test meters I've encountered in the past decide to give screwy readings when the internal battery is going flat; no obvious symptoms bar getting duff readings.
Let me know what you measure when the board is disconnected and you've got a 12V bulb across +5 and GND. If you're not sure what bulb to use, a fruit machine or most car bulbs will suffice.
cheers
t