Check the fuse (No.19) first to see if it's bad.

Here's what my Helm's says for my B-series transmission, hope you can glean something from it.

If your fuse and bulbs are alright, disconnect the connector from the back-up light switch on the tranmission. With the shift lever in reverse, check for continuity between the A and B terminals inside that connector. If there is none, replace the switch. If there is continuity and you're still experiencing the reverse light problem, check for a poor ground or an open in the wire.