Assuming your headunit is working (ignoring the camera issue) the adapter must be working. Just leave that alone. It is important to eventually get the reverse sensor working - mainly to switch to rear view when you go into reverse. But that is a separate issue. It sounds like you have a handle on that.
For camera testing purposes, ground the camera's black to some metal on the vehicle chassis. Connect the RCA to the brown connection on the unit. Figure any easy way to run +12v wire to the camera's red. Even if it just goes from the battery. In fact, for testing, just put the camera under the hood (or wherever a RAV4's battery is located) and directly connect the camera's red and black to the battery. Then go to the head-unit and see if anything happens when your ignition switch is on ACC and go to camera view mode.
Does the long RCA cable that came with your camera have a loose red wire at both ends? If so, that is just a convenience to make it easier to power the camera - not having to pull another wire from the head-unit to the camera's location. Use of it is optional. Some people just tap into the backup light power, but then they loose the ability to use the camera when driving.
You could just splice the camera's red to the headunit's red. What I did was add a tap-a-fuse pigtail in my fuse box and power the camera from any switched fuse location. That doesn't mess with the existing harness (no splicing) and is easily removable. But either way works.