First, I'll volunteer some info you didn't ask. I have a 2006 with factory navigation, and the 2004 that I added the AVIC to. The AVIC has many features that the factory nav doesn't have, but overall I'm not too happy with it. Compared to the factory nav, it is slow, jerky, and unreliable. If they don't have an update for the bluetooth sound quality issues that it has, I will probably remove it. I wonder if an AVIC Z3 would solve those problems or not?
On to what you did ask about...
The LS /w nav has eight speakers and two subwoofers, all are amplified. You don't want to use any of the speaker outputs from the AVIC. Use the front, back, and subwoofer line outputs only. I don't know where you got your adapter harness, or even if you are using one. All that I have seen so far do not take into account the fact that everything is amplified with the nav setup. They provide the right connectors, but they are wired for speakers and not for the RCA's that you will need.
Two positive things that you get for having the factory nav is that the VSS wire is already coming to the big connector at the back of the headunit, and the accessory power feed is the delayed turn off one.
More you didn't ask for...
I found the best place to run the GPS antenna wire was in the factory wiring raceway under the carpet on the right side of the car. If you are going to be using a reverse camera, you can pick up the reverse signal lead at a connector behind the right rear trim over the package tray. It's the reverse wire that runs to the auto-dimming mirror. The trim over the package tray is actually not all that hard to remove to put the antenna in.