My wife and I both drive the same car, but it seems like we have to manually connect our android phones every time we switch cars. Is this normal?

If I drive, I manually connect and it forgets my wifes. If she manually connects, it forgets my phone.

Is there a way to have two phones automatically connect?

It will only auto connect with the last connected phone. There is no way to overcome this.