I definitely agree that it is temperature related. We got some colder temperatures for about a week and the problem went away, only to return once it warmed up again. I now make sure to use the A/C when I know I'm going to remove the faceplate and never have problems. Still it is strange that the motor for the faceplate works fine when the faceplate is connected, regardless of temperature. Only when the faceplate is disconnected does it give me trouble.
I suspect that the angle I have the stereo mounted might be contributing to the issue as well. The stock mounts for the stereo in my Mazda