I definitely agree that it is temperature related. We got some colder temperatures for about a week and the problem went away, only to return once it warmed up again. I now make sure to use the A/C when I know I'm going to remove the faceplate and never have problems. Still it is strange that the motor for the faceplate works fine when the faceplate is connected, regardless of temperature. Only when the faceplate is disconnected does it give me trouble.
I suspect that the angle I have the stereo mounted might be contributing to the issue as well. The stock mounts for the stereo in my Mazda6 put the stereo at a significant upwards angle, possibly more than the 30 degrees they recommend. I'm working on getting it level and will report back if that helps at all. How level is the stereo mounted in your Jeep?
I have checked if it is rubbing against anything and it definitely isn't. Like I said, this is only an issue when I remove the faceplate. Everything moves fine when it's still connected.