I have an iPhone 4s with appex beta 11. I have experienced the slightly resized screen with triangles a few times. It has only ever been on unsupported apps, but not all the time. I use Navigon frequently and sometimes it is slightly smaller with triangles on the sides and sometimes it is normal sized with black bars on the sides. I don't know exactly what I did to fix it, but I think it was either closing Navigon on the phone and reopening it or maybe I unplugged the phone and plugged it back in. It might have to do with whether the unsupported app was already running on the phone in the background when the phone is plugged into the radio. I'll try to be more conscious of what I do to resolve it the next time it happens, or even purposely try to see if I can recreate the issue.
I've noticed same thing with any unsupported app. Sometimes it fills up most of the screen (w/triangles) and sometimes slightly smaller. No idea what causes it for me either.