I've always had the behaviour
@Kakodaemon describes, and it DOES happen more often when changing keys ... except it seems to happen a "random handful" of restarts later.
For example my better-half mainly drives the car, she has her key & I have mine, and it sticks to "the same random playlist" the whole time she drives it for two weeks. I then drive it with my key on the weekend, it continues with "the same random playlist", I stop somewhere & when I get back in it continues again with "the same random playlist" ... but the NEXT time I stop, it switches to the other one. Sometimes it won't switch at all.
At first we wondered if it was because I keep my key in a cupboard that's about 5m from where the car parks, so maybe it was detecting my key when it started on occasion. But that doesn't seem to be related in any way (and besides, it won't start if my key's in the cupboard 'cos I forgot to grab it
).