My SV6 takes about 20 secs to begin blowing cooler air, and it's very cold within about 40 to 60 seconds.
One of the reasons you may get a stale small when you put the a/c on is the water condensing in the ductwork, causing a little mold or mildew. I never smell it in this car as I never turn the car off with the a/s running. I usually turn the a/c button off when I'm parking, and leave the fan running. By the time I'm out of the car it's blowing warm air into the car, but only for 30 secs or so.
When I get back in and turn on the airconditioning, the air is not tainted, stale etc. That hint was given to me by a dealer here and has worked for me in the new car so far