I went to my local Holden dealer to look at buying an SS but came away having ordered an SV8. At the time, the difference between the 2 was between $9k and $10k (the dealer offered the same discount % on both models). There was also a much longer waiting time for the SS than the SV8, and that influenced me. I also got a window tint, SS steering wheel, rear electric windows and side airbags (all factory options) thrown in for free.
The SV8 is mechanically almost identical to the SS; in fact some magazines like "MOTOR" have found the SV8 marginally faster because it weighs about 45kg less than big brother. I also find that the SV8's bodykit (more Exec than SS) is better suited to country roads- less damage.
But if you're concerned about the SV8's plain jane looks, consider the most obvious differences there are: wheels, bodykit, and interior. If you shop around, you'll find:
1) You can easily buy a set of genuine VZ 18" SS alloys with new tyres for a touch over $1800 delivered to your door;
2) the front and rear bars are about $350 each from Holden (yes you will have to have them painted, but even so...). And the 2005 SV8 models now have the same bumper bars and fog lights as the SS;
3) A quick look on eBay or magazine classifieds can usually find a full SS interior second hand for around $1000 (velour) and $2000 (leather).
I agree that for a lot of people, particularly to whom road image is important, the SS is "the one to have", and that's fully understandable- the SS is an awesome looking car. But if you really want to save money, or to have a very similar ride to your mate with an SS- (but for a lot less)- the SV8 is definitely a terrific bargain.