My VX SS was fun and responsive with a little cam, full exhaust (pacey tri-y to single 3.5"), 2800 stall, Craigs built 4l60e with a trans cooler, 3.9 gears and some new FE2 suspension with 255/40 17 Hankook RS4 on the rear. It would bag the tyres through the gears if I wanted to, but I could still sit in traffic or drive from Sydney to Brisbane with the family, and it was a pleasure
Not the quickest thing, but it sounded good and you could give it a hard time, and it was a great street car with good manners, and quicker than most SUV's. This sounds like what you want
1 year later I did think of going a bigger cam and some Higgins heads, but after going for a drive in a few "really lumpy" cars that really only made power up high, I chose a different path, and I ticked a bucket list item, a blower (HTV2300)
And whilst the blower was slightly more expensive, I'm glad I went that way, the car drives brilliantly and has excellent street manners, and way more streetable power than a big head and cam package could ever give
Opinions may vary