Honestly if performance is the main reason your keeping it, an XR6 will be a better choice. The problem is, most BF XR6's have been modded and tuned, and this itself presents a problem. A lot of tuners have NFI what their doing, and will be a costly exercise if it was never tuned right. Finding a stock BF XR6 MK2 is nearly impossible, and even if you do, there is a price tag problem, the owners think its worth more than it is. If you enjoy music off your iPod/phone the VE is the better choice, however its pretty easy to stick a powerful aftermarket unit in BF with bluetooth and steering wheel controls.
Honestly, if you find a BF thats not a MK2 (everyone wants the MK2 because of the gearbox - 6 speed auto) and without premium sound or leather, its not worth it. If your wanting a manual - your choices open up a lot more. A BF MK3 with all the bells and whistles as standard is rare to come by, but I'd be looking for one of them first.
At the end of the day, If the acceleration from 0-100 in your VE is enough - just add an exhaust. If its not, drive a few BF's and compare.
But if its outright performance and "my car is faster than yours" to all your mates, then all we'll see is another "I've lost my license" thread eventually. I held off for ages to get my G6ET tuned, because I knew I'd lose my license in it. I've lost a few points - but I also don't drive it everyday. because I know deep down, I can't help myself with the feeling of being thrown back into the seat from a standing start. But I do tend to back off before blitzing or doubling the speed limit. Its just thats its soon easy to do so in a turbo Falcon.
Long winded post, but I went through a similar thought process between a G6ET and a VE Calais V V8, and I thought I'd share.
One word of advice though - If your looking at a BF - Diff bushes are a massive issue and check the climate controls work, particularly if its duel zone.