The Snow Leopard VPN is not very configurable from the GUI, but behind the scenes it is using a racoon configuration.
To grab the configuration it is generating, configure the VPN in the System Preferences GUI, then rename /usr/sbin/racoon and try connecting. The config file will be written in /var/run/racoon/. Grap a copy of that file and customize it to your needs. Once you have the config file, rename racoon back to its original name.
Then to make the GUI use your custom config file instead of the one it generates, edit /etc/racoon/racoon.conf to include your custom config file and comment out the line:
include "/var/run/racoon/*.conf" ;
By making a few changes I was able to get a successful connection to our Cisco VPN Concentrators.
I’m hoping there is a less hacky way to accomplish this. If you know of one, let me know. Otherwise file a bug with Apple.