There is no perfect security. Ever. A sufficiently determined attacker can and will succeed against anything you can do, given that sooner or later somebody involved will make an opsec mistake somewhere. Or the existing attacks against MD5, RC4 and SSL/TLS security — attacks including BEAST, CRIME Lucky 13, etc — will continue to be "weaponized".
Firewalls and VPNs only get you so far, and it's common for attackers to use a variety of attacks to try to breach those; to bypass the network security. So-called "spearphishing" tries to get somebody on the network to breach security for the attacker. The best VPN and the best firewall are worth nothing if you have Java lit in your web browser and the Java JVM sandbox gets breached (again), or if you receive and open a document that contains malware, for instance.
Facebook and other entities were recently breached using what was known as a watering hole attack, and that was only spotted based on detecting "odd" out-bound network traffic. The attack got around the firewalls and the VPNs and the rest of the security, and was active on the organizations' internal networks.
If you're securing nuclear secrets or large sums of money or exceedingly embarassing or sensitive data, then you definitely and certainly do need to focus on this stuff, and you're going to be spending time and effort and money on making your organization harder (emphasis on harder) to attack. But attacks will continue.
If you're dealing with a home network or a typical a small business network, then you just don't want to be the lowest of the low-hanging fruit around, and you want to avoid opsec mistakes such as open ports or weak passwords, and you don't want to give the good folks of the Internet reasons to attack you. You want to be not worth attacking, or not as "fun" and not as valuable to attack.
Even if your security is not attacked, a DDoS can still ruin your day.
As I've mentioned elsewhere, I much prefer using a VPN server in a gateway-firewall-router device — as VPNs and NAT don't mix very well — and I do use private certificate authority chains. But in terms of attacks? Keep your software and your security current, review your logs and your rules, DMZ any services you provide to "outside", maintain and verify backups — those backups can be your recovery path from a breach — and start looking at "odd" or "unexpected" outbound traffic, too. VPNs are just part of avoiding the mess of a cleanup.
I am in the middle of setting up a Cisco IPSec (compatible) VPN server using Racoon running in a Linux VM connected to Open Directory via LDAP for user authentication. I am now adding a Certificate rather than a pre-shared key to this setup.
I have created a self-signed CA, and server certificate (which are working), and understand how to also create a client certificate using OpenSSL for use with the VPN server. I was however wondering if there is any better way other than tediously creating client certificates one at a time by hand for each device.
I have chosen to go the Cisco IPSec route rather than an SSL VPN to avoid the use of Java and also having to install additional client software.
There's certtool available in the OS X distro that can streamline some of this (particularly if you're willing to do some scripting), and there's Profile Manager in recent OS X Server releases. (And see HT4837, depending on what you're up to with LDAP authentication.) There are various commercial provisioning and device management tools around, too. Apple has a high-level iOS provisiong overview document available, which can be helpful with the buzz-phrases currently in use. There are also related open source around, such as the Java-based Odyssi PKI package, though I haven't tried that one.
As for your set-up, that would appear quite solid. I'm fond of setting up the private CA chain, as you're doing.
IP routing among and through a VM can sometimes get "interesting", but that's another discussion, and you probably already know that.
My preference would be a VPN server implemented in the gateway box, as that can then allows access to the LAN, and to the other hosts.
The usual attacks against these configurations — assuming you're valuable enough or interesting enough or controversial enough to target — are then spearphishing or watering hole or related attacks. Attacks that bypass the firewall and the VPNs. Or DDoS attacks, as those can be rented for cheap.
Thanks for the pointers, Odyssi looks particularly interesting. I had contemplated writing something very similar to that in FileMaker Pro as a front-end to OpenSSL.
My gateway supports L2TP and SSL style VPNs but not Cisco IPSec and as explained previously I wanted to use Cisco IPSec. I already have the Linux VM acting as a Cisco IPSec server working (and routing sorted) in currently pre-shared key mode.
Of course I have now seen the very bad news about Apple vs VirtnetX and the fact Apple is going to disable the current VPN on Demand function.
Hmm, Odyssi does not look like it has been updated since verison 0.1 released in 2006! That's a shame.