Setting up AWS dynamic VPN using Transit Gateway attachments and BGP

Note

This blog post is inspired from one of Adrian Cantrill’s labs in his AWS Advanced Networking course. I highly recommend his content, it helped me pass all of my AWS certs. Feel free to check his courses. The setup for strongswan and FRR BGP was taken from this blog post.

Architecture

The purpose of this blog post is to demo highly-available AWS VPN with Transit Gateway (TGW) attachments and highlight the steps to setup the TGW, the IPSec tunnels and BGP routing. Setting up VPN is an essential feature in all hybrid cloud estates because it improves the organization’s security posture by encrypting data transmissions and facilitating secure access across both cloud and on-prem environments.

As part of this lab we will implement the architecture below:

Detailed steps

Deploy the Terraform code. It will create 1 cloud VPC with 2 private subnets and 2 EC2 instances. It will also create 1 onprem VPC with 2 private subnets and 1 public subnet. The 2 private subnets will contain 2 private EC2 instances and the public subnet will contain 2 Ubuntu 18 instances which will serve as onprem routers. The cloud VPC will be attached to a TGW. 2 VPN connections will be created from the TGW to the onprem routers and Global Accelerator will be enabled on both connections.
If you go and check the tunnel details for the 2 VPN connections under the ‘Site-to-Site VPN connections’, you will see that both tunnels are down and ‘IPSEC IS DOWN’ under details.

Let’s configure the IPSec tunnels for each of the onprem routers. Download the configuration for each of the VPN connections and keep track of which file corresponds to which router (this can be done by comparing the public IP of the customer gateway with the public IP of the onprem routers and making sure they match).
Connect to the onprem-router-a via SSM. Change directory to /home/ubuntu/vpn_setup. Make the necessary changes to the ipsec.conf, ipsec.secrets and ipsec-vti.sh by replacing the placeholder values with the right inside/outside IP addresses and pre-shared keys from the corresponding configuration file you downloaded in step 3. The private ip of the router in ipsec.conf needs to be replaced with the private ip listed in the Terraform outputs (NOTE: this is the private ip of the public ENI, not the ENI responsible for BGP).
Copy all the files you modified to /etc, make /etc/ipsec-vti.sh executable and restart strongswan:

systemctl restart strongswan

You can see the newly created tunnels server-side in the below screenshot:

The VPN connection corresponding to this onprem router now shows that IPSEC is up:

Repeat the same process for the second onprem router. Now we need to setup the Dynamic BGP connection.
When we add the BGP session, the onprem routers will be able to exchange routes with the TGW and vice-versa. We need to install FRR on the onprem routers. Connect via SSM to the onprem routers, make the frr-install.sh script executable and run it. This will take around 10 mins, you can run the script on both onprem routers at the same time.
After FRR is installed, you will interact with it using vtysh, a CLI for FRR daemons. You will need to run the following sequence of commands:

vtysh
conf t
frr defaults traditional
router bgp 65020
neighbor TUNNEL1_AWS_BGP_IP remote-as 64512
neighbor TUNNEL2_AWS_BGP_IP remote-as 64512
no bgp ebgp-requires-policy
address-family ipv4 unicast
redistribute connected
exit-address-family
exit
exit
wr
exit
sudo reboot

Run the above commands on both onprem routers and wait for them to reboot.

After the onprem routers rebooted, the tunnels should now have an UP status:

The BGP routes can be seen from the onprem router (you first need to run vtysh before running the below command):

Testing

After completing all the steps, now it is testing time! The purpose of this lab is to achieve connectivity between onprem servers and cloud servers.

Below you can see the output from the ping commands from onprem to cloud and vice-versa:

Cloud to onprem: alt

Onprem to cloud: alt

Conclusion

Congratulations, now you have a highly available, dynamic VPN! You achieved the following:

Deployed 2 customer gateway objects for your onprem routers and assigned them your onprem BGP ASN.
Created 2 VPN connections and terminated them at the TGW. This generated 2 VPN TGW attachments and propagated the routes in the TGW route table, making the routes advertised via BGP from onprem available to the cloud instances. You also enabled acceleration for the connections. This will use Global Accelerator to route traffic to an AWS edge location closest to your customer gateway, improving latency.
Directed all onprem traffic to the TGW for the cloud instances and all cloud traffic to the Network Interfaces of the onprem routers for the onprem instances. You created the necessary route tables, routes and route table associations to achieve that.
Downloaded the VPN connection configuration and configured IPSec tunnels using strongswan.
Configured BGP routing using FRR for enabling dynamic routes.

I hope this demo was useful and provided a good foundation for deploying a VPN in AWS. Happy learning!

Alex Stan