Although most of our effort goes into building infrastructure as code, we sometimes set up temporary resources like a VPN connection used for migrating data. Things like these need to be decommissioned after a project goes live, so we need to be sure we're deleting the right resource. This can be a challenge in some situations. We're going to dive into an example situation where we need to delete a VPN connection while maintaining another. ## The issue In short, when using `aws_cdk.aws_ec2.Vpc(vpn_connections=[connection1, connection2])` and we want to delete `connection1` from the list of VPN connections, `connection2` will be inheriting the logical id from `connection1`. So instead of deleting `connection1`, CloudFormation will reconfigure `connection1` with the connection details from `connection2`. As you can imagine, the connection details are already in use by `connection2` as it is also still there, resulting in a failed deployment as you cannot configure two customer gateways with the same IP address. ## The setup AWS CDK handles Logical IDs automatically while ensuring uniqueness by combining names of some elements. For example, let's assume the following structure: ``` - Stack (id: MyStack) - VPC (id: Vpc) ``` In this example, the VPC has the following path: `MyStack/Vpc`. If we were to add two VPN connections using the VPC parameter `vpn_connections`, two children will be created under VPC, like so: ``` - Stack (id: MyStack) - VPC (id: Vpc) - 0 - CustomerGateway - VPNConnection - 1 - CustomerGateway - VPNConnection ``` As you can see, because we set `vpn_connections` using a list, the items in this list are numbered by CDK. Removing 0 would move 1 up to 0. You can see the path is also numbered like this: `MyStack/Vpc/0/CustomerGateway`. Because of how CloudFormation handles Logical IDs, they behave the same, blocking the removal of the first VPN connection. ![Resource Path](//images.contentful.com/9gzi1io5uqx8/5aALcyNh2HhR52xs4O9Pnh/0f89f4c149782a5f229c4a6e91ff3ce6/ResourcePath.png) ## Logical IDs explained Logical IDs are a combination of the following elements: - Resource ID - Child Number (if present) - Resource Type (if child present) - Hash of above elements (Unique ID) As the logical ID can't contain any non-alphanumeric characters, they are removed from the resouce path part of the ID, but the hash is created with the resource path still containing the slashes. ![Logical ID](//images.ctfassets.net/9gzi1io5uqx8/1qBffsyE8o50DqRnuMPA2D/12afdceaaead6c8ab6c6844f73a82539/LogicalID.png) Let's recreate the logical ID for both VPN Gateways in this example. We have `Vpc` which has child `0` that contains resource `CustomerGateway`. Put together the path is `Vpc/0/CustomerGateway`. Now let's hash this path. CDK uses MD5 to calculate the hash, only uses the first eight characters and uppercases it. So the result of `Vpc/0/CustomerGateway` is `b31c975a527d15502aca002fec5376cd`. Just taking the first eight characters and uppercasing them results in `B31C975A`. The last steps are to remove all non-alphanumeric characters from the path, and joining the result with the hash. This gives us `Vpc0CustomerGatewayB31C975A`. Quickly summarizing the second example: - Path: `Vpc/1/CustomerGateway` - MD5 Hash: `3df078000a381dd296f21293646f6c50` - Uppercased + shortened hash: `3DF07800` - Result: `Vpc1CustomerGateway3DF07800` The logical ID for the VPN Connections themselves are a bit shorter, as they do not contain the resource type in the resource path. So the Logical ID for the second VPN Connection in this example is as follows: - Path: `Vpc/1/Resource` - MD5 Hash: `2a1da4e9d329e9d56f21382623c13dc8` - Uppercased + shortened hash: `2A1DA4E9` - Result: `Vpc12A1DA4E9` Do note the first number (`1`) in this logical ID is not part of the hash, but the index of the list. ## Overriding the Logical ID Now we know how the Logical ID is created, we know we should manually set it to prevent it from being renamed. In our example, we have two VPN connections of which we need to delete the first. That means we need to override the Logical ID of the second VPN connection. To do this, we need to access the resource in the CDK structure. To help us find the proper resource to override, check out [this article](https://www.sentiatechblog.com/debugging-a-cdk-python-project-in-visual-studio-code "Debugging a CDK Python Project in Visual Studio Code") about debugging a Python CDK project. The following code creates a VPC and uses the `vpn_connection` parameter to set up two VPN connections. Let's go from there to see what we need to do to remove the first of the two VPN connections. ```python #!/usr/bin/env python3 import os from aws_cdk import ( core as cdk, aws_ec2 as ec2 ) app = cdk.App() class MyStack(cdk.Stack): def __init__(self, scope: cdk.Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) self.vpc = ec2.Vpc( self, f'Vpc', vpn_connections=[ ec2.VpnConnectionOptions( ip='8.8.4.4' ), ec2.VpnConnectionOptions( ip='8.8.8.8' ) ] ) MyStack(app, "MyStack", env=cdk.Environment()) app.synth() ``` As we've added the VPC resource to `self.vpc`, we can access its children like so: ```python self.vpc.node.children[00] # First VPN connection self.vpc.node.children[01] # Second VPN connection self.vpc.node.children[00].node.children[0] # Customer Gateway of first VPN connection self.vpc.node.children[00].node.children[1] # VPN Connection of first VPN connection ``` As you can see, both VPN connections have two children of their own. A Customer Gateway resource and a VPN Connection resource. Both need to have their logical IDs replaced, otherwise you'll end up with only half the configuration. Accessing the children using the list index isn't very reliable as we are never sure CDK will keep the VPN connections on top of the list. There are many children following `00` and `01`, like subnets and routes. Since there is one value we can pin a VPN connection on, namely the customer gateway IP, we can iterate the list and only get the VPN connection with the correct IP address. We can do that like so: ```python def pinVpnLogicalID( vpc: ec2.Vpc, customer_gateway_ip: str, vpn_logical_id: str, gateway_logical_id: str ): """ Overrides the Logical ID of the VPN connection matching the customer gateway address. Arguments: vpc -- The VPC resource containing the VPN connection customer_gateway_ip -- The IP address of the remote gateway vpn_logical_id -- The logical ID the VPN will have gateway_logical_id -- The logical ID the Customer Gateway will have """ for child in vpc.node.children: if isinstance(child, ec2.VpnConnection): if (child.customer_gateway_ip == customer_gateway_ip): for resource in child.node.children: if isinstance(resource, ec2.CfnVPNConnection): resource.override_logical_id(vpn_logical_id) elif isinstance(resource, ec2.CfnCustomerGateway): resource.override_logical_id(gateway_logical_id) ``` This function ensures we only adjust the VPN connection we want. To use it, first we need to get the current logical IDs of both the VPN connection and the Customer Gateway. You can easily find these in the CloudFormation Template output of your project. In this case, the VPC configuration is written to `cdk.out/MyStack.template.json`, so we can look up the details there. Let's call the new function with this information: ```python pinVpnLogicalID(vpc=self.vpc, customer_gateway_ip='8.8.8.8', vpn_logical_id='Vpc12A1DA4E9', gateway_logical_id='Vpc1CustomerGateway3DF07800') ``` If everything went as should, no changes should be visible in `MyStack.template.json`, meaning we've pinned the logical id correctly. If you want to be sure it is working properly, you can change the to be Logical IDs to something arbitrary to verify the template is indeed altered. Next, delete the first VPN connection from the `vpn_connections` parameter. The CloudFormation template should now only show the first VPN connection was deleted. See the example below: ```diff $ diff before.json cdk.out/MyStack.template.json 3,41d2 < "Vpc0CustomerGatewayB31C975A": { < "Type": "AWS::EC2::CustomerGateway", < "Properties": { < "BgpAsn": 65000, < "IpAddress": "8.8.4.4", < "Type": "ipsec.1", < "Tags": [ < { < "Key": "Name", < "Value": "MyStack/Vpc" < } < ] < }, < "Metadata": { < "aws:cdk:path": "MyStack/Vpc/0/CustomerGateway" < } < }, < "Vpc09B4D858B": { < "Type": "AWS::EC2::VPNConnection", < "Properties": { < "CustomerGatewayId": { < "Ref": "Vpc0CustomerGatewayB31C975A" < }, < "Type": "ipsec.1", < "StaticRoutesOnly": false, < "Tags": [ < { < "Key": "Name", < "Value": "MyStack/Vpc" < } < ], < "VpnGatewayId": { < "Ref": "VpcVpnGatewayB9B8A904" < } < }, < "Metadata": { < "aws:cdk:path": "MyStack/Vpc/0/Resource" < } < }, 56c17 < "aws:cdk:path": "MyStack/Vpc/1/CustomerGateway" --- > "aws:cdk:path": "MyStack/Vpc/0/CustomerGateway" 78c39 < "aws:cdk:path": "MyStack/Vpc/1/Resource" --- > "aws:cdk:path": "MyStack/Vpc/0/Resource" ``` This is the complete example with the first VPN commented out: ```python #!/usr/bin/env python3 import os from aws_cdk import ( core as cdk, aws_ec2 as ec2 ) app = cdk.App() class MyStack(cdk.Stack): def __init__(self, scope: cdk.Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) self.vpc = ec2.Vpc( self, f'Vpc', vpn_connections=[ # ec2.VpnConnectionOptions( # ip='8.8.4.4' # ), ec2.VpnConnectionOptions( ip='8.8.8.8' ) ] ) pinVpnLogicalID(vpc=self.vpc, customer_gateway_ip='8.8.8.8', vpn_logical_id='Vpc12A1DA4E9', gateway_logical_id='Vpc1CustomerGateway3DF07800') def pinVpnLogicalID( vpc: ec2.Vpc, customer_gateway_ip: str, vpn_logical_id: str, gateway_logical_id: str ): """ Overrides the Logical ID of the VPN connection matching the customer gateway address. Arguments: vpc -- The VPC resource containing the VPN connection customer_gateway_ip -- The IP address of the remote gateway vpn_logical_id -- The logical ID the VPN will have gateway_logical_id -- The logical ID the Customer Gateway will have """ for child in vpc.node.children: if isinstance(child, ec2.VpnConnection): if (child.customer_gateway_ip == customer_gateway_ip): for resource in child.node.children: if isinstance(resource, ec2.CfnVPNConnection): resource.override_logical_id(vpn_logical_id) elif isinstance(resource, ec2.CfnCustomerGateway): resource.override_logical_id(gateway_logical_id) MyStack(app, "MyStack", env=cdk.Environment()) app.synth() ``` ## Other ways After finishing this article and sharing it internally for reviewing, my colleague Lourenz Alcantara came to me with some interesting information. For `vpn_connections`, a dictionary can also be supplied. This is because this parameter is mapped. When you have this specific situation, you can resolve it by replacing the list with the following dictionary: ```python vpn_connections={ 0: ec2.VpnConnectionOptions( ip='8.8.4.4' ), 1: ec2.VpnConnectionOptions( ip='8.8.8.8' ) } ``` This will result in exactly the same output as with the list. Just delete key 0 and key 1 will be intact.

Articles by Nathan Wouda

Decommissioning as Code: Overriding Logical IDs when resources in a list are no longer needed