Cloud Migration Rookie Mistakes – Virtual IP Addressing

TCP/IP all things! The public Cloud is the ultimate software-defined network. By design, Layer-2 networking isn’t exposed to customers in any major cloud provider except for Oracle Cloud Infrastructure. For these networks, implementing concepts such as zero-trust becomes a default part of the experience and service. For native cloud applications, developers adopt the idea of IP to IP application connectivity. However, what does it mean for traditional applications you wish to run in the Public Cloud?

Layer-2 networking enables physical Ethernet to physical Ethernet communication without a Layer-3 addressing protocol such as IP or if you are an old school network admin IPX.

What does that mean, practically? Well, let us dig in on how this will impact some of your plans to migrate workloads to the private Cloud.

Quick Refresher on Network Addressing

Amazon Web Services (AWS), Azure, or Google Cloud Platform (GCP) didn’t wholly re-invent network concepts. While there aren’t physical Ethernet addresses exposed to customers, the notion of MAC address resolution exists. MAC addresses are the physical addresses assigned to network adaptor cards. In a physical network, MAC addresses resolve to a logical IP address. It’s these IP addresses developers use to establish node to node communication at the application layer.

ARP is the protocol used to perform MAC address to IP address resolution. ARP requests are layer-2 network packets. Here’s the process for establishing IP connectivity between two nodes on a single Ethernet network. 

1. The requesting node floods the network with an ARP Request 

2. Each node on the internet segment receives the ARP request

3. The target workstation responds to the ARP request using the targets MAC address.

4. To limit the impact of broadcasts in a corporate network, Ethernet switches have a MAC address table that helps the network identify where machines are on the network. 

ARP represents the foundation for allowing two or more nodes on the same physical network to communicate via IP address. Remember, in the Public Cloud, layer-2 does not exist

How Does it Work in the Cloud?

An operating system running inside a VM instance still sends an ARP request to start communicating on a network. Instead of flooding the network with the request, a centralized service responds to each ARP request. 

Here’s a video I did a few years ago of what happens when two virtual machines establish IP communication, including the ARP request.

So What?

So, that means Public cloud providers can control the accessible IP addresses on their networks. The node or service must have the authority to register with the ARP service of the target cloud. Think of a VM running inside of a Linux machine. What needs to happen to expose a VM’s IP address to nodes or services external to the virtualization host?

On a private network, you leverage the virtual switch on the host node. The VM connects it’s virtual NIC to the virtual switch in bridge mode. Bridge mode allows the VM to flood the physical network with ARP request as if the VM were simply another physical interface on the network. It simply works.

However, in the public Cloud, this VM isn’t an authorized node on the network. Meaning the firmware or MAC Address isn’t something the public cloud provider recognizes. So, how do you implement technologies such as virtualization or containers?

You must leverage technologies such as application load balancers or NAT. The host OS must intercept, and forward requests for the VM or service running on the Cloud provided VM instance. The use case isn’t always this obvious. Think of network function virtualization as another example. These solutions use virtual IP addresses that may use bridge mode by default. It’s surprising how many applications leverage advanced bridge networking to expose services directly onto the Ethernet network attached to the host. 

A good indicator if your application will have network problems in the Public Cloud? Check to see if more than a single IP address resides on behind the target host or if the service you are migrating uses a virtual IP address.