VPC: Connecting Networks

In part thirteen we built an isolated network. But real applications rarely stay alone: you have a second VPC for another team, servers that need S3 and an office that needs to reach the database.

Here's this part's map, to come back to whenever you're unsure:

VPC peering #

Peering connects two VPCs directly, as if they were one network. It works even between different accounts or different Regions. Its rules:

• Ranges can't overlap. This is what we warned you about in the previous part.
• It isn't transitive. If A connects to B and B connects to C, A can't reach C.
• You need a route on both sides. You add one in each network's route table.

There's no hourly charge; you pay only for data transfer.

VPC endpoints: AWS services without the internet #

By default, your private server's request to S3 goes out through NAT, and you pay for every gigabyte, even though the destination is already inside AWS. Endpoints give you a private, direct path:

An interface endpoint is built on PrivateLink, which also lets you offer your own service to other networks through an NLB, with no peering and even if their ranges overlap.

Connecting your office: Site-to-Site VPN #

Site-to-Site VPN connects your office network to the VPC over the internet through an encrypted tunnel, and you can set it up quickly. It has three pieces:

1. Customer gateway: represents your office device and its public address.
2. Virtual private gateway: the VPN gateway on AWS's side, attached to the VPC.
3. VPN connection: the connection itself, which comes with two tunnels for availability.

Then turn on route propagation in the route table so the VPC learns the office's routes. Each tunnel gives you up to 1.25 Gbps, or up to 5 with Transit Gateway.

• VPN CloudHub: connects several offices to each other through the same gateway.
• Client VPN: for employees' devices rather than offices.

If you want a VPN fully under your control on your own server, see the WireGuard article.

Direct Connect: a private line #

Direct Connect is a real private line between your site and AWS through a Direct Connect location, so it doesn't cross the public internet. Its speed is steady, and outgoing data transfer is cheaper.

• Dedicated: a port just for you, from 1 to 400 Gbps.
• Hosted: through a partner, with speeds starting far lower.
• Direct Connect Gateway: connects you to VPCs in several Regions.

Watch out for three things: setup takes weeks and sometimes months, the line isn't encrypted automatically, so use MACsec or a VPN on top, and one line is a single point of failure, so for your most important systems have two lines in two locations, or a backup VPN.

Transit Gateway: one hub #

As networks multiply, peering becomes tiring: since it isn't transitive, you need a link for every pair:

Transit Gateway is a central hub you connect networks to, and each one reaches the others through it:

• It connects everything: VPCs, VPN connections, Direct Connect and Transit Gateways in other Regions.
• Its own route tables: you decide who talks to whom, for example separating production from development.
• Across accounts: you share it with other accounts through the RAM service.
• Higher VPN speed: it spreads traffic across several tunnels.

You pay by the hour for each attachment, and for data. For two or three networks, peering is simpler and cheaper.

Network costs #

Data transfer is the item that most often surprises people. These are approximate rates for US Regions, so check yours:

How to save:

• Use private addresses between servers, not public ones.
• Keep services that talk a lot in the same Availability Zone, if your availability design allows it.
• Turn on a gateway endpoint for S3.
• Serve content through CloudFront, since transfer from your origin to it is free.

Layers of protection #

As for IPv6: you can add it to a VPC, but all its addresses are public. If you want servers to go out with nobody coming in, use an egress-only internet gateway, which is like NAT for IPv6.

Hands-on: an endpoint and peering #

We'll continue with the network from part thirteen, after deleting the NAT Gateway, so private-server has no route to the internet. The commands use the variables VPC and RT_PRIVATE; if your CloudShell session ended, set them by hand from the VPC page.

Gateway endpoint #

Give private-server an IAM role with AmazonS3ReadOnlyAccess, as we explained in part two. On the server, try:

aws s3 ls --cli-connect-timeout 5

It times out after several attempts, because there's no route to S3. Now add an endpoint from CloudShell:

aws ec2 create-vpc-endpoint --vpc-id $VPC \
  --service-name com.amazonaws.$AWS_REGION.s3 \
  --route-table-ids $RT_PRIVATE

aws ec2 describe-route-tables --route-table-ids $RT_PRIVATE \
  --query "RouteTables[0].Routes[].[DestinationCidrBlock,DestinationPrefixListId,GatewayId]" \
  --output table

You'll see a new route starting with pl- and pointing to vpce-. Repeat aws s3 ls on the server, and it returns at once with your buckets, or empty if you have none, with no internet at all.

VPC peering #

Now we'll connect this network to the default one, which holds web-1 from part three. Get its private address:

aws ec2 describe-instances \
  --filters "Name=tag:Name,Values=web-1" "Name=instance-state-name,Values=running" \
  --query "Reservations[0].Instances[0].PrivateIpAddress" --output text

From private-server, try curl -m 5 http://WEB-PRIVATE-IP, and it times out. Now create the connection and accept it:

DEFAULT_VPC=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
  --query "Vpcs[0].VpcId" --output text)

PEER=$(aws ec2 create-vpc-peering-connection \
  --vpc-id $VPC --peer-vpc-id $DEFAULT_VPC \
  --query VpcPeeringConnection.VpcPeeringConnectionId --output text)

aws ec2 accept-vpc-peering-connection --vpc-peering-connection-id $PEER

The connection alone isn't enough, so add a route on both sides:

aws ec2 create-route --route-table-id $RT_PRIVATE \
  --destination-cidr-block 172.31.0.0/16 --vpc-peering-connection-id $PEER

DEFAULT_RT=$(aws ec2 describe-route-tables \
  --filters Name=vpc-id,Values=$DEFAULT_VPC Name=association.main,Values=true \
  --query "RouteTables[0].RouteTableId" --output text)

aws ec2 create-route --route-table-id $DEFAULT_RT \
  --destination-cidr-block 10.0.0.0/16 --vpc-peering-connection-id $PEER

Repeat curl from private-server, and web-1's page comes back over the private address.

aws ec2 delete-route --route-table-id $DEFAULT_RT \
  --destination-cidr-block 10.0.0.0/16

aws ec2 delete-vpc-peering-connection --vpc-peering-connection-id $PEER

Summary #

• Peering is for two networks and isn't transitive; Transit Gateway is for when networks multiply.
• A gateway endpoint is free for S3 and DynamoDB; an interface endpoint covers the other services.
• VPN is quick to set up over the internet; Direct Connect is a private line that takes time.
• Data transfer is among the biggest surprises on a bill, so plan for it from the start.

Test yourself #

Before you move on, try these questions to check you've got this part. Each one is a practical scenario, and the explanation appears once you answer.

That completes the networking section. You now know the visitor's path from your site's name to its data, and what's left is a place to store anything at any size. That's the next section's subject, starting in part fifteen with the basics of S3.