Kubernetes backup data private with Azure private link
At CloudCasa, as a cloud-native backup-as-a-service provider, we are often dealing with customer concerns about the security and privacy of their data. Sometimes the concern is almost paradoxical because the data and application they are protecting is publicly accessible and running inside the public cloud. However, there are times when organizations use public cloud infrastructure with a network architecture that resembles a private cloud. This allows them to leverage the elasticity and economies of public cloud while maintaining an elevated security posture.
Azure Private Link (like AWS PrivateLink) helps SaaS providers running in the public cloud conform to client security postures by restricting network traffic to be private and secure – regardless of the encryption used. Both AWS and Azure offer similar constructs and while we will highlight the benefits of both services, this article will showcase how you can keep your Kubernetes backup data from traversing public networks using Azure Private Link.
What are Private Links?
Private links provide private connectivity between VPCs/vNets and your cloud services without exposing your traffic to the public internet. Private links help simplify your network architecture without compromising on security and scalability. These also serve as an abstraction for a user, who would otherwise need to deal with NAT, Direct Connect and VPN configuration.
Azure highlights 5 key reasons for Azure Private Link service:
- Privately access services on the Azure platform
- On-premises and peered networks without ExpressRoute Peering
- Protection against data leakage
- Global reach and private connection to services across regions
- Extension to your own services
Why Keep your Backup Data Private?
Network traffic that uses private links does not traverse the public internet, reducing exposure to brute force and distributed denial-of-service attacks, along with other threats. Cloud providers allow network policies to be used alongside these links to allow you to control precisely who has access to a specified service.
Backups are quickly becoming the new borders for organizations. Anything critical to your organization is being backed up and protected (one would hope!). Cyber-attackers scour through backup traffic to potentially exfiltrate important data and even worse, potentially compromise your primary insurance against data loss. Private links help avoid that risk altogether and help mimic in-house backup behavior.
How Prevalent are Private Links?
Azure Private Link was GA in March 2020 for Kubernetes as well as Azure Storage – the two areas we care most about. A quick search online shows support for Azure Private Link from Elasticsearch, Confluent, Snowflake, NetApp, Telstra and more.
Amazon highlights AWS PrivateLink the benefits and usage by large organizations such as Discover, Salesforce, Autodesk, and Goldman Sachs. While AWS Backup also supports PrivateLink, it does not yet support backups for Kubernetes.
Network Layout of Azure Private Link
Azure offers two related components: Private Endpoint and Private Link service.
A Private Endpoint is a network interface that uses a private IP address from your virtual network. Multiple Private Link resource types support access via the Private Endpoint. These resources include Azure platform services and your own Private Link Service. It is a one-to-many relationship.
Private Link enables you to access Azure services like Azure Storage and SQL Database, and Azure hosted and customer-owned, partner services over a private endpoint in your virtual network. In short, it acts as a bridge between Azure services and your virtual network.
The Private Link service receives connections from multiple Private Endpoints. A Private Endpoint connects to only one Private Link service.
Setting up an Azure Private Endpoint for Azure Storage
We will demonstrate creating an Azure Private Endpoint for an Azure Blob Storage bucket and then use it for directing backups from an AKS cluster.
Steps to create a private endpoint in Azure:
- In the search box at the top of the portal, enter Private endpoint and select Private endpoint.
- Select + Create in Private endpoints.
- In the Basics tab of Create a private endpoint, enter, or select the following information.
5. In the Resource pane, enter or select the following information.
7. In Virtual Network, enter or select the following information.
10. Leave the defaults in DNS. Select Next: Tags, then Next: Review + create.
11. Select Create.
For those who are not mouse-friendly, the setup can also be created using the CLI:
Setting up a Private S3 Object Store in CloudCasa
CloudCasa supports three different storage types:
- CloudCasa Managed Storage – select the Cloud Provider and Region of your choice.
- Customer Managed Public Storage – for storage reachable through the public Internet. All storage and retention management functions are managed by the CloudCasa service.
- Customer Managed Private Storage – for storage inaccessible to the CloudCasa service. All storage and retention management functions are offloaded to a CloudCasa agent on a customer cluster.
Cloud object storage accessed via Private Link will fall under the third category. It can be configured in CloudCasa as follows:
- Login in to the CloudCasa service.
- From the menu bar, go to Configuration > My Storage.
3. Click Add storage on the right-hand side above.
In the next screen, select Yes to the question on whether your storage is isolated/restricted from the Internet.
Next enter a proxy Kubernetes cluster where the CloudCasa agent is already installed. This cluster will be responsible for maintenance operations on the storage such as removing expired backups, and it must be able to access the Private Endpoint that you created in the previous step.
We named this storage AzurePrivate in our example. The remaining fields below correspond to your Azure account.
- You can select the new AzurePrivate storage as the default destination for the clusters whose backup data needs to remain on private networks.
2. Alternatively, you can choose it during the backup job definition.
- Recovery within Azure: An obvious but important consideration is that you can only restore to a cluster that can access the blob storage through the Private Link/Endpoint. You cannot use this recovery point to restore to a vNet that has no connectivity to this Private Endpoint.
- Private endpoints may not be your answer if you are trying to do cross-cloud protection. While each cloud provider offers Private Endpoint functionality, they are not compatible with each other.
- Regional Failures could be designed around in some cases, but we did not dig deep into this possibility.
- Private Endpoints are not free. Data traversing through these Private Links could incur a charge.
- Set up your DNS wisely with Private Links. Remember that the DNS names of various services are designed to resolve to their Public IP addresses. A “split horizon” DNS configuration may be necessary.
- You can setup multiple private endpoints in the same vNet. If you are already using Private Endpoint for a database service, you can create a new one for backup storage. You also do not require a dedicated subnet for each Private Endpoint.
- One Azure Private Link Service can handle up to 8 Azure Private Endpoints (i.e., a maximum of 8 IP addresses)
- Azure Private Link requires network policies to be disabled.
- A read-only NIC is assigned to your Private Endpoint on creation. It is immutable and cannot be changed during its life.
- Standard Load Balancer must be used with Azure Private Link. Basic Load Balancer is not compatible with Azure Private Link.
Why use a SaaS Backup
There are many reasons why to use CloudCasa, a Backup-as-a-Service offering, including: zero infrastructure (software and storage included), cloud account integration including auto discovery, cloud services awareness and integration, security and compliance, and more. Keeping backup data private is the most common concern with using a cloud service. Private links from Azure (and AWS) really allow users to get the best of both worlds.
References
https://azure.microsoft.com/en-us/blog/announcing-azure-private-link/
https://aws.amazon.com/privatelink/
https://docs.microsoft.com/en-us/azure/private-link/private-link-faq
Bob Adair
Brian Sietsma