Tape media management, media costs, 3rd party offsite contracts and the sheer volume of data growth makes tape backup challenging in any organization. The AWS Storage Gateway service offers a Tape Gateway configuration that gives you an alternative to physical backup tapes that fits seamlessly into your existing backup process. Now you have the local performance of disk, a low-cost highly scalable cloud back-end, and minimal disruption to existing systems.
What you’ll accomplish:
Provision a virtual host with local storage for your Tape Gateway.
Create virtual tapes in your gateway’s Virtual Tape Library. Virtual tapes are stored in Amazon S3 and available to your backup software through the VTL interface. This replaces tape and tape automation, so you can start using the cloud as a backup target with minimal disruption to existing systems and processes.
Back up to these virtual tapes using your existing backup software. The Tape Gateway enables backups to the cloud and preserves your existing software licensing investment, backup jobs, and catalogs.
Reduce your costs with long-term archival storage using Amazon Glacier. You can use your backup software to move a virtual tape into Amazon Glacier for further cost reductions.
What you’ll need before starting:
An AWS account: You will need an AWS account to download an AWS Storage Gateway and to create a VTL and virtual tapes. Sign up for AWS.
Hypervisor and host: The simplest approach is to run Storage Gateway on-premises as a virtual machine (VM). For more information, see the list of supported Hypervisors and Host requirements.
Existing 3rd party backup software to manage the backup jobs and catalog. For more information, see the list of Compatible Third-Party Backup Software.
Skill level: Experience with storage and backup technologies, networking, virtualization, backup tools, and tape solutions is recommended, but not required.
AWS experience: No working knowledge of Amazon S3 or Amazon Glacier is required.
Estimating your costs:
You are charged based on the type and amount of storage you use, the requests you make, and the amount of data transferred out of AWS. There are monthly data storage fees for tapes stored in S3 and for tapes archived in Glacier.
To see a breakdown of the services used and their associated costs, see Services Used and Costs.
Service usage and cost: AWS pricing is typically based on your usage of each individual service, but this project simplifies billing even further. You will see a single set of monthly charges that includes the AWS Storage Gateway service and the storage in S3 and/or Glacier that your virtual tapes consume.
Description: AWS Storage Gateway is a service connecting an on-premises software appliance with cloud-based storage to provide seamless and secure integration between an organization’s on-premises IT environment and AWS’s storage infrastructure. The Tape Gateway configuration provides you with a cost-effective, scalable, and durable virtual tape infrastructure that allows you to eliminate the challenges associated with owning and operating an on-premises physical tape infrastructure.
How Pricing Works: Replacing tape infrastructure includes a number of services and pricing is based in a number of factors: a monthly fee for each of your gateways, storage on the gateways, the virtual tape data you archive in AWS, retrieval fees, and data transfer charges. You are only billed for the portion of virtual tape capacity that you use, not for the size of the virtual tape you create. Data stored on your virtual tapes is compressed on the gateway, which can reduce both transfer and data charges, and you have both online and offline storage options in Amazon S3 and Amazon Glacier. The first 100GB of storage is free. For more details, see AWS Storage Gateway Pricing.
Example: Consider a single Tape Gateway configured with 10 full 100 GB tapes storing online backups using the US East (N. Virginia) region. We’ll plan for 5 more 100 GB tapes archived in the cloud, and plan to retrieve one of these tapes to restore a backup.
The first 100GB is free. In this example, that’s one tape stored locally.
Online (local) tape storage capacity costs $.023/GB/month, so 900 GB would cost $20.70/month (assuming zero compression).
Offline (cloud archive) tape storage capacity costs $.004/GB/month, so 500 GB would cost $2/month.
Archived tape retrieval costs $.01/GB, so a full 100 GB tape would cost $3 to retrieve.
The data transfer costs are $.09/GB, so 100 GB would cost $9 to transfer back to your premises.
This scenario protects 1TB of capacity and includes an additional 500 GB archive for a total or $22.70 per month. Retrieving a single, complete 100 GB virtual tape from your archive and reading the contents would cost $12 each time. Remember that this scenario assumes zero compression; your compression ratios will reduce these results.
Description: NOTE: In this project, the Storage Gateway service automatically manages Amazon S3 for you when you use a virtual tape. This section provide additional background.
How Pricing Works: Amazon S3 pricing is based on five components: the type of Amazon S3 storage you use, where you store your objects (e.g. US East vs. Asia Pacific – Sydney), the amount of data you store, the number of requests made to store or retrieve objects, and data transfer fees. For more details, see Amazon S3 Pricing.
Example: Let’s say you are using Amazon S3 Standard Storage in the US East (N. Virginia) region. To store 1 TB of content, you’d pay $0.03 per month.
PUT, COPY, and POST requests cost $0.005/1000 requests; GET and all others are $0.004/10,000 requests
Data transfers in are free. Transfers within region are free and to other regions are $0.020/GB. Transfers out to the Internet start at $0.090 after the first 1 TB and scale up after 10 TB.
Description: NOTE: In this project, the Storage Gateway service automatically manages Amazon Glacier for you when you use a virtual tape shelf. This section provide additional background.
Amazon Glacier is a secure, durable, and extremely low-cost cloud storage service for data archiving and long-term backup. Amazon Glacier is optimized for infrequently accessed data where a retrieval time of several hours is suitable.
How Pricing Works: Amazon Glacier pricing is based on the amount of data stored, the location the data is stored in, the number of requests made, and data transfer fees. For more details, see Amazon Glacier Pricing.
Example: Using Glacier in the US East (N. Virginia) region, storing 1 TB at $0.004/GB/month would be $4.00/month.
Upload and retrieval requests cost $0.05/1000.
Transfers in are free. Transfers within the same region are free and to other regions are $0.020/GB. Transfers out to the Internet start at $0.090 after the first 1 TB and scale up after 10 TB.
Q: What is cloud backup?
Cloud backup refers to the use of cloud storage as the media target for backups. Instead of backing up to disks, or tapes, you point to Amazon storage and perform backups and recoveries over your Internet (or Direct Connect) connection.
Q: Why should I back up data to the cloud?
The largest reason is cost. The combination of tape and disk media, the hours required for management, and offsite storage costs with a 3rd-party archival service can be significantly more expensive.
Q: What is the AWS Storage Gateway service?
The Storage Gateway service seamlessly extends your storage platforms into the cloud. It provides low-latency access to frequently accessed data and maintains durable remote storage, to help you match your storage costs with the value of your data. Use it with existing backup and recovery applications to replace tape automation without disrupting existing processes, or to supplement “bursty” on-premises workloads with storage capacity on demand.
Q: What are the benefits of using AWS Storage Gateway for backup?
Storage Gateway makes it very simple to eliminate tape. It can emulate tape automation, so now you can still perform a tape backup locally and move virtual tapes into the cloud. It delivers the performance of local infrastructure, the capacity of the cloud, and seamless integration with your existing backup environment.
Q. What if I don’t want to use up data center space for another appliance?
Storage Gateway is a virtual appliance, so it doesn’t take up any more physical space. But more importantly it can help you replace tape hardware and media space and costs, and replace them with lower-cost and higher-scale cloud storage.
Q: How much will it cost to back up my data using this implementation?
The costs associated with your backup project will be based on your Storage Gateway, which is billed per gateway per month, the amount of data you store and the type of virtual tape storage you choose (Amazon S3 Standard or Amazon Glacier). For more details, see Services Used and Costs.
Q: How long will it take to move data using a Storage Gateway?
It will vary based on your backup software job performance and your network bandwidth.
What Is AWS Storage Gateway?
Are You a First-Time AWS Storage Gateway User?
How AWS Storage Gateway Works (Architecture)
AWS Storage Gateway Pricing
Plan Your AWS Storage Gateway Deployment
AWS Storage Gateway connects an on-premises software appliance with cloud-based storage to provide seamless integration with data security features between your on-premises IT environment and the Amazon Web Services (AWS) storage infrastructure. You can use the service to store data in the AWS Cloud for scalable and cost-effective storage that helps maintain data security. AWS Storage Gateway offers file-based, volume-based and tape-based storage solutions:
File Gateway – File gateway is a type of AWS Storage Gateway that supports a file interface into Amazon S3 and that adds to the current block-based volume and VTL storage. File gateway combines a service and virtual software appliance, enabling you to store and retrieve objects in Amazon S3 using industry-standard file protocols such as Network File System (NFS). The software appliance, or gateway, is deployed into your on-premises environment as a virtual machine (VM) running on VMware ESXi. The gateway provides access to objects in S3 as files on a NFS mount point.
With file gateway, you can do the following:
You can now store and retrieve files directly using NFS 3 or 4.1 protocol.
You can access your data directly in S3 from any cloud application or service.
You can manage your data directly in Amazon S3 using lifecycle policies, cross-region replication, and versioning. We refer to this new capability as file gateway. You can think of this as an NFS mount on S3.
File gateway simplifies file storage in Amazon S3, integrates to existing applications through industry standard file system protocols and provides a cost-effective alternative to on-premises storage. It also provides low-latency access to data through transparent local caching. File gateway manages data transfer to and from AWS, buffers applications from network congestion, optimizes and streams data in parallel, and manages bandwidth consumption. File gateway integrates with the AWS platform. For example, it integrates AWS Identity and Access Management (IAM) to provide common access management, encryption using AWS Key Management Service (AWS KMS), monitoring using Amazon CloudWatch (CloudWatch), audit using AWS CloudTrail (CloudTrail), operations using the AWS Management Console and AWS Command Line Interface (AWS CLI), billing and cost management, and data management using S3 lifecycle policies and cross-region replication.
Volume Gateway – Volume gateway provides cloud-backed storage volumes that you can mount as Internet Small Computer System Interface (iSCSI) devices from your on-premises application servers. The gateway supports the following volume configurations:
Cached Volumes – You store your data in Amazon Simple Storage Service (Amazon S3) and retain a copy of frequently accessed data subsets locally. Cached volumes offer a substantial cost savings on primary storage and minimize the need to scale your storage on-premises. You also retain low-latency access to your frequently accessed data.
Stored Volumes – If you need low-latency access to your entire data set, you can configure your on-premises gateway to store all your data locally and then asynchronously back up point-in-time snapshots of this data to Amazon S3. This configuration provides durable and inexpensive off-site backups that you can recover to your local data center or Amazon EC2. For example, if you need replacement capacity for disaster recovery, you can recover the backups to Amazon EC2.
Tape Gateway – You can cost-effectively and durably archive backup data in Amazon Glacier. Tape Gateway provides a virtual tape infrastructure that scales seamlessly with your business needs and eliminates the operational burden of provisioning, scaling, and maintaining a physical tape infrastructure.
You can choose to run AWS Storage Gateway either on-premises as a virtual machine (VM) appliance, or in AWS as an EC2 instance. You deploy your gateway on an EC2 instance to provision iSCSI storage volumes in AWS. Gateways hosted on EC2 instances can be used for disaster recovery, data mirroring, and providing storage for applications hosted on Amazon EC2.
For an architectural overview, see How AWS Storage Gateway Works (Architecture).
AWS Storage Gateway enables a wide range of use cases. For more information, see the AWS Storage Gateway detail page.
Are You a First-Time AWS Storage Gateway User?
The preceding section summarizes the storage offerings provided by the AWS Storage Gateway. For a detailed architectural overview of these offerings, see How AWS Storage Gateway Works (Architecture).
This documentation provides a Getting Started section that covers setup information common to all gateways and also gateway-specific setup sections. The Getting Started section shows you how to deploy, activate and configure storage a gateway. The management section shows you how to manage your gateway and resources:
Creating a File Gateway provides instructions on how to create and use a file gateway. It shows you how to create a file share, map your drive to an Amazon S3 bucket and upload files and folders from your to Amazon S3.
Creating a Volume Gateway provides instructions on how to create and use a volume gateway. It shows you how to create storage volumes and back up data to the volumes.
Creating a Tape Gateway provides instructions on how to create and use a tape gateway. It shows you how to back up data to virtual tapes and archive the tapes.
Managing Your Gateway provides instructions on how to perform management tasks for all gateways types and resources.
The instructions in this guide primarily show the gateway operations by using the AWS Management Console. If you want to perform these operations programmatically, see the AWS Storage Gateway API Reference for information about the supported operations.
AWS Storage Gateway Pricing
For current information about pricing, see the Pricing on the AWS Storage Gateway details page.
Plan Your AWS Storage Gateway Deployment
The AWS Storage Gateway software appliance lets you connect your existing on-premises application infrastructure with scalable, cost-effective AWS cloud storage that provides data security features.
To deploy an AWS Storage Gateway solution, you first need to decide on the following two things:
Storage solution – Depending on your need, you can choose from one of the following storage solutions:
File Gateway – The primary uses of File Gateway include file ingest to S3 for use by object-based workloads, cost-effective storage for traditional backup applications, and tiering of on-premises file storage to S3. (If you want additional use cases covered, let us know.) You can cost-effectively and durably store and retrieve your on-premises objects in Amazon S3 using industry standard file protocols. File storage is a new addition to the set of interfaces on AWS Storage Gateway, alongside the current block-based volume and virtual tape library (VTL) storage.
Volume Gateway – Volume gateways let you create storage volumes in the AWS Cloud that your on-premises applications can access as Internet Small Computer System Interface (iSCSI) targets. There are two options—cached or stored volumes.
With cached volumes, you store volume data in AWS, with a small portion of recently accessed data in the cache on-premises. This approach enables low-latency access to your frequently accessed dataset and also provides seamless access to your entire dataset stored in AWS. This type of data access lets you scale your storage resource without having to provision additional hardware.
With stored volumes, you store the entire set of volume data on-premises and store periodic point-in-time backups (snapshots) in AWS. In this model, your on-premises storage is primary, delivering low-latency access to your entire dataset, and AWS storage is the backup that you can restore in the event of a disaster in your data center.
For an architectural overview of volume gateways, see Cached Volumes Architecture and Stored Volumes Architecture.
Tape Gateway – If you are looking for a cost-effective, durable, long-term, off-site alternative for data archiving, you can deploy the tape gateway solution. The virtual tape library (VTL) interface it provides lets you leverage your existing tape-based backup software infrastructure to store data on virtual tape cartridges that you create on the gateway—for more information, see Supported Third-Party Backup Applications for Tape Gateway. When you archive tapes, you don’t worry about managing tapes on your premises and arranging shipments of tapes off-site. For an architectural overview, see Tape Gateway.
Hosting option – You can choose to run AWS Storage Gateway either on-premises, as a virtual machine (VM) appliance, or in AWS, as an Amazon EC2 instance. For more information, see Requirements. If your data center goes offline and you don’t have an available host, you can deploy a gateway on an EC2 instance. AWS Storage Gateway provides an Amazon Machine Image (AMI) that contains the gateway VM image.
Additionally, as you configure a host to deploy a gateway software appliance, you will need to allocate sufficient storage for the gateway VM.
Before you continue to the next step, make sure you have done the following:
For a gateway deployed on-premises, you have decided the type of host you want to set up (VMware ESXi Hypervisor or Microsoft Hyper-V) and set it up. For more information, see Requirements. If you deploy the gateway behind a firewall, you must make sure certain ports are accessible.