Storage Migration
Last reviewed: August 2026
Overview
Section titled “Overview”Migration refers to moving large volumes of data from on-premises or another cloud into the cloud. It might seem as simple as running aws s3 cp, but at a scale of several TB to several PB, network bandwidth, cost, and time all become major constraints.
Migration Types
Section titled “Migration Types”| Type | Description | Target |
|---|---|---|
| Online transfer | Transfers data over the network | A few GB to a few TB, fast network environments |
| Offline transfer | Ships data on a physical device | Tens of TB to several PB, limited network environments |
| Hybrid replication | Initial offline transfer followed by ongoing online increments | Large data volumes requiring continuous updates |
| File gateway | On-premises cache + cloud storage backend | Gradual transition, tolerant of access latency |
Choosing a Transfer Method
Section titled “Choosing a Transfer Method”Transfer time is determined by data size and network speed.
| Data size | 1Gbps network | 10Gbps network | Recommended method |
|---|---|---|---|
| 100GB | ~15 minutes | ~2 minutes | Online (standard transfer) |
| 1TB | ~2.5 hours | ~15 minutes | Online (DataSync, Storage Transfer Service) |
| 10TB | ~25 hours | ~2.5 hours | Online + dedicated connection (Direct Connect, ExpressRoute) |
| 100TB | ~10 days | ~25 hours | Offline (Snowball, Data Box) |
| 1PB | ~100 days | ~10 days | Offline (Data Box Heavy, Transfer Appliance) |
The figures above are illustrative and vary by region/time. Check each vendor’s official pricing page for current rates.
Cost Considerations
Section titled “Cost Considerations”For storage migration, the choice of transfer method has a major impact on cost.
| Item | Online transfer | Offline transfer |
|---|---|---|
| Egress cost (on export) | $0.05–$0.126 per GB | Fixed device fee |
| Ingestion cost | Free (mostly) | Free |
| Storage cost | Same | Same |
| Device rental fee | None | $50–$15,000 (varies by size) |
| Transfer time | Depends on network speed | Fixed at 2–3 weeks |
| Labor cost | Can be automated | Requires handling device pickup/return |
The figures above are illustrative and vary by region/time. Check each vendor’s official pricing page for current rates.
Online Transfer Services
Section titled “Online Transfer Services”Large File Transfer
Section titled “Large File Transfer”| Vendor | Product | Notes |
|---|---|---|
| AWS | DataSync | Online transfer between NFS/SMB/S3. Supports incremental transfer |
| AWS | Storage Gateway | On-premises cache + S3 backend |
| Azure | AzCopy | CLI-based high-performance Blob transfer |
| Azure | Azure File Sync | Synchronizes an on-premises Windows file server with Azure Files |
| Google Cloud | Storage Transfer Service | Transfers from S3/Azure Blob/HTTP into Cloud Storage |
| Google Cloud | gsutil / gcloud storage | CLI-based parallel transfer |
| OCI | OCI Data Transfer | Supports both CLI and offline methods |
| OCI | rclone / OCI CLI | General-purpose sync tool |
Object Storage-to-Object Storage Replication
Section titled “Object Storage-to-Object Storage Replication”Continuously replicates from one object storage to another. Used for multi-cloud environments or DR purposes.
| Vendor | Product | Notes |
|---|---|---|
| AWS | S3 Cross-Region Replication (CRR) | Supports both same-account and cross-account |
| AWS | S3 Replication Time Control (RTC) | SLA of 99.99% replicated within 15 minutes |
| Azure | Blob Object Replication | Asynchronous replication across accounts |
| Google Cloud | Cloud Storage Cross-Region / Multi-Region | Automatic replication when selecting the storage class |
| OCI | Object Storage Replication | Replication across regions/namespaces |
Offline Transfer Services
Section titled “Offline Transfer Services”Used when network transfer is impractical due to data size, or when the network environment is limited.
| Vendor | Product | Capacity | Feature |
|---|---|---|---|
| AWS | Snowball Edge | ~80 TB | Not available to new customers as of Nov 7, 2025 (existing customers only). New customers should use Data Transfer Terminal or DataSync |
| AWS | Snowcone / Snowmobile | — | Discontinued (Snowcone HDD/SSD retired Nov 2024; Snowmobile service ended) |
| Azure | Data Box Disk | ~35 TB | SSD-based, small capacity |
| Azure | Data Box | ~100 TB | Standard device |
| Azure | Data Box Heavy | ~1 PB | Large-capacity device |
| Google Cloud | Transfer Appliance | TA40: ~40 TB, TA300: ~300 TB | Standard/large-capacity |
| OCI | Data Transfer Appliance | ~150 TB | Transfer via a rented device |
| OCI | Data Transfer Disk | ~32 TB | Customer purchases the disk and ships it |
The figures above are illustrative and vary by region/time. Check each vendor’s official pricing page for current rates.
Offline Transfer Steps
Section titled “Offline Transfer Steps”- Order the device through the vendor’s console/API
- The vendor ships the device
- Copy data onto the device on-premises
- Ship the device back to the vendor
- The vendor uploads the data from its data center to cloud storage
- After verification, the data on the device is securely erased
The process typically takes 2–3 weeks and can significantly reduce network egress costs.
Hybrid Cache/Gateway
Section titled “Hybrid Cache/Gateway”An approach that lets on-premises applications use cloud storage without any changes.
| Vendor | Product | Delivery form |
|---|---|---|
| AWS | Storage Gateway | File/Volume/Tape Gateway |
| Azure | Azure File Sync | File server extension |
| Google Cloud | Cloud Storage FUSE | Mounted like a file system |
| OCI | Storage Gateway | NFS v4 interface |
Verification and Integrity
Section titled “Verification and Integrity”Integrity verification is essential after large-scale data transfer.
- Checksum verification — Compare the MD5/SHA256 hash of each file
- File count/size comparison — Confirm metadata matches between source and destination
- Sampling test — Download random files to confirm they can actually be opened
- Permission preservation check — Verify owner, read/write permissions, and ACLs are preserved
Vendor tools mostly perform automatic verification, but it’s safer to also perform manual verification for critical data.
Common Mistakes
Section titled “Common Mistakes”- Underestimating network transfer time — Calculating based on theoretical bandwidth, when actual transfers take 1.5–2x as long. Retries, verification, and network efficiency must be factored in
- Skipping integrity verification after transfer — Confirming only file counts without checksum comparison, discovering corrupted files too late
- Not estimating egress cost in advance — For tens of TB of online transfer, egress cost can exceed the rental fee for an offline device
Checklist
Section titled “Checklist”- Have you chosen an online/offline transfer method based on data size and network speed?
- Do you perform checksum (MD5/SHA256)-based integrity verification after the transfer completes?
- Have you compared egress cost against device rental fee to choose the most cost-effective method?
References
Section titled “References”- AWS Cloud Data Migration
- AWS DataSync documentation
- AWS Snow Family documentation
- AWS Storage Gateway documentation
- Azure Storage migration overview
- AzCopy documentation
- Azure Data Box documentation
- Azure File Sync documentation