Cold Storage in Azure

Microsoft Azure provides a wide range of storage solutions. But recently Microsoft Azure has declared its latest storage service which is known as Cold Storage. So, in this article, we will entirely discuss Azure’s Cold Storage service. Let us begin.

Introduction of Azure Cool Blob Storage

We’ve been working on strategies to assist you to manage the expense of storing data in the cloud because it’s rising at an exponential rate. Tiering your data depending on criteria like frequency of access, retention term, and so on is a key part of managing storage expenses.

Cool data is a type of consumer data that is accessed infrequently but has the same latency and performance requirements as hot data.

Cool Blob Storage is a low-cost storage solution for cool object data which was recently made generally available by Microsoft.

Backups, media material, scientific data, compliance, and archive data are all examples of cool storage use cases.

Any data that is rarely accessed is a good candidate for cool storage in general.

You can choose between Hot and Cool access tiers to store object data based on its access behavior using the new Blob storage accounts.

Now, let us have a look at Blob Storage Accounts in Azure.

What are Blob Storage Accounts in Azure?

The amount of data stored in the cloud is increasing at a breakneck speed. To keep expenses down as your storage needs grow, it’s a good idea to arrange your data by how often it’ll be accessed and how long it’ll be kept.

Several access tiers are available in Azure storage, allowing you to store your blob data in the most cost-effective way possible based on how it will be used. Below are the following are the Azure Storage access tiers:

Hot Tier

An online layer is designed for storing data that is regularly viewed or changed. The storage costs in the Hot tier are the greatest, but the access fees are the lowest.

Cool Tier

An online layer designed for storing data that is only viewed or updated occasionally. The data in the Cool tier should be kept or resided for at least 30 days. In comparison to the Hot tier, the Cool tier offers reduced storage costs but greater access charges.

Archive Tier

An offline tier is designed for storing data that is accessed infrequently and has variable latency requirements on the order of hours. The data in the Archive layer should be kept for at least 180 days.

Storage capacity limits in Azure are determined at the account level rather than by the access tier.

You can opt to use all of your capacity in one tier or spread it out over two or more tiers.

Azure Cool Storage Tier

First, a little history on the Azure cold storage option. General-purpose v1 (GPv1), General-purpose v2 (GPv2), and Blob storage are the three types of Azure cloud storage that can now be built.

All Azure storage choices, such as blobs, files, queues, and tables, are supported by GPv1. Except in the following cases, GPv1 is currently considered a legacy account type, and Azure encourages using GPv2 whenever possible:

• Apps that need to be deployed using the Azure old deployment approach.
• Applications that don’t need a lot of storage but are transaction-heavy or utilise a lot of geo-replication bandwidth, such as GPv2 may raise storage costs.
• Applications that employ storage services with a REST API version prior to 2014-02-14 and a client library version less than 4. x.

Azure Blob storage classes offer data tiering into hot and cold tiers. Because General-Purpose v2 storage enables blob storage in the same way as GPv1 storage does, it also supports tiered storage.

The cool access tier, also known as Azure cool Blob storage, is for data that is accessed infrequently and needs to be kept for at least 30 days. Backing up data before tiering to archival systems, legal data, media assets, system audit information, datasets utilized for big data research, and more are examples of common use cases.

This Azure cold storage tier has a cheaper storage cost than the hot storage tier. Because the data in this tier is likely to be viewed less frequently, the data access charges are higher than in the hot tier. These tiers can be accessed via APIs in the same way that Azure storage can be accessed, thus no further changes to your apps are necessary.

A single blob on both the cool and hot tiers has a pace of 60 Mib per second, or up to 500 requests per second when it comes to Azure storage performance.

The availability of the cool tier is 99 percent, which is somewhat less than the 99.9% available for the hot tier. The cold tier availability for Read-only Geo-redundant Storage (RA-GRS) is 99.9%, whereas the hot tier is 99.99 percent. These variances are primarily due to the nature of the data stored in the Azure cold storage tier, which is less affected by availability targets.

The Azure cold tier is akin to Amazon S3 Infrequent Access (S3-IA) storage in AWS, which offers low-cost, high-performance storage for data that is accessed infrequently.

Online Access Tiers in Azure

Users can view your data right away if it’s stored in an online access tier (Hot or Cool). The Hot tier is suitable for data that is regularly accessible, while the Cool tier is best for data that is accessed less frequently but still needs to be read and written on.

The following are some examples of how the Hot tier could be used:

• Data that is actively used or expected to be read and written to regularly.
• Data that has been queued for processing before being moved to the Cool access tier.

The following are some examples of how the Cool access tier could be used:

• Data backup and disaster recovery on a short-term basis.
• Older data sets that aren’t used very often yet are anticipated to be accessible right away.
• Large data collections must be held economically while more data is received for processing.

The Cool tier’s data is slightly less available, but it still has the same high durability, retrieval latency, and throughput as the Hot tiers.

In comparison to the Hot tier, significantly decreased availability and higher access prices for data in the Cool tier may be acceptable trade-offs for lower overall storage costs.

If a blob in the Cool tier in a general-purpose v2 account is removed or relocated to a different tier before 30 days have passed, it will be subject to an early deletion penalty. This fee is calculated on a prorated basis.

For example, if you shift a blob to the Cool tier and then delete it after 21 days, you’ll be charged an early deletion cost equal to 9 (30 minutes 21) days of Cool tier storage.

All redundancy configurations are supported by the Hot and Cool tiers.

Archive Access Tier in Azure

The Archive tier is a stand-alone storage system for material that is infrequently accessed. When compared to the Hot and Cool tiers, the Archive access tier has the lowest storage costs but the highest data retrieval prices and latency.

The following are some examples of how the Archive access tier could be used:

• Datasets for long-term backup, secondary backup, and archiving
• Even after it has been processed into a final usable version, original (raw) data must be retained.
• Data for compliance and archive purposes that must be kept for a long period but is rarely accessed

Data must be kept in the Archive tier for at least 180 days or it will be deleted prematurely.

For example, if you move a blob to the Archive tier and subsequently delete or move it to the Hot tier after 45 days, you’ll be charged an early deletion fee equal to 135 (180 minuses 45) days of Archive tier storage.

A blob can’t be read or updated while it’s in the Archive tier.

You must first rehydrate a blob in the Archive tier to an online tier, either Hot or Cool, before reading or downloading it.

Depending on the priority you set for the rehydration procedure, data in the Archive layer can take up to 15 hours to rehydrate.

The metadata of an archived blob is still available for reading access, allowing you to list the blob’s attributes, metadata, and index tags.

In the Archive layer, metadata for a blob is read-only, however, blob index tags can be read or written. For archived blobs, snapshots are not supported.

Blobs in the Archive layer can perform the following operations:

• Copy Blob
• Delete Blob
• Find Blobs by Tags
• Get Blob Metadata
• Get Blob Properties
• List Blobs
• Get Blob Tags
• Set Blob Tags
• Set Blob Tier

Blobs can only be moved to the Archive tier if the storage account is configured for LRS, GRS, or RA-GRS. For ZRS, GZRS, and RA-GZRS accounts, the Archive tier is not available.

You must first rehydrate all archived blobs to the Hot or Cool tier before changing the redundancy settings for a storage account that contains blobs in the Archive tier.

Because rehydration processes can be costly and time-consuming, Microsoft recommends that you avoid modifying the redundancy configuration for a storage account that contains archived blobs if at all feasible. It is possible to migrate a storage account from LRS to GRS if no blobs were migrated to the Archive tier while the account was configured for LRS.

If the update is performed within 30 days of when the account was switched to LRS, and no blobs were moved to the Archive tier while the account was set to LRS, the account can be moved back to GRS.

Default Account Access Tier Setting in Azure

The default access tier setting for storage accounts specifies the online tier in which a new blog is created.

You can choose between Hot and Cool as the default access tier. When uploading or altering the tier of an individual blob, users can override the default option.

A new general-purpose v2 storage account’s default access tier is set to the Hot tier by default.

When you create a storage account or after it has been created, you can adjust the default access tier level.

By default, a new blob is posted to the Hot tier until you modify this setting on the storage account or explicitly set the tier while uploading a blob.

The default account access tier setting infers the tier of a blob that doesn’t have one explicitly given.

The Azure portal displays the access tier as Hot (inferred) or Cool (inferred) if a blob’s access tier is inferred from the default account access tier setting (inferred).

Changing a storage account’s default access tier affects any blobs in the account for which an access tier has not been expressly established.

If you switch a general-purpose v2 account’s default access tier from Hot to Cool, you’ll be paid for write operations (per 10,000) for all blobs for which the access tier is inferred.

If you switch from Cool to Hot in a general-purpose v2 account, you’ll be charged for both read operations (per 10,000) and data retrieval (per GB).

When you create a legacy Blob Storage account, you must choose between Hot and Cool as the default access tier.

In a legacy Blob Storage account, changing the default account access tier option from Hot to Cool is free.

When you switch a Blob Storage account from Cool to Hot, you’ll be charged for both read operations (per 10,000) and data retrieval (per GB).

When possible, Microsoft advises utilizing general-purpose v2 storage accounts instead of Blob Storage accounts.

Azure Cool Access Tier vs. Archive Access Tier

The cool access tier is for data that you want to keep for a short period of time but won’t be able to access on a daily basis, such as banking transaction history, e-commerce transaction history, and so on.

If you need to retrieve data, you’ll need it to be available online. The speed with which the data can be accessed if a need arises is nearly identical to that of the hot storage layer.

Short-term data backup is a popular use case for the cool storage layer, which can later be moved to the hot tier for restoration or to the archive tier for long-term storage. It can also be used to store analytical data, huge media files, telemetry, and other types of information.

The nature of the data typically necessitates a high-capacity, low-cost storage solution. In such instances, the cool tier is ideal.

The archive layer is for data that can be kept offline for a long time. There will be no immediate or pressing need to access the information.

If a situation arises when the data must be retrieved, a 15-hour wait period should be acceptable. Medical records from an archive could be retrieved for research purposes, for example.

The archive tier, unlike the cool tier, is not activated at the Azure Storage level. It does, however, offer blob-level tiering, which allows you to set the archiving tier at the object level.

The Archive tier became generally accessible in December 2017, but it is not yet available in all Azure regions.

Long-term data preservation for compliance is a prominent use case for the archive tier.

Other scenarios where the archive tier can be used include low-cost storage for data sets such as patient records in hospitals, remote Azure blob backup, and raw media files.

When data access patterns are unclear, it’s best to start with the hot storage tier, monitor for a while, and then shift the data to the cold storage or archive storage layer based on the frequency of access.

Cost-Benefit Analysis between Azure Cool Access Tier and Archive Access Tier

The Archive tier has the lowest Azure storage charges on the Azure cloud, with rates as low as $0.00099 to $0.002 per GB up to the first 50 TBs depending on the region. Reading data from an archive tier, on the other hand, is a costly action that costs $5 for every 10,000 read operations.

Beginning March 1, 2018, any data erased before the required time of 180 days for the archive tier will be subject to an early deletion charge. So, if your data was retained for less than 180 days, the charge would be for the days that were not used. The charge would be for 50 days of storage if the data was held for 130 days.

Cool tier data storage is more expensive than archive tier, costing $0.01 GB up to 50 TB. Data read activities, on the other hand, are not subject to large fees. In the cool tier, 10,000 operations will cost only $0.01.

As the accompanying prices show, choosing the best low-cost storage is primarily determined by the quantity of the data and the usage pattern.

The Archive tier can store terabytes of data for relatively little money.

If a need develops for the data to be read more frequently, it is always more cost-effective to rehydrate and store it in the cold or hot tier.

Benefits of Azure Cold Storage

1. Cost-Effective

Your less often accessible data can now be stored in the Cool access tier for a low storage cost (as low as $0.01 per GB in some locations), while your more frequently accessed data can now be stored in the Hot access tier for a lower access cost. See Azure Storage Pricing for more information on regional pricing.

2. Compatibility

Blob storage accounts have been designed to be 100 percent API compatible with our existing Blob storage service, allowing you to simply integrate the new storage accounts into your existing applications.

3. Performance

In terms of latency and throughput, data in both access levels has a comparable performance profile.

4. Availability

The Hot access tier provides 99.9% uptime, while the Cool access tier guarantees a slightly lower 99.9% availability.

Microsoft provides a higher read SLA of 99.99 per cent for the Hot access tier and 99.9% for the Cool access tier with the RA-GRS redundancy option.

5. Durability

Both access tiers offer the same high levels of durability and data replication options that you’ve come to expect from Azure Storage.

6. Scalability and Security

Blob storage accounts offer the same scalability and security as our current service.

7. Global Reach

Blob storage accounts are available in most Azure areas starting today, with more regions coming soon. The Azure Services by Regions page has an updated list of available regions.

Apart from using the Azure Interface or PowerShell to manage Blob storage accounts, we can also utilize the Microsoft Azure Storage Explorer, which is a GUI-based portal that is very straightforward to use.

Some third-party organizations have provided updates on their support for Cool storage:

1. Commvault

We can utilize this Commvault software product for enterprise data management, which is a Windows/Azure-centric solution.

With this new Cool Storage product, Microsoft increases their Azure offering once again, and we expect this service to be a key driver of new opportunities for both Commvault and Microsoft.

2. Veritas

Veritas NetBackupTM, the market leader in terms of administration and performance, protects enterprise data on a global scale for any workload, on any storage device, and in any location.

3. SoftNAS

Soon, SoftNASTM® will support Azure Cool storage. By the summer of 2016, customers will be able to employ SoftNAS Cloud NAS with Azure Cool storage as a cost-effective solution to growing storage costs.

SoftNAS allows customers to transfer to the cloud without changing their apps while simultaneously providing enterprise-class NAS features such as de-duplication, compression, directory integration, encryption, snapshotting, and more.

4. Cohesity

Cohesity is the world’s first hyper-converged storage system for business data. It unites disparate, inefficient secondary storage islands into an infinitely expandable storage platform that may be used on-premises or in the cloud.

The Cohesity platform can quickly and effectively interface with Azure cool storages for long-term data preservation and archival, tiering of infrequently accessed data onto the cloud, and disaster recovery replication.

5. CloudBerry Lab

It’s a backup tool for Microsoft’s Azure service. It compresses and encrypts data before it leaves the computer, using a password given by the user. The data is then securely sent to the cloud, either on a schedule or in real-time.

CloudBerry Backup also features file-system and image-based backup, SQL Server and MS Exchange compatibility, and adjustable retention settings and incremental backup.

According to the latest information, CloudBerry Backup is now available for storing backup data, and it now supports Azure Blob storage accounts as well.

Use Scenarios

The following are some examples of how the Cool access tier could be used:

• Data backup and disaster recovery on a short-term basis.
• Older data sets that aren’t used very often yet are anticipated to be accessible right away.
• Large data collections must be held economically while more data is received for processing.

Conclusion

Many businesses are turning to cloud storage or hybrid cloud storage to address the problem of exponential data expansion and backup data storage.

When compared to on-premises data storage, cloud storage is a more cost-effective choice. Utilizing tiered low-cost Azure storage choices for least-accessed data will help you save even more money.