Azure Queue – Services Bus and Storage Bus

In our previous articles, we have learnt about multiple storage services and compute modules offered by Microsoft Azure. And, now in today’s article, we will learn about one more popular service of Azure which is known as Queue.

What is Azure Queue?

In technical words, the queue is a set of entities that are available in computers and data structures. And, this set is arranged and managed in a sequence and the users are authorized to modify these sequences by adding multiple entities at the ends or by removing the entities from the other end.

The sequence where the elements can be added is known as back, trail or rear in the queue. Simultaneously, if head or front in the queue is known as the point from where the users can remove the elements.

Azure Queue in Simple Words

For a technical person basically, the queue is a data structure that is used to store data in a sequential manner that follows the FIFO rule which means (First-In-First-Out). Here, the data will be added in the tail and it gets deleted from the front.

Along with that, the data will be added to the tail while it gets deleted from the front. Enqueue is known as adding the data to the rear and deque is known as the process of removing the data from the front.

Microsoft Azure Queue

In Microsoft, Azure queues are a similar concept that is used to store messages in the queue. The following process followed inside Microsoft Azure’s messaging queue service. Now let us look at the working mechanism of the Azure queue.

Step 1: In the initial step the sender will send a message which is delivered to the client and then the further process takes place. The message might consist of attributes in it.

Step 2: In the next step the client will process the message and once it is processed the user deletes the message. Azure will store the message for seven days and then will automatically delete it further if it is not done by the user.
In some situations, there is only a sender and one client or sometimes there might be one sender and multiple clients.

One of the major advantages of the Azure queue service is the decoupling segments. It works in an offbeat climate where the messages can be delivered to various sections of the application. As a result, the queue offers a productive response to the users for an overload work process.

Architecture of Microsoft Azure Queue Storage

Microsoft Azure’s Queue Storage service is mainly used by the users to manage asynchronous tasks and for building the workflow process. Below we have mentioned the architectural diagram of the Microsoft Azure Queue Storage.

1. URL Format

In Microsoft’s Azure account the user can create the queues by using a unique namespace and addressable with the help of URL format.

2. Storage Account

The users mainly use it to offer access and manage all the tasks related to the storage account and it is considered as the building block of Azure services.

For transferring or migrating the incoming messages from one application to another in the storage the users should have a storage account as it provides a unique namespace. It will hold all the data objects that are used in Microsoft Azure such as

• Blob Storage
• File Storage
• Queues Storage
• Disks

4. Azure Table Storage

For accessing the Azure Table Storage service, the users must create a storage account.

5. Queue

It is the set of messages in a queue but the queue name should be in lowercase and unique with a valid DNS name having characters ranging from 3 to 63.

6. Messages

In Azure, a single message can be up to 64kb in size and it also can be in any format.

How to Create Azure Storage Account?

The most convenient method to create Azure File Storage Account is with the help of Azure Portal. The administrators can also make an Azure Storage Account with the help of the following:

• Azure PowerShell
• Azure CLI
• Azure Storage Resource Provider for .NET

If the administrators do not want to create a Storage Account then they can also use the Azurite Storage Emulator to deploy and test the code in their local system.

How to set up the Development Environment

In Visual Studio the developers can create a new Windows console application. Below, we have mentioned the steps to explain how the developers can create a console application in Visual Studio 2019. The steps are as follows:

1: Firstly, the developer should select the File option

2: In the second step they must click on the New button

3: Click on the project option

4: Select the Platform option

5: Choose the Windows option

6: Select the Console App

7: Next, the Project Name field enter the Application Name

8: Lastly, click on the Create button.

How to get the Storage Connection String

Azure Storage Client libraries for .NET supports the storage connection string for configuring the endpoints and credentials for accessing the storage services.

Peek at the Next Message

.NET v12 SDK

The developers can peek or have a look at the messages in the queue without removing or deleting them from the queue with the help of the PeekMessages method. If the developer doesn’t pass the value for the maxMessages parameter the default is to peek at one message.

Below we have mentioned the sample C# code

//-------------------------------------------------
// Peek at a message in the queue
//-------------------------------------------------
public void PeekMessage(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
// Peek at the next message
PeekedMessage[] peekedMessage = queueClient.PeekMessages();
// Display the message
Console.WriteLine($"Peeked message: '{peekedMessage[0].Body}'");
}
}

.NET v11 SDK

The developer can peek at the message in front of the queue without deleting it from the queue with the help of the PeekMessage method.

Below we have mentioned the sample C# code

// Retrieve storage account from connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the queue client
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Retrieve a reference to a queue
CloudQueue queue = queueClient.GetQueueReference("myqueue");
// Peek at the next message
CloudQueueMessage peekedMessage = queue.PeekMessage();
// Display message.
Console.WriteLine(peekedMessage.AsString);

Changing the Contents of the Queued Message

The developers can edit the message contents in the queue. If the message is represented in a work task, then they can use this feature to update the status of the work task.

Below, we have mentioned a code that updates the queue with new contents and will set the visibility timeout for extending another sixty seconds.

This will help the developers and will minimize their state of work connected with the message and will provide the client another minute for continuing working on the message.

The developers can use this technique to track the multistep workflows on the queue messages and this will avoid the task of beginning the task from all over. Basically, it will retry the count as well and if the message is retried more than n times the developers have to delete it.

Thus, it will protect against a message that triggers the application error every time they process.
Below we have mentioned the C# code

.NET v12 SDK

//-------------------------------------------------
// Update an existing message in the queue
//-------------------------------------------------
public void UpdateMessage(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
// Get the message from the queue
QueueMessage[] message = queueClient.ReceiveMessages();
// Update the message contents
queueClient.UpdateMessage(message[0].MessageId,
message[0].PopReceipt,
"Updated contents",
TimeSpan.FromSeconds(60.0) // Make it invisible for another 60 seconds
);
}
}

.NET v11 SDK

// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the queue client.
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Retrieve a reference to a queue.
CloudQueue queue = queueClient.GetQueueReference("myqueue");
// Get the message from the queue and update the message contents.
CloudQueueMessage message = queue.GetMessage();
message.SetMessageContent2("Updated contents.", false);
queue.UpdateMessage(message,
TimeSpan.FromSeconds(60.0), // Make it invisible for another 60 seconds.
MessageUpdateFields.Content | MessageUpdateFields.Visibility);

Dequeue the Next Message in Azure Queue

.NET v12 SDK

Dequeuing the message from the queue can be done in two steps. When the developer calls the ReceiveMessages the developer will get the next message in a queue.

A message will be returned from the ReveiveMessages and will become invisible to any other code for reading the messages from this queue. By default, the following message will be invisible for thirty seconds.

For finishing the remove message from the queue the developers should also call the DeleteMessage.

Thus, these two-step processes for removing the message will assure that if their code fails a message because of hardware or software failure, another instance of the code will also get the same message and retry.

The developer’s code will call DeleteMessage right after the message will be processed.

C# Code

//-------------------------------------------------
// Process and remove a message from the queue
//-------------------------------------------------
public void DequeueMessage(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
// Get the next message
QueueMessage[] retrievedMessage = queueClient.ReceiveMessages();

// Process (i.e. print) the message in less than 30 seconds
Console.WriteLine($"Dequeued message: '{retrievedMessage[0].Body}'");
// Delete the message
queueClient.DeleteMessage(retrievedMessage[0].MessageId, retrievedMessage[0].PopReceipt);
}
}

NET v11 SDK

The developer’s code will dequeue the message from a queue in two steps. When the developer calls for GetMessage they will get the next message in a queue.

A message returned from GetMessage becomes invisible to any other code for reading the messages from these queues. By default the following message will be invisible for thirty seconds.

Simultaneously, for removing the message from the queue the developer must also call for the DeleteMessage service.

Thus, these two processes of removing the message will make sure that if their code fails to process a message because of hardware or software failure, then another instance of the code will get the same message and retry it.

The code will call for DeleteMessage right after the processing of the message.

C# Code

// Retrieve storage account from connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Create the queue client
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Retrieve a reference to a queue
CloudQueue queue = queueClient.GetQueueReference("myqueue");
// Get the next message
CloudQueueMessage retrievedMessage = queue.GetMessage();
//Process the message in less than 30 seconds, and then delete the message
queue.DeleteMessage(retrievedMessage);

Using Async-await Patterns with Common Queue Storage

In the following sample, it will call the asynchronous version of each of the provided methods as highlighted by the Async suffix of every method.

When an async method is utilized the Async-Await pattern will suspend the local execution until the call is successfully completed.

The behavior will permit the current thread to perform other tasks that will help to lower the performance issues and will improve the overall actions and responsiveness of the application.

.NET v12 SDK

Below we have mentioned the C# code

//-------------------------------------------------
// Perform queue operations asynchronously
//-------------------------------------------------
public async Task QueueAsync(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];

// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);

// Create the queue if it doesn't already exist
await queueClient.CreateIfNotExistsAsync();

if (await queueClient.ExistsAsync())
{
Console.WriteLine($"Queue '{queueClient.Name}' created");
}
else
{
Console.WriteLine($"Queue '{queueClient.Name}' exists");
}

// Async enqueue the message
await queueClient.SendMessageAsync("Hello, World");
Console.WriteLine($"Message added");

// Async receive the message
QueueMessage[] retrievedMessage = await queueClient.ReceiveMessagesAsync();
Console.WriteLine($"Retrieved message with content '{retrievedMessage[0].Body}'");

// Async delete the message
await queueClient.DeleteMessageAsync(retrievedMessage[0].MessageId, retrievedMessage[0].PopReceipt);
Console.WriteLine($"Deleted message: '{retrievedMessage[0].Body}'");

// Async delete the queue
await queueClient.DeleteAsync();
Console.WriteLine($"Deleted queue: '{queueClient.Name}'");
}

.NET v11 SDK

Below we have mentioned the C# code

// Create the queue if it doesn't already exist
if(await queue.CreateIfNotExistsAsync())
{
Console.WriteLine("Queue '{0}' Created", queue.Name);
}
else
{
Console.WriteLine("Queue '{0}' Exists", queue.Name);
}
// Create a message to put in the queue
CloudQueueMessage cloudQueueMessage = new CloudQueueMessage("My message");
// Async enqueue the message
await queue.AddMessageAsync(cloudQueueMessage);
Console.WriteLine("Message added");
// Async dequeue the message
CloudQueueMessage retrievedMessage = await queue.GetMessageAsync();
Console.WriteLine("Retrieved message with content '{0}'", retrievedMessage.AsString);
// Async delete the message
await queue.DeleteMessageAsync(retrievedMessage);
Console.WriteLine("Deleted message");

Using Additional Options for Dequeuing Messages in Azure Queue

Here, there are two possible techniques to customize the message retrieval from a queue. Firstly, the developer can get the batch of messages.

Secondly, the developer can set a longer or smaller invisibility timeout which allows their code more or less time to completely process every message.

.NET v12 SDK

The following code will use the ReceiveMessages method to get twenty messages in one call. Further, it will process every message with the help of a foreach loop.

It will also set the invisibility timeout rate to five minutes for every message. But note the five minutes starts for all the messages at the same moment so once the five minutes are surpassed as the call to ReceiveMessages any messages that have not been deleted will become visible once again.

Below we have mentioned the C# Code

//-----------------------------------------------------
// Process and remove multiple messages from the queue
//-----------------------------------------------------
public void DequeueMessages(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];

// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);

if (queueClient.Exists())
{
// Receive and process 20 messages
QueueMessage[] receivedMessages = queueClient.ReceiveMessages(20, TimeSpan.FromMinutes(5));

foreach (QueueMessage message in receivedMessages)
{
// Process (i.e. print) the messages in less than 5 minutes
Console.WriteLine($"De-queued message: '{message.Body}'");

// Delete the message
queueClient.DeleteMessage(message.MessageId, message.PopReceipt);
}
}
}

.NET v11 SDK

The following code will use the GetMessages method to get twenty messages in one call. Further, it will process every message with the help of a foreach loop.

It will also set the invisibility timeout rate to five minutes for every message.

But note the five minutes starts for all the messages at the same moment so once the five minutes are surpassed as the call to GetMessages any messages that have not been deleted will become visible once again.

Below we have mentioned the C# Code

// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the queue client.
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Retrieve a reference to a queue.
CloudQueue queue = queueClient.GetQueueReference("myqueue");
foreach (CloudQueueMessage message in queue.GetMessages(20, TimeSpan.FromMinutes(5)))
{
// Process all messages in less than 5 minutes, deleting each message after processing.
queue.DeleteMessage(message);
}
Getting the Queue Length

.NET v12 SDK

The developer can get an estimated count of messages in a queue. The GetProperties method will return back the queue properties which includes the message count.

The ApproximateMessageCount property will have all the approximate number of messages in the queue.

The following number will not be lower than the actual number of messages in the queue but it can be more.

Below we have mentioned the C# Code

//-----------------------------------------------------
// Get the approximate number of messages in the queue
//-----------------------------------------------------
public void GetQueueLength(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];

// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);

if (queueClient.Exists())
{
QueueProperties properties = queueClient.GetProperties();

// Retrieve the cached approximate message count.
int cachedMessagesCount = properties.ApproximateMessagesCount;

// Display the number of messages.
Console.WriteLine($"Number of messages in queue: {cachedMessagesCount}");
}
}

.NET v11 SDK

The developer can get an estimated count of messages in a queue. The FetchAttributes method will return back the queue properties which includes the message count.

The ApproximateMessageCount property will have to return back the last value which is retrieved by the FetchAttributes method without calling the Queue Storage.

Below we have mentioned the C# Code

// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the queue client.
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Retrieve a reference to a queue.
CloudQueue queue = queueClient.GetQueueReference("myqueue");
// Fetch the queue attributes.
queue.FetchAttributes();
// Retrieve the cached approximate message count.
int cachedMessageCount = queue.ApproximateMessageCount;
// Display the number of messages.
Console.WriteLine("Number of messages in queue: " + cachedMessageCount);
Deleting the Queue

.NET v12 SDK

For deleting a queue and all the messages the developers can call the Delete method on the queue object.
Below we have mentioned the C# code

//-------------------------------------------------
// Delete the queue
//-------------------------------------------------
public void DeleteQueue(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];

// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);

if (queueClient.Exists())
{
// Delete the queue
queueClient.Delete();
}

Console.WriteLine($"Queue deleted: '{queueClient.Name}'");
}

.NET v11 SDK

For deleting a queue and all the messages the developers can call the Delete method on the queue object.
Below we have mentioned the C# code

// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Create the queue client.
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();

// Retrieve a reference to a queue.
CloudQueue queue = queueClient.GetQueueReference("myqueue");

// Delete the queue.
queue.Delete();

Comparison Between Azure Storage Bus vs Service Bus Queues

Steps to Create Queue in Microsoft Azure Portal

Before making a queue, the user should already have a resource group and a storage account. Below are the steps to create Queue in Azure portal:

1: The user should firstly log in to their Azure Portal

2: In the second step, the user should click on the storage account and open it.

3: Slowly the user should scroll down and then go to the Queue services

4: Now, click on the “+Queue” option to make a new queue

5: Once you click on the “+Queue” button, a new window comes on the screen for “Add Queue”. Here the user must insert the queue name in the box and then click on the Ok button.

Finally, the queue is ready and the user can see their queue in the Microsoft Azure Storage Explorer.

Steps to Add Message in Azure Queue

Follow the below steps to add message in Azure Queues:

1: After creating a queue the user should select it located in the Queue Page.

2: Secondly, click on the Add Message From option

3: Now, the user should type a message in the text field along with that they can set the time limitation in the expire section.

4: Finally, the user must click on the OK button for adding a new message in the queue.

How to manage the Queue using PowerShell

Follow the below steps to manage Azure queue using PowerShell:

1: Firstly, the developer should Right-Click on the PowerShell option available in the taskbar and then select the Run ISE as Administrator option.

2: Now the developer should run the following command for accessing the account.

$context = New-AzureStorageContext -StorageAccountName AccountName StorageAccountKey
iUZNeeJD+ChFHt9XHL6D5rkKFWjzyW4FhV0iLyvweDi+Xtzfy76juPzJ+mWtDmbqCWjsu/nr+1pqBJj rdOO2+A==

Note: Remember to replace the “AccountName” with your Account Name.

3: Now the developer should mention the storage account in which they want to create a queue

Set-AzureSubscription –SubscriptionName "BizSpark" -CurrentStorageAccount AccountName

4: Lastly, create a queue

$QueueName = "thisisaqueue"
$Queue = New-AzureStorageQueue –Name $QueueName -Context $Ctx

Dequeue Next Message from Queue

Follow the below steps to deque message from queue:

1: In the first step the developer should connect their account and specify the storage account by running the following commands

2: In the second step the developer should retrieve the queue by running the following command.

$QueueName = "myqueue"
$Queue = Get-AzureStorageQueue -Name $QueueName -Context $ctx
$InvisibleTimeout = [System.TimeSpan]::FromSeconds(10)

3: In the next step the developer should deque the next message by running the following command.

$QueueMessage = $Queue.CloudQueue.GetMessage($InvisibleTimeout)

4: In the last step the developer should run the following code to delete the dequeued message, by running the following command.

$Queue.CloudQueue.DeleteMessage($QueueMessage)

Uses of Microsoft Azure Queues

• Users primarily use Microsoft Azure’s Queue service for the assured delivery system messages that take place between the applications.
• Secondly, Azure’s Storage Queue allows storing the users to store the messages until the system is available to process further queues.
• Microsoft Azure Storage will also permit the user’s application components to communicate with each other in the cloud premises, in the desktops, on-premises, or the mobile devices with the help of decoupling system segments.
• Microsoft’s Azure Storage offers a wide range of client libraries to the users which helps them to interact with the Azure Storage Queues which offers support when there is a failure in the application component.

Features of Microsoft Azure Queues

1. Decoupling the Components

The developers can make use of the Azure Queue Storage service to develop flexible applications and individual functions for better durability across massive workloads.

When the developers design the application for scale then the application’s components can be decoupled and they can scale up individually.

Queue storage offers asynchronous message queueing for better communication between the application components.

2. Build-in Resilience

Azure Queue Storage helps the developers to make their applications scalable and less sensitive towards every component failure.

If any component of the architecture fails, the messages will be buffered and further, they will be naturally raised by other message processing nodes which will take the responsibility to maintain the integrity of the workload.

3. Scale for Bursts

Developers can utilize Azure Queue Storage for right-sizing their service deployment. Applications will take unexpected traffic bursts which will prevent the servers from being overwhelmed with the sudden rise of requests.

The developers can monitor the length of the queue for adding the elasticity of the application and then hibernate or deploy the additional worker nodes depending upon the requirement.

Limitations of Microsoft Azure Queue

Microsoft’s Azure Queue storage service is useful to store a massive number of messages. It permits the users to access the messages globally with the help of authenticated calls by using HTTP and HTTPS.

Also, the maximum size of a single message can be only 64kb. But a queue can have millions of messages and the maximum capacity of it is dependent upon the storage account.

Conclusion

Now, we are into the last section of our article and today we learnt about Microsoft’s Azure Queue Storage which is popular among users to store a wide range of messages. It also helps them to easily transfer the messages between multiple applications.

Along with that Microsoft takes the responsibility to manage the data in the queue and the users only have to pay for the resources they use.