Implementing Azure File Share for VM Scale Set (Auto-Scaling)

Abstract:

At Pirai we had a client request to implement auto scaling for a project that relied on standalone Virtual Machines (VMs) to store files locally, in order to accommodate the increasing traffic. To achieve this, Virtual Machine Scale Set was chosen which runs multiple VMs in the background to handle requests. However, a new challenge arose. With multiple VMs running, file storage became a problem. Storing files separately on different VMs was no longer a viable option. Therefore, the team had to find a common storage space for all VMs to access.

Problem Statement:

Downtime - If your application experiences sudden spikes in traffic or usage, it may become overwhelmed and crash. This can lead to downtime, which can be costly and damaging to your reputation.

Poor performance - Without auto-scaling, your application may not have the resources it needs to perform optimally during peak usage periods. This can lead to slow load times, timeouts, and other performance issues.

Overprovisioning - To ensure that your application can handle peak usage, you may need to overprovision resources, which can lead to unnecessary costs. This can be especially problematic if your application experiences periods of low usage, as you'll be paying for unused resources.

Manual management - Without auto-scaling, you'll need to manually manage the scaling of your infrastructure. This can be time-consuming and can distract your team from other important tasks.

Overall, without auto-scaling, your application may struggle to handle fluctuating demand, leading to downtime, poor performance, and unnecessary costs. Auto-scaling can help you avoid these issues by automatically adjusting resources based on demand, ensuring optimal performance and cost-effectiveness.

Benefits of Auto-scaling for virtual machines (VMs):

Increased availability - Auto-scaling ensures that your application can handle fluctuations in traffic without experiencing downtime or reduced performance. By automatically adding or removing VM instances, your application can scale up or down seamlessly, based on demand.

Improved performance - With auto-scaling, you can ensure that your application always has the necessary resources to perform at its best. By scaling up resources during peak usage periods, you can avoid bottlenecks and ensure consistent performance.

Reduced costs - Auto-scaling allows you to provision resources based on demand, rather than over-provisioning and incurring unnecessary costs. This can help you optimize your infrastructure costs and avoid paying for unused resources.

Increased agility - Auto-scaling enables you to quickly respond to changing business requirements and scale your application up or down as needed. This can help you stay competitive and respond to market trends.

Simplified management - With auto-scaling, you don't need to manually manage the scaling of your infrastructure. Instead, the system automatically adjusts resources based on demand, freeing up your team to focus on other tasks.

Challenges:

The main challenge was choosing the right storage solution for the client's setup. There were two possible options: Azure File Share and Blob Storage.

Solution:

Azure File Share is a cloud-based file share solution that provides a single, centralized location for storing files. The solution is available on Azure and can be accessed from any VM that is part of the Virtual Machine Scale Set. Azure File Share was selected because it offered a number of key advantages:

Shared Storage - All VMs in the Virtual Machine Scale Set could access the files stored in the same location. This allowed the client to store files centrally, making it easier to manage and retrieve files, and eliminated the need to copy files across multiple VMs.

Easy Implementation - Azure File Share is an easy-to-use storage solution that can be quickly deployed, making it ideal for the client's needs. It is simple to integrate with existing Azure services and can be accessed via standard SMB (Server Message Block) protocols.

Cost-Effective - Azure File Share is a cost-effective solution, as it eliminates the need for additional storage hardware, reduces the number of VMs required, and provides a single storage location for all files.

Architecture Diagram:

Conclusion:

In summary, implementing auto scaling with Azure File Share was a successful solution for the client's project. The team was able to choose the right storage solution based on the client's setup and requirements. Azure File Share was the best option because it provided shared storage, was easy to implement, and cost-effective. By using Azure File Share, the client was able to store files centrally, access them from any VM in the Virtual Machine Scale Set, and reduce the overall cost of storage.