Considering moving your SQL Server data platform to Azure?

Cloud computing has changed the way we live our lives and conduct business, with 60% of companies utilising a cloud workplace, and the prediction that by 2025, there will be over 100 zettabytes of data stored in the cloud. Looking at the benefits the cloud offers to the business; flexibility, scalability of costs, increased security and minimal downtime, it is easy to understand the eagerness to adopt a more cloud-centred strategy. Unsurprisingly, online or cloud data storage is becoming increasingly more popular with organisations looking to switch away from on-premise servers and data centres.

Cloud Data Warehouses

The vast majority of businesses rely on SQL databases that store the data and transform it into information in order to allow for the extraction of valuable insights. Traditionally, SQL data bases from across multiple applications were stored in on-premise physical servers, maintained by local IT departments. However, with the development and wider adoption of cloud services, more and more businesses are looking toward cloud-enabled storage solutions that would also allow them to better harness their data. Switching from an on-premises SQL-based database to a cloud solution can significantly reduce costs and maintenance overheads for IT departments, and provides increased security, as well as scalability and easier administration. However, data warehouses are complex and the migration from an on-premises server to a cloud data warehouse is far from easy. Microsoft Azure offers powerful solutions that allow you to recreate your data warehouse on the cloud: Microsoft Azure SQL Database and Azure Data Factory.

The Basics of Cloud Service Models

Before we touch upon what Azure SQL Database and Azure Data Factory are, we’re going to briefly cover the basics of the three most common service models on the cloud. In software as a Service (SaaS), the vendor provides all resources, including any apps you would like to use. Infrastructure as a Service (IaaS), on the other hand, comes with instant computing infrastructure, along with maintenance and support, but the vendor is only in charge of securing the infrastructure—which puts you, the client, in charge of securing everything else you set up on the cloud, including your own apps. Platform as a Service (PaaS) lies somewhere in between, where the vendor provides the runtime environment (which could consist of storage, servers, and network bandwidth); you can use the environment to deploy applications, but you are responsible for managing any apps and services you develop. Both the Azure Data Factory and the Azure SQL Database are PaaS solutions.

What is Azure SQL Database?

Simply put, Azure SQL Database is an infrastructure built for the cloud that lets companies manage their growing data platform. It comes with built-in features such as smart performance tuning that uses previously learned usage patterns, automatic systems to improve data security and reliability, and adaptive processes that ensure high speed and availability. Because it leverages the benefits of the cloud, it offers companies a huge advantage: virtually no downtime for the databases hosted on it, and a stated 99.99% availability. Any updates and patches are handled automatically and intelligently managed. There are two methods for migrating on-premises SQL databases to the cloud Azure SQL database, both with their advantages and disadvantages. In general, however, the more conventional, simpler method comes with a more substantial downtime during the migration. An alternative is using Azure Data Factory: A PaaS solution for data integration and migration that enables you to move your data from anon-premises SQL database into Azure SQL database on the cloud.

What is Azure Data Factory?

Azure Data Factory (ADF) is a cloud-based ETL and data integration service that allows users to create data-driven workflows for orchestrating data movement and transforming data at scale.

Azure users can move their data to the cloud by building custom ETL (“Extract, Transform, Load”) pipelines: a series of processes that pull the data from one database to another. A pipeline simply put is, a group of activities which define specific actions to be performed on the data contained in your datasets or databases. The custom data pipelines can ingest, prepare, transform, analyse or publish the data, while ADF handles more complex processing dependencies. In a nutshell, Data Factory is Azure’s ETL tool that takes the data from your on-premises warehouse (or from different sources), transforms it into information and loads it into your destination—the cloud warehouse. Any organisation that uses the Azure Data Factory can build their custom integration pipelines without the need to write even a single line of code. ADF is particularly well suited to businesses that need to continuously migrate their data in a hybrid scenario that uses both the on-premises and cloud resources, or when the data needs to be transformed during the migration.

Advantages of Azure Data Factory

The biggest advantage is its integrability: Azure Data Factory can collect and integrate data from nearly a hundred discrete sources and integrate it with Oracle, SQL Server, MySQL and other data sources, and can even be used with any competitor cloud service (such as AWS). It is available globally, with over 25 countries enabling data movement, which allows for greater accessibility. Additionally, the business owner is in charge of defining accessibility roles and assigning permissions—for instance, to ensure only certain authorised users gain access to customers’ billing information.

The simplicity, security and availability of Azure’s Data Factory are what make this cloud data migration tool the best option for small to medium-sized businesses and especially companies with a multi-cloud architecture. The services it offers come with a consumption-based pricing model and a pay-as-you-go pricing option which is a better fit for smaller businesses than the typically large upfront investment that the majority of on-premises data integration tools require. However, while coding is not a prerequisite to taking full advantage of Azure Data Factory’s offerings, configuring nonstandard data sources might require some custom code, as the pre-built connectors might not be able to handle specific proprietary databases. This links to the second disadvantage of the Azure Data Factory: while it does support some data sources that are non-native to or hosted outside of Azure, it’s mostly designed to build integration pipelines that connect to Azure.

Making the switch from on-premises data storage to a cloud-based data warehouse such as the one offered by Azure can help you significantly cut costs as well as IT-related overhead expenses, while also improving your security and bringing downtime to a minimum. If you’re considering a migration from an on-premises data warehouse to Azure SQL database, Azure Data Factory PaaS solution can handle even the most complex transformations, while minimising any migration-related downtime. However, while Azure’s tools simplify the process significantly, on-premises to cloud migration can still be challenging for most business owners: get in touch with us and we at INFuse Data Solutions would be more than happy to guide you through the process and support your cloud migration.