Make Data Integration Exciting Again: Data Fabric Announcements at Microsoft’s Build 2024 Tries to Tie Technology to Use Cases, Providing Important Lessons for the Industry

Microsoft’s Annual Build Conference took place from May 21 to May 23, 2024 at the Seattle Convention Center. Aimed at software engineers and web developers that use Microsoft Azure and other Microsoft offerings, the annual conference serves as one of the key events for the company to announce key updates to its technology offering for the developer community.

As data integration continues to be an important topic for enterprises—and consequently, the developer community—around the world, particular attention was given to Microsoft’s Fabric product with a range of key updates. Alongside some very developer-specific back end enhancements, the company also announced interesting upgrades to its data fabric offering that will affect its applicability to different enterprise use cases. With the data integration domain still being relatively new and therefore very dynamic, analyzing these announced improvements carry interesting implications for the likes of AWS, Google Cloud, HPE and others to successfully enhance their data integration capabilities.

Broadly speaking, the announcements around Microsoft Fabric center around three key themes: improvements to the data fabric services, improved integration of data fabric products with other technologies, and strengthening the collaboration by allowing easer integration of the data fabric offering into other Microsoft offerings.

• Enhanced Azure Data Fabric Services: First and foremost, Microsoft announced key improvements within its data fabric offering, including better real-time data analysis and a simplified process for moving data from operational to analytical databases through the Azure Synapse Link for Dataverse. Additionally, the company introduced new features such as more connectors and enhanced data transformation tools to streamline data pipeline processes. These enhancements make the data fabric offering more applicable to enterprise verticals beyond industrial manufacturing, thereby increasing the Total Addressable Market (TAM).
• Improved AI and ML Integration: Realizing that Artificial Intelligence (AI)/Machine Learning (ML) use cases, in particular, are driving enterprise demand for better integration of their currently very fragmented and siloed data, specific focus was devoted to improving the ML and AI capabilities of Microsoft data fabric through closer integration with Microsoft’s Azure Machine Learning offering for more seamless training and deployment of ML models.
• Strengthening Integration of Data Fabric into Other Microsoft Offerings: Specifically in this context, Microsoft announced the introduction of a new data collaboration tool within Microsoft Teams, enabling Teams to collaborate on data projects more effectively with built-in data visualization and analytics tools.

All of these announcements link the Microsoft Fabric solution closer to specific use cases, although a thorough integration of on-premises edge deployments still seems to be missing. In combination with AI and ML capabilities, the data fabric approach can become even more important for fostering enterprise digitization, as several existing deployments show:

• Novartis, a global healthcare company, has leveraged Microsoft's data fabric solution to create a unified data platform. This platform integrates data from various sources to support drug development and patient care.
• HSBC has adopted Microsoft's Azure Synapse Analytics to integrate and analyze vast amounts of financial data. This integration helps the bank with effective risk management and compliance.
• Siemens leverages Azure IoT and data services to integrate data from various industrial equipment. This integration enables predictive maintenance, reducing downtime and operational costs. Furthermore, the automation giant uses Azure to create digital twins of its manufacturing processes, allowing for real-time monitoring and optimization.
• In the energy generation environment, Shell integrates data from numerous sensors and systems using Azure. This integration helps optimize oil extraction processes and improve operational efficiency. In addition, Microsoft’s data fabric solution is used to manage and integrate data from various renewable energy sources.

The developments and announcements around Microsoft Fabric at Build 2024 carry a few interesting implications for cloud and data integration service providers and lead to a few recommendations. As with most technology initiatives, data integration solutions need to be tied to specific applications to generate any interest within enterprise implementers. Only by bundling these technology solutions together with certain use cases and specific outcomes (e.g., productivity/quality improvements), will key enterprise decision makers understand the real value these technology innovations can offer. There are a few distinct recommendations that other cloud and data integration service providers can follow to achieve this:

• Enhance Data Integration and Analytics Capabilities: To ensure that a data integration solution is applicable to as many different enterprise verticals and use cases as possible, these service providers should look at developing or improving real-time data integration services to ensure seamless data flow between operational and analytical databases. Furthermore, they should invest in advanced analytics and ML integration within their data platforms to provide comprehensive data-to-insight pipelines. This will bring the currently fragmented, siloed enterprise data closer together and tie them to specific use cases, which will make it easier for enterprise owners to realize the value of data integration platforms.
• Expand Multi-Cloud and Hybrid Cloud Support: Different enterprises have different requirements regarding the integrity of their data and the investment volume they are willing to spend. Consequently, data might be processed on-premises, or in a public, private, or hybrid cloud. In order to maximize the total addressable enterprise market, cloud and data integration service providers will need to expand support for multi-cloud and hybrid environments, providing consistent data management and governance capabilities similar to Azure Arc. This includes ensuring that data services are portable and manageable across different cloud platforms and on-premises.
• Invest in AI and ML Services: In tying data integration platforms to specific use cases, particular focus should be given to AI and ML applications, as these are the primary motivators for enterprises to invest in new technology. Consequently, cloud and data integration service providers should look at ways to enhance AI and ML services to be more tightly integrated with data fabric offerings. This includes providing automated ML capabilities, better tooling for data scientists, and scalable infrastructure for model training and deployment.