This week at the Build conference, Microsoft is announcing its usual range of new cloud services and enhancements to existing options, from a new Automatic service for Azure Kubernetes Service to automate cluster setup and management, to VMs running the new custom Arm Cobalt 100 processors or the latest AMD Instinct MI300x GPUs, and a new Azure Compute Fleet service to help customers provision the right mix of instances.
Microsoft Copilot for Azure is a generative AI companion to help IT pros manage Azure resources using natural language, initially through a chat interface in the Azure portal.
With so many options, it can be hard just to keep up with what’s available, let alone pick the best service for each project and get the experience to configure them all securely and optimally. Microsoft Copilot for Azure is a generative AI companion to help IT pros manage Azure resources using natural language, initially through a chat interface in the Azure portal. It can explain Azure concepts, link to the right blade in the portal, change portal settings, retrieve information about your resources, analyze metrics and costs, troubleshoot or optimize deployments and write KQL queries or deployment scripts.
Built on top of Azure Resource Manager and the Azure Resource Graph, it works for Azure Arc workloads on the edge, or even running on another cloud, using Azure as the control plane.
Asking Questions About Your Environment
“We’re going to be able to help customers through that whole lifecycle, of managing Azure from their design phase to operate to optimizing to troubleshooting,” Erin Chapple, corporate vice president, Azure Core PM & Design, told The New Stack.
We want to be able to help increase the productivity of users, irrespective of what way they’re interacting with Azure, by having this ability to engage in natural language.”
– Erin Chapple, Microsoft
There’s already a lot of information in Azure, but finding it means stepping out of the process you’re working on. “That’s one of the things that’s most relevant: how do I find the answer to my problem, how do you really help bring that into the context of the challenge or the problem that I’m trying to solve? We want to be able to help increase the productivity of users, irrespective of what way they’re interacting with Azure, by having this ability to engage in natural language.”
Copilot for Azure has been in private preview since it was announced at Ignite last year; over the next few weeks the public preview will become available (with the option of rolling it out to everyone in your tenant or choosing who can use it).
Customers in the private preview say the generative AI tool has helped with daily workflow in multiple ways, starting with understanding Azure capabilities, especially for IT admins who are less familiar with Azure, not just generally but for their own environment. “It’s a great educational tool in many ways; for example, asking ‘What are the settings that I can use to restrict inbound connectivity?’ Then as the sprawl starts to grow in the environment, asking questions like ‘What VMs in this resource group have restricted outbound connectivity?’”
With the high interest in FinOps and controlling cloud budgets, cost management was a popular feature in the private preview, Chapple noted. “Customers are saying ‘Why did my bill go up last month?’… just trying to get a general sense around that.”
They found it also simplified the user experience for automation and working at the command line. “How do I generate queries, config files, command line scripts? If I’m in an SQL database, show me a pivot summary table that displays the number of properties sold each year and give me the actual query or config file or command line script that drives right that action that you want to take. It’s not just about Q&A, it can interact with your environment and take those actions.”
New Copilot Features
The advanced RAG architecture used for Copilot for Azure makes it extremely extensible, allowing the teams who build different Azure features and services to contribute new skills using Copilot extensions that act as chat handlers for different areas of expertise. Copilot gets grounding data from an API that Microsoft calls Ask Learn, originally created to run Q&A Assist, an AI-powered search tool for Microsoft Learn’s equivalent of Stack Overflow; that means all requests incorporate the content of Microsoft Learn, either to keep the answer accurate or as a fallback if Copilot can’t offer a more specific reply.
There are several new Copilot skills for the public preview, including app troubleshooting, expertise in configuring and managing database and analytics services (including performance tuning and troubleshooting), converting natural language requests into T-SQL queries, and integration into Defender for Cloud. There are also new skills for Azure Kubernetes Service: you can use Copilot to configure AKS backups, change tiers, track down YAML files that you need to edit and construct kubectl commands.
“We’re continuing to build out the underlying set of skills that that we can expose, because that creates richer and richer scenarios that the customer can reason over.”
“We’re building out more and more capabilities,” Chapple said. “A customer can ask ‘Why is my app slow?’ or ‘How do I fix this error?’ to diagnose and resolve application issues and we’re using insights from Azure App Services diagnostics. We’re adding new capabilities to support Azure SQL database-driven applications: natural language to SQL conversion, self-help for database administrators, so Copilot can provide both relevant answers to user questions, leveraging the underlying database context, the documentation, dynamic management, views, all of those things. We’re also helping developers particularly to be able to ask questions about their data in plain text and then generating the corresponding T-SQL query.”
The security integration is particularly interesting because that’s one generative AI tool handing queries on to another: Copilot for Azure generates prompts for Defender for Cloud; and when you ask Defender External Attack Surface Management a question in natural language, it uses Copilot to write inventory queries over the data it discovers.
“That will require the customer to purchase and provision a security compute unit to enable those features, but we’re looking at how does that integration start to blend beyond simply the Azure services into some of these other Copilot experiences, so that the user experience is seamless across different services when you ask a question.”
Chapple described this as a “work in progress,” but also an example of how to surface the right skills in the right experiences across multiple Copilots.
“We’re continuing to build out the underlying set of skills that that we can expose, because that creates richer and richer scenarios that the customer can reason over. The more we have the underlying information and knowledge, when a user asks a question that may span across services, we can equip them with the best answer that’s going to help them be delighted using Azure.”
“One of these things we’re doing is adding a set of dynamic prompts to the first run experience that can help start them down a successful path.”
Getting the best results from Copilot for Azure requires knowing both what scenarios it can help with and how to write a good prompt. Microsoft has added more guidance on composing those prompts, but first-time users will also find some suggestions of what they can do with Copilot to get them started, Chapple told us. “We’ve found that customers have high expectations around what Copilot will do, but they don’t know how they can use Copilot in the environment. How do we get the customer from knowing that there’s a powerful tool they can use to be able to leverage it? One of these things we’re doing is adding a set of dynamic prompts to the first run experience that can help start them down a successful path.”
“So, for policy, we have an example prompt that is, ‘Can you show me how to use Azure policy for compliance’ or ‘Write me an Azure policy that doesn’t create a VM outside of the US’.” Whether or not you need that specific policy, it gives you an idea of what Copilot for Azure can be useful for.
She wouldn’t commit to exactly how many other Azure services would be supported by Copilot or when, but told us that “we do have aspirations that we will continue to add capabilities across the set of services that exist within Azure.” It’s also not yet clear if third-party vendors like DataStax and Elastic, who offer native services on Azure, would be able to contribute skills to Copilot.
“It’s extensible but beyond that, I’m not sure that we’ve entirely thought through that path at this point in time, because we really are focused on driving up the quality of the experiences that we are providing with first-party services.”
An AI Companion For IT
There are already multiple ways to manage Azure: the Azure CLI as well as the portal (and the mobile app version of that), ARM templates, PowerShell cmdlets, Azure SDKs for a range of programming languages, and the Bicep language for deploying Azure resource programmatically.
“We want to expose [Copilot for Azure] through the mobile app, through the command line, through other experiences…”
While the first place Copilot for Azure is built into is the Azure portal, in time it will show up as a companion for many different Azure management options, Chapple suggested.
“We want to expose it through the mobile app, through the command line, through other experiences… so that customers can have this companion right through that operational phase, from design through deployment to troubleshooting, to gain access to the information that we have, both in terms of documentation and best practices, and then on the back end, in that operations stage, to the diagnostics.”
It’s also important that what you get back isn’t just a summary of the documentation, but artifacts that you can use and reuse in your environment, like an ARM template. “In the SQL case, it’s a T-SQL query. In the case of deployment, it would be a Bicep template, or if we went down the path of extensibility it could even be Terraform.”
There are plenty of generative AI code generation tools for developers, whether that’s GitHub Copilot and Amazon Q Developer for pro developers, or tools built into low and no-code platforms. But there are fewer generative AI tools designed for IT admins, despite the popularity of CLIs, GitOps and Infrastructure as Code approaches generally.
It’s a fundamentally new way of approaching how IT pros manage resources and services that’s not dictated by how a vendor like Microsoft thinks they should work.
Copilot for Azure is aimed squarely at the IT pro, and it’s a welcome addition to their toolbox.
It’s a fundamentally new way of approaching how IT pros get to manage a broad range of resources and services that’s not dictated by how a vendor like Microsoft thinks they’re going to work, because they can stitch together the specific set of management settings they need for their workflow in their environment.
Until now, tools for IT pros have been limited to what the tool developer chose to expose from a manageability standpoint, Chapple pointed out. “To be able, post tool creation, to create a flow that goes across tools is very difficult: you have to build code to do that. I think Copilot changes that: it has access to the skills it can reason over, it can take a question that spans multiple services and pull them together.”
Of course, that same expertise in infrastructure provisioning and management is something many developers are already looking for. Copilot doesn’t just write T-SQL queries: it also explains how they’re constructed, which could help developers write their code faster. The new GitHub Copilot for Azure is an extension for Copilot Chat that will help developers discover and manage Azure resources, track down logs and troubleshoot issues; and you can imagine similar features from Copilot for Azure showing up elsewhere, perhaps directly in an IDE like VS Code.
That kind of integration would be some way down the line though, Chapple suggested.
“At Build you will start to see how some of the dev experiences can leverage some of the Azure Copilot skills, but that is part of the journey that we’re on right now: figuring out how right and how those things come together as we develop the underlying skills and make those scenarios richer and richer.”
Not Just For Beginners
Copilot has access to the live resources in your Azure tenant and while it respects the policies you have set and the resource locks, privileged identity management and Role-Based Action Control (RBAC) you have in place, if you ask it to and you have the access to do them, it can do things like shutting down all your running VMs. That means you have to check the code it gives you and the actions it suggests taking, to make sure you’re not accidentally running code that instantly and irretrievably deletes resources the way Australian financial provider UniSuper seems to have done with Google Cloud.
As Microsoft builds more Copilots for different services, Copilot for Azure could become a central place for IT pros to get answers from multiple tools.
But using Copilot to explore Azure is also a good way to gain the skills that will help you check the scripts it suggests: it would be an excellent companion for working through the Azure tutorials and certifications in Microsoft Learn that use a live sandbox on Azure to let you run real commands, like an AI tutor. That follows the same pattern Microsoft has seen with GitHub Copilot or helping both novices and experts, Chapple noted.
“Copilot for Azure is applicable to the full range of people who use Azure, anywhere from ‘I’m getting started and maybe I’m more nascent in my understanding of the operations of Azure and whatnot, helping me to really get educated and engage and ask questions and learn and understand my environment’ all the way to the more complicated scenarios where I’m trying to deeply troubleshoot an application, or I’m getting guidance on some architectural deployment of what I’m doing.”
As Microsoft builds more Copilots for different services, Copilot for Azure could become a central place for IT pros to get answers from multiple tools. It can also be a way to get access to the kind of higher-level guidance about how to get the most from cloud services like reference architectures and accelerators that Microsoft currently puts into white papers, like the new patterns and practices for private chatbots, making it easier to get things set up correctly in the first place.
“We would like customers to be able to ask Copilot ‘I’m deploying this multi-tiered application’ or ‘I’m deploying this application and I need load balancing,’ what load balancers should I use and have that follow the Well-Architected Framework,” Chapple explained. “We want to be able to bring in not only the documentation but the knowledge we have about best practices […] in order to guide you down the right path from the beginning.”
The post Microsoft Copilot for Azure: Managing Cloud Ops Through Chat appeared first on The New Stack.
IT admins get a generative AI service to help them manage Azure cloud and edge. Will this AI companion make you an instant Azure expert?