- New “computer use” feature for Copilot Studio gives AI agents more tools
- Agents can now interact directly with apps or websites
- This can help with data entry, analysis, research and much more
Microsoft is giving its Copilot Studio AI platform more autonomy in accessing websites and apps directly – without any human interaction.
The company has revealed a new “computer use” feature for Copilot Studio, which will allow the platform to directly interact with desktop apps and websites alike, such as clicking buttons, selecting menus, and even typing into fields on the screen.
This means companies will be able to build AI agents with a much greater understanding of such systems (essentially anything with a graphical user interface), treating them as if they were tools, allowing for improved functionality across the board.
“Computer use”
“Copilot Studio is the end-to-end agent platform designed to help organizations achieve their AI and operational goals,” Charles Lamanna, corporate vice president of Microsoft’s business & industry Copilot, wrote in a blog post announcing the news.
“We want to empower you to streamline processes, enhance productivity, and drive innovation.”
Computer use will allow agents to handle tasks even when there is no API available to connect to the system directly, and can automatically adapt to any changes in the app or website.
Developers can use the tool to build agents across both desktop and browser applications, with Edge, Chrome, and Firefox all included from the launch.
It works in real-time, using built-in reasoning tools to work out solutions to problems independently, so development can continue uninterrupted.
Lamanna laid out several possible scenarios where computer use could prove especially handy for developers, including automating the input of large volumes of data from various sources into a centralized system, carrying out widespread market research from various online sources, and processing invoices much faster and more accurately than before.
As the tool runs on Microsoft-hosted infrastructure, users don’t need to manage their own servers, and it includes Copilot Studio’s existing security measures and governance frameworks to help ensure compliance with industry standards.
The launch, which will see more attention at Microsoft Build in May 2025, follows a similar announcement of “Actions” in the commercial version of Copilot recently. This system allows for background completion of tasks such as making restaurant reservations, booking event tickets, and purchasing items from online stores.
Leave a comment