Copilot extensibility, let’s add some knowledge, or maybe not…

Now that Copilot studio has been uplifted post Microsoft Build 2024, there is a load of new features to make your Copilot authoring process more user friendly, and even pull in your own enterprise data, for which there is a new fancy term “Copilot extensibility”.

So as I like to stay on the bleeding edge, as painful as it may be… I have recently been trying to create a “Jira Assistance” bot in an attempt to surface details of currently open Jira issues from a demo Jira instance via the power of Copilot and it’s new extensible nature.

Entering this brave new world of data interoperability, one of the new features in Copilot Studio is the ability to add existing M365 Search and Intelligence connectors as “Knowledge”, this in theory surfaces data to Copilot which has already been pulled into the Graph through existing connectors.

Given the data is already available it should just be a case of giving Copilot access and then you will be able to ask conversational generative ai based questions to find out details for a specific Jira issue.

This sounds great, “Lets click some buttons and away we go”… unfortunately with any new technology it’s never quite that simple, and I may have stumbled on a slight bug which stands in the way, which I shall share below!

So this world of Copilot Extensibility all sounds very complicated, well it is and it isn’t. Like everything in the world of cloud computing it all depends on what you are trying to achieve and how you want the user to interact with your data.

In the main if the data you want to connect to is unstructured you would probably be best looking to use a Microsoft graph connector to bring knowledge into your Microsoft 365 platform.

Microsoft Graph connectors provide the easiest capability for you to ingest your unstructured, line-of-business data into your Microsoft 365 environment so that Copilot for Microsoft 365 can then reason over it. Content ingested through Graph connectors is added to the Microsoft Graph unlocking semantic understanding for prompts by users in Copilot for Microsoft 365.

Plugins, On the flip side if you are looking at connecting to a more structured data environment then you are into the more complex world of “Plugins”, here we have a few different options, depending on whether you are a “Pro Code” or “Low Code” developer.

Low-code, allows you to create your own declarative Copilots within Copilot Studio using existing actions and existing (or new) Power Platform connectors. To use these you can do most of the building work in a low-code manner within Copilot Studio.

Pro-code, is where you are entering the realms of API plugins, teams message extensions and custom graph connectors. To build these you will be working in the likes of Visual Studio Code, however you will be able to create more bespoke UI’s and workflows to custom applications to match their standards; so get your code hats on.

So as you can see there is a few Copilot extensibility few options at play; thankfully there is some decent content provided on the Microsoft Learn website for Extensibility, including a helpful table showing the benefits and limitations of each type.

There is a pre-existing template for connecting to a Jira instance from the graph, all that is required for it to work was the configuring of an OAuth2 client on the Jira side and the plugging those details into the the Client ID and Client Secret configuration screen within data source settings of the connection in M365. The main steps to do this are as follows:

• Open “Search and Intelligence” from the M365 admin centre
• Go to the “Data Sources: tab and select “Add” new connection
• Choose “Jira” from the “Other data sources” list and click next
• Give the connection and name and description
• Enter the datasource url and authentication settings.
• Proceed through the rest of the config settings taking defaults as needed
• Then review and publish the datasource

Once published you can then add some “Result Types” and “Verticals” customisations to create a more user friendly view of the results from within the earch capability within Microsoft 365

So with our Jira system connected and now being indexed, we can move onto surfacing the data within Copilot!

So we have our data, but what next? Time to expose that “knowledge” through creation of a custom Copilot, we can just use the inbuilt Copilot studio capabilities here, can’t we?

Well yes you can, but here’s where the interesting little bug appears, as we have the connector setup through Microsoft 365 Search, all we have to do, in theory add this as a knowledge source to our Copilot, and this couldn’t be more simple:

• Create a new Copilot using Copilot Studio, follow all the prompts and guidance, you will then be given an option to add existing knowledge to your Copilot
• Select the Jira connector type from the “Connect to your enterprise data” section
• You will then be shown an list of available “Jira” connectors, choose the one you need from the list.
• The “Jira” knowledge source will then be added and hey presto, your Copilot will be able to reason over the data and away you go.

But this is where the problem comes in; when you start to question the Copilot bot it keeps coming back with unable to find details for a specific issue, with a generative AI response saying to manually look in the Jira system for the details.

Not ideal Copilot to be honest, as that’s what you are supposed to be there for to save a user context switching!

Well it’s quite simple, I think….

The AI orchestration engine behind copilot tends to use descriptions of connectors and plugins in making decisions about how should rank and return data from these areas.

Each connector should have a description of the system you are connecting to and the types of data to return. Copilot then uses these textual descriptions to help it reason over the data and in effect choose when to surface it (or not) to the end user in a response.

The Microsoft documentation states: “Provide meaningful descriptions in the description property when creating connections. Rich descriptions improve the likelihood of content displaying in Copilot.” The connection description should briefly answer the following questions:

• What kind of content does this connection have?
• How do users refer to this content source in their respective organizations?
• During what part of the workflow do users refer to this content in their day-to-day work?
• What are some characteristics of the content?