Copilot Toolkit: Function Calling

What is Function Calling?

Function calling in the AL Copilot Toolkit is a powerful feature that allows developers to extend the capabilities of natural language models. With function calling, you can make the copilot call specific functions or APIs based on what the user says.

So, what does this mean for you? Imagine describing a list of functions to the copilot, and it knows exactly which function to call and what information to use. This makes the system much more interactive and user-friendly.

Key Benefits:

  • Extract Key Information: The copilot can pull out important details from what the user says and use them as arguments for functions.
  • Stay Updated: The model can interact with your Business Central, get the latest information from external sources, and perform actions beyond just generating text.
  • Determine User Intent: The copilot can understand what the user wants to do. For example, if someone says, “I need items from the last invoice,” or “I need to order bicycles,” the copilot can figure out the intent and act accordingly.

By leveraging function calling, developers can create dynamic responses, provide relevant information, and make the system more interactive. This allows users to interact with the system in a natural, conversational way.

Hotel Copilot Example

Let’s make a practical example to learn how to develop a Hotel Copilot using function calling.

The Hotel Copilot will assist users by providing information about hotels in the city, such as prices, amenities, and checking room availability.

So, what should we do, if user asks: “What hotels are available at Antwerp in June 2024 less than 250$ per night?”

You might think, “It’s simple—just send all the data (hotel lists and reservations) to Azure OpenAI along with the user’s question and get the answer.”

But that’s not the best approach.

First, there’s a limit to the number of tokens you can send to Azure OpenAI. Second, large language models (LLMs) are good at generating text, not analyzing data.

Function Calling

With the introduction of Business Central v.24.2, we’ve got much better solution – Function Calling. Let’s take a closer look at the architecture of this process.

This diagram illustrates how the Hotel Copilot uses function calling to find hotels in Antwerp for less than 100 EUR per night.

  1. User Input: The user asks, “Find hotels in Antwerp for less than 100 EUR per night.”

  2. Add Function Tool: The Copilot Toolkit adds the FindHotels function tool to the chat messages.

  3. Generate Chat Completion: The user’s message and the available tools are sent to Azure OpenAI for processing.

  4. Function Call Identification: Azure OpenAI identifies the function to call (FindHotels) and the necessary arguments based on the user’s request.

  5. Execute Function: The FindHotels function is executed in Business Central with the specified arguments (location and max price).

  6. Retrieve Function Response: The response from the FindHotels function, including the list of available hotels, is gathered.

  7. Display Results: The results are shown to the user or sent back to Azure OpenAI for further processing.

How many times do we call Azure OpenAI in this process?

Exactly, just once. Instead of sending customer data, we only send the list of available AL functions. This approach ensures that no customer data leaves the system.

AI Generated Final Answer

In some scenarios, the output from the AL function might not be very user-friendly, to show it directly to the user. To address this, we can add a final step to the process: generating a user-friendly response. We pass the function response and chat history to Azure OpenAI, get the refined answer, and display it to the user.

In this case, a total of two Azure OpenAI calls are made, and only relevant data is sent.

This is part of my PR 719 that was Merged into version 24.2. 

Prerequisites

To implement functions in the Copilot Toolkit, you need to have the following basic AL structure:

  1. Base Module: Includes hotels and reservations tables, and pages.
  2. Copilot Setup: Authentication to Azure OpenAI with a key and endpoint.
  3. Registration: The Hotel Copilot is registered in the Copilot Capabilities.
  4. Prompt Dialog: The copilot page contains a prompt, content areas, and a prompt guide with common user questions.

You can refer to my previous blog on how to create Prompt Dialog Page here.

Add Function Tool

To add a function tool, you first need to implement the “AOAI Function” codeunit. Here’s an example:

				
					codeunit 50302 "GPT GetHotelInfo" implements "AOAI Function"
{
    procedure GetPrompt() ToolJObj: JsonObject
    begin
    end;

    procedure Execute(Arguments: JsonObject): Variant
    begin
    end;

    procedure GetName(): Text
    begin
    end;
}

				
			

This codeunit will implement one function in the Copilot Toolkit. If you want to add more functions, you should implement additional codeunits.

GetName()

In this procedure, define the function name, such as ‘GetHotelInfo’.

GetPrompt()

Next, describe your function in JSON format. This description tells Azure OpenAI what your function is for, including its intent, input, output, and required parameters. The JSON should follow a specific format.

				
					{
    "type": "function",
    "function": {
        "name": "GetHotelInfo",
        "description": "Returns information about a hotel and booking options.",
        "parameters": {
            "type": "object",
            "properties": {
                "hotel": {
                    "type": "string",
                    "description": "The name of the hotel."
                },
                "fromDate": {
                    "type": "string",
                    "description": "The starting date for booking."
                },
                "toDate": {
                    "type": "string",
                    "description": "The ending date for booking."
                }
            },
            "required": ["hotel"]
        }
    }
}

				
			

The highlighted lines are essential. If your AL function requires input parameters, each must be described in the parameters.

The required property indicates which parameters are mandatory. If the user does not specify this information in the prompt, Azure OpenAI will prompt the user to rephrase the query to include them.

You can define this JSON in AL, import it from Azure KeyVault, or use isolated storage. Here’s how you might describe the function in AL:

				
					    procedure GetPrompt() ToolJObj: JsonObject
    var
        FunctionJObject: JsonObject;
        ParametersJObject: JsonObject;
        PropertiesJObject: JsonObject;
        HotelJObject: JsonObject;
        FromDateJObject: JsonObject;
        ToDateJObject: JsonObject;
        RequiredArray: JsonArray;
    begin

        // describe the function
        ToolJObj.Add('type', 'function');
        FunctionJObject.Add('name', 'GetHotelInfo');
        FunctionJObject.Add('description', 'Returns information about a hotel and booking options.');
        
        // describe the parameters
        ParametersJObject.Add('type', 'object');
        
        // describe the hotel parameter
        HotelJObject.Add('type', 'string');
        HotelJObject.Add('description', 'The name of the hotel.');
        PropertiesJObject.Add('hotel', HotelJObject);
        
        // describe the fromDate parameter
        FromDateJObject.Add('type', 'string');
        FromDateJObject.Add('description', 'The starting date for booking.');
        PropertiesJObject.Add('fromDate', FromDateJObject);
        
        // describe the toDate parameter
        ToDateJObject.Add('type', 'string');
        ToDateJObject.Add('description', 'The ending date for booking.');
        PropertiesJObject.Add('toDate', ToDateJObject);
        
        // add the properties to the parameters
        ParametersJObject.Add('properties', PropertiesJObject);
        
        // describe the required parameters
        RequiredArray.Add('hotel');
        ParametersJObject.Add('required', RequiredArray);
        
        // add the parameters to the function
        FunctionJObject.Add('parameters', ParametersJObject);
        
        // add the function to the tool
        ToolJObj.Add('function', FunctionJObject);
    end

				
			

Execute()

In this procedure, define the business logic of your function. The input is a JsonObject with all AI-generated arguments based on the function JSON definition. Extract these arguments first:

				
					ExtractArguments(Arguments, HotelName, FromDate, ToDate);

				
			

Once you have the arguments, run your business logic:

				
					FindHotelInformation(HotelName, FromDate, ToDate, HotelInfo);

				
			

The output from the Execute() function is a Variant, so you can return any type. For this example, we will return a text with information about the found hotels.

				
					    local procedure FindHotelInformation(HotelName: Text; FromDate: Date; ToDate: Date; var HotelInfo: TextBuilder)
    var
        Hotel: Record "GPT Hotel";
    begin
        Hotel.SetFilter("Name", StrSubstNo('@*%1*', HotelName));
        if Hotel.IsEmpty then begin
            HotelInfo.AppendLine('No hotels are found');
            exit;
        end;

        HotelInfo.AppendLine(StrSubstNo('Found %1 hotels.', Hotel.Count));
        Hotel.FindSet();
        repeat
            HotelInfo.AppendLine('## Hotel Information');
            HotelInfo.AppendLine(Hotel.Name);
            HotelInfo.AppendLine(Hotel.City);
            HotelInfo.AppendLine('Rating: ' + Format(Hotel.Rating));
            HotelInfo.AppendLine(Hotel.Amenities);
            HotelInfo.AppendLine('Room Price ($): ' + Format(Hotel.Price));

            FindAvailabilityInformation(Hotel.Code, FromDate, ToDate, HotelInfo);
        until Hotel.Next() = 0;
    end;

    local procedure FindAvailabilityInformation(HotelCode: Code[20]; FromDate: Date; ToDate: Date; var HotelInfo: TextBuilder)
    var
        HotelReservation: Record "GPT Hotel Reservation";
    begin
        HotelInfo.AppendLine('## Availability');
        HotelInfo.AppendLine(StrSubstNo('As of %1, the hotel is: ', FromDate));

        HotelReservation.SetRange("Hotel Code", HotelCode);
        HotelReservation.SetFilter("Check-In Date", '<%1', ToDate);
        HotelReservation.SetFilter("Check-Out Date", '>%1', FromDate);
        if HotelReservation.IsEmpty then
            HotelInfo.Append('available')
        else
            HotelInfo.Append('not available');
    end;

				
			

Important Considerations for the Execute Function

When Azure OpenAI returns the selected function along with the generated arguments, the Copilot Toolkit automatically runs the Execute procedure. This means you should not save any data to the database within this function. The Copilot should be a safe environment, keeping the user in the loop, and any data should only be committed to the system upon user approval.

Additionally, note that the Copilot Toolkit runs the Execute function in a [TryFunction] mode. This means that any errors occurring inside the Execute function will not be visible to the user.

Wrapping Up: Implementing the GetAnswer Function

Now that you have implemented the function, you need to integrate everything into the GetAnswer procedure. This procedure is called when the user clicks on the ‘Ask’ action in the Hotel Copilot.

				
					procedure GetAnswer(Question: Text; var Answer: Text)
var
    AzureOpenAI: Codeunit "Azure OpenAi";
    CopilotSetup: Record "GPT Booking Copilot Setup";
    AOAIChatCompletionParams: Codeunit "AOAI Chat Completion Params";
    AOAIChatMessages: Codeunit "AOAI Chat Messages";
    GetHotelInfo: Codeunit "GPT GetHotelInfo";
    GetAvailabilityByCity: Codeunit "GPT GetAvailabilityByCity";
    AOAIOperationResponse: Codeunit "AOAI Operation Response";
    AOAIFunctionResponse: Codeunit "AOAI Function Response";
begin
    if not AzureOpenAI.IsEnabled(Enum::"Copilot Capability"::"GPT Booking Copilot") then
        exit;

    AzureOpenAI.SetCopilotCapability(Enum::"Copilot Capability"::"GPT Booking Copilot");
    AzureOpenAI.SetAuthorization(Enum::"AOAI Model Type"::"Chat Completions", CopilotSetup.GetEndpoint(), CopilotSetup.GetDeployment(), CopilotSetup.GetSecretKey());

    AOAIChatCompletionParams.SetTemperature(0);

    AOAIChatMessages.AddTool(GetHotelInfo);
    AOAIChatMessages.AddTool(GetAvailabilityByCity);
    AOAIChatMessages.SetToolChoice('auto');

    AOAIChatMessages.SetPrimarySystemMessage(GetSystemMetaprompt());
    AOAIChatMessages.AddUserMessage(Question);

    AzureOpenAI.GenerateChatCompletion(AOAIChatMessages, AOAIChatCompletionParams, AOAIOperationResponse);

    if not AOAIOperationResponse.IsSuccess() then
        Error(AOAIOperationResponse.GetError());

    if AOAIOperationResponse.IsFunctionCall() then begin
        AOAIFunctionResponse := AOAIOperationResponse.GetFunctionResponse();
        if AOAIFunctionResponse.IsSuccess() then
            Answer := GenerateFinalResponse(AzureOpenAI, AOAIChatMessages, AOAIChatCompletionParams, AOAIFunctionResponse);
    end else
        Answer := AOAIChatMessages.GetLastMessage();
end;

				
			

This code is consistent across all copilots with function support that you implement.

System Message

In the primary system message, describe to Azure OpenAI how it should select functions to call.

				
					You are the hotel booking assistant.
Select one of the following functions to resolve the user query:
1. `GetHotelInfo`: to get information about a hotel.
2. `GetAvailabilityByCity`: to get available hotels in a city.
In case user asks for something else, don''t answer and ask to rephrase the question.
				
			

While you could avoid adding a system message since the function descriptions are already defined in tools, better results are achieved by additionally describing this in the system message. It also provides a place to describe what Azure OpenAI should do if the user asks something unsupported.

Generating the Final Response

The GenerateFinalResponse procedure is crucial in determining how to present the function execution result to the user. Here’s an example of the implementation:

				
					local procedure GenerateFinalResponse(AzureOpenAI: Codeunit "Azure OpenAi"; AOAIChatMessages: Codeunit "AOAI Chat Messages"; AOAIChatCompletionParams: Codeunit "AOAI Chat Completion Params"; AOAIFunctionResponse: Codeunit "AOAI Function Response"): Text
var
    AOAIOperationResponse: Codeunit "AOAI Operation Response";
begin
    case AOAIFunctionResponse.GetFunctionName() of
        'GetHotelInfo':
            begin
                AOAIChatMessages.ClearTools();
                AOAIChatMessages.AddToolMessage(AOAIFunctionResponse.GetFunctionId(), AOAIFunctionResponse.GetFunctionName(), AOAIFunctionResponse.GetResult());
                AzureOpenAI.GenerateChatCompletion(AOAIChatMessages, AOAIChatCompletionParams, AOAIOperationResponse);

                if not AOAIOperationResponse.IsSuccess() then
                    Error(AOAIOperationResponse.GetError());

                exit(AOAIChatMessages.GetLastMessage());
            end;
        'GetAvailabilityByCity':
            exit(AOAIFunctionResponse.GetResult());
    end;
end;

				
			

Key Considerations

  1. Direct Result Return:

    • If the function result is acceptable to return directly to the user, you should do so using AOAIFunctionResponse.GetResult(). This approach provides faster results and eliminates the need for a second Azure OpenAI call.
    • This is particularly useful if the output is structured data, like records, that need to be displayed in the prompt dialog.
  2. Generating a Final Answer:

    • If a more polished response is needed, use AOAIChatMessages.AddToolMessage to add the function execution result to the chat history. This allows Azure OpenAI to see the original user query and the function result, enabling it to generate a refined final message.
    • Before making this call, it’s a good practice to clear the tools using AOAIChatMessages.ClearTools(). This ensures that Azure OpenAI focuses solely on generating the answer without considering additional tools.

By carefully choosing between direct result return and generating a final answer, you can optimize the user experience based on the context and the nature of the function’s output.

Final Thoughts

Function calling in the Copilot Toolkit is a powerful tool that allows you to use the internal business logic of Business Central or external APIs to answer user queries and look up data without actually sending it. This enhances natural language models by enabling them to perform actions and retrieve information based on user input, making interactions more intuitive and user-friendly. It provides a seamless way to bridge human language and business logic, improving interactivity and responsiveness.

Additional Resources

There are many other possibilities you can explore with function calling, such as defining ‘magic functions,’ adding context to functions, counting tokens, and more. I highly recommend watching these BC TechDays recordings for more insights:

Microsoft Launch Event:

Microsoft Learn:

Share Post:

Leave a Reply

About Me

DMITRY KATSON

A Microsoft MVP, Business Central architect and a project manager, blogger and a speaker, husband and a twice a father. With more than 15 years in business, I went from developer to company owner. Having a great team, still love to put my hands on code and create AI powered Business Central Apps that just works.

Follow Me

Recent Posts

Tags