What is Function Calling?
Function calling in the AL Copilot Toolkit is a powerful feature that allows developers to extend the capabilities of natural language models. With function calling, you can make the copilot call specific functions or APIs based on what the user says.
So, what does this mean for you? Imagine describing a list of functions to the copilot, and it knows exactly which function to call and what information to use. This makes the system much more interactive and user-friendly.
Key Benefits:
- Extract Key Information: The copilot can pull out important details from what the user says and use them as arguments for functions.
- Stay Updated: The model can interact with your Business Central, get the latest information from external sources, and perform actions beyond just generating text.
- Determine User Intent: The copilot can understand what the user wants to do. For example, if someone says, “I need items from the last invoice,” or “I need to order bicycles,” the copilot can figure out the intent and act accordingly.
By leveraging function calling, developers can create dynamic responses, provide relevant information, and make the system more interactive. This allows users to interact with the system in a natural, conversational way.
Hotel Copilot Example
Let’s make a practical example to learn how to develop a Hotel Copilot using function calling.
The Hotel Copilot will assist users by providing information about hotels in the city, such as prices, amenities, and checking room availability.
So, what should we do, if user asks: “What hotels are available at Antwerp in June 2024 less than 250$ per night?”
You might think, “It’s simple—just send all the data (hotel lists and reservations) to Azure OpenAI along with the user’s question and get the answer.”
But that’s not the best approach.
First, there’s a limit to the number of tokens you can send to Azure OpenAI. Second, large language models (LLMs) are good at generating text, not analyzing data.
Function Calling
With the introduction of Business Central v.24.2, we’ve got much better solution – Function Calling. Let’s take a closer look at the architecture of this process.
This diagram illustrates how the Hotel Copilot uses function calling to find hotels in Antwerp for less than 100 EUR per night.
User Input: The user asks, “Find hotels in Antwerp for less than 100 EUR per night.”
Add Function Tool: The Copilot Toolkit adds the
FindHotelsfunction tool to the chat messages.Generate Chat Completion: The user’s message and the available tools are sent to Azure OpenAI for processing.
Function Call Identification: Azure OpenAI identifies the function to call (
FindHotels) and the necessary arguments based on the user’s request.Execute Function: The
FindHotelsfunction is executed in Business Central with the specified arguments (location and max price).Retrieve Function Response: The response from the
FindHotelsfunction, including the list of available hotels, is gathered.Display Results: The results are shown to the user or sent back to Azure OpenAI for further processing.
How many times do we call Azure OpenAI in this process?
Exactly, just once. Instead of sending customer data, we only send the list of available AL functions. This approach ensures that no customer data leaves the system.
AI Generated Final Answer
In some scenarios, the output from the AL function might not be very user-friendly, to show it directly to the user. To address this, we can add a final step to the process: generating a user-friendly response. We pass the function response and chat history to Azure OpenAI, get the refined answer, and display it to the user.
Prerequisites
To implement functions in the Copilot Toolkit, you need to have the following basic AL structure:
- Base Module: Includes hotels and reservations tables, and pages.
- Copilot Setup: Authentication to Azure OpenAI with a key and endpoint.
- Registration: The Hotel Copilot is registered in the Copilot Capabilities.
- Prompt Dialog: The copilot page contains a prompt, content areas, and a prompt guide with common user questions.
You can refer to my previous blog on how to create Prompt Dialog Page here.
Add Function Tool
To add a function tool, you first need to implement the “AOAI Function” codeunit. Here’s an example:
codeunit 50302 "GPT GetHotelInfo" implements "AOAI Function"
{
procedure GetPrompt() ToolJObj: JsonObject
begin
end;
procedure Execute(Arguments: JsonObject): Variant
begin
end;
procedure GetName(): Text
begin
end;
}
This codeunit will implement one function in the Copilot Toolkit. If you want to add more functions, you should implement additional codeunits.
GetName()
In this procedure, define the function name, such as ‘GetHotelInfo’.
GetPrompt()
Next, describe your function in JSON format. This description tells Azure OpenAI what your function is for, including its intent, input, output, and required parameters. The JSON should follow a specific format.
{
"type": "function",
"function": {
"name": "GetHotelInfo",
"description": "Returns information about a hotel and booking options.",
"parameters": {
"type": "object",
"properties": {
"hotel": {
"type": "string",
"description": "The name of the hotel."
},
"fromDate": {
"type": "string",
"description": "The starting date for booking."
},
"toDate": {
"type": "string",
"description": "The ending date for booking."
}
},
"required": ["hotel"]
}
}
}
The highlighted lines are essential. If your AL function requires input parameters, each must be described in the parameters.
The required property indicates which parameters are mandatory. If the user does not specify this information in the prompt, Azure OpenAI will prompt the user to rephrase the query to include them.
You can define this JSON in AL, import it from Azure KeyVault, or use isolated storage. Here’s how you might describe the function in AL:
procedure GetPrompt() ToolJObj: JsonObject
var
FunctionJObject: JsonObject;
ParametersJObject: JsonObject;
PropertiesJObject: JsonObject;
HotelJObject: JsonObject;
FromDateJObject: JsonObject;
ToDateJObject: JsonObject;
RequiredArray: JsonArray;
begin
// describe the function
ToolJObj.Add('type', 'function');
FunctionJObject.Add('name', 'GetHotelInfo');
FunctionJObject.Add('description', 'Returns information about a hotel and booking options.');
// describe the parameters
ParametersJObject.Add('type', 'object');
// describe the hotel parameter
HotelJObject.Add('type', 'string');
HotelJObject.Add('description', 'The name of the hotel.');
PropertiesJObject.Add('hotel', HotelJObject);
// describe the fromDate parameter
FromDateJObject.Add('type', 'string');
FromDateJObject.Add('description', 'The starting date for booking.');
PropertiesJObject.Add('fromDate', FromDateJObject);
// describe the toDate parameter
ToDateJObject.Add('type', 'string');
ToDateJObject.Add('description', 'The ending date for booking.');
PropertiesJObject.Add('toDate', ToDateJObject);
// add the properties to the parameters
ParametersJObject.Add('properties', PropertiesJObject);
// describe the required parameters
RequiredArray.Add('hotel');
ParametersJObject.Add('required', RequiredArray);
// add the parameters to the function
FunctionJObject.Add('parameters', ParametersJObject);
// add the function to the tool
ToolJObj.Add('function', FunctionJObject);
end
Execute()
In this procedure, define the business logic of your function. The input is a JsonObject with all AI-generated arguments based on the function JSON definition. Extract these arguments first:
ExtractArguments(Arguments, HotelName, FromDate, ToDate);
Once you have the arguments, run your business logic:
FindHotelInformation(HotelName, FromDate, ToDate, HotelInfo);
The output from the Execute() function is a Variant, so you can return any type. For this example, we will return a text with information about the found hotels.
local procedure FindHotelInformation(HotelName: Text; FromDate: Date; ToDate: Date; var HotelInfo: TextBuilder)
var
Hotel: Record "GPT Hotel";
begin
Hotel.SetFilter("Name", StrSubstNo('@*%1*', HotelName));
if Hotel.IsEmpty then begin
HotelInfo.AppendLine('No hotels are found');
exit;
end;
HotelInfo.AppendLine(StrSubstNo('Found %1 hotels.', Hotel.Count));
Hotel.FindSet();
repeat
HotelInfo.AppendLine('## Hotel Information');
HotelInfo.AppendLine(Hotel.Name);
HotelInfo.AppendLine(Hotel.City);
HotelInfo.AppendLine('Rating: ' + Format(Hotel.Rating));
HotelInfo.AppendLine(Hotel.Amenities);
HotelInfo.AppendLine('Room Price ($): ' + Format(Hotel.Price));
FindAvailabilityInformation(Hotel.Code, FromDate, ToDate, HotelInfo);
until Hotel.Next() = 0;
end;
local procedure FindAvailabilityInformation(HotelCode: Code[20]; FromDate: Date; ToDate: Date; var HotelInfo: TextBuilder)
var
HotelReservation: Record "GPT Hotel Reservation";
begin
HotelInfo.AppendLine('## Availability');
HotelInfo.AppendLine(StrSubstNo('As of %1, the hotel is: ', FromDate));
HotelReservation.SetRange("Hotel Code", HotelCode);
HotelReservation.SetFilter("Check-In Date", '<%1', ToDate);
HotelReservation.SetFilter("Check-Out Date", '>%1', FromDate);
if HotelReservation.IsEmpty then
HotelInfo.Append('available')
else
HotelInfo.Append('not available');
end;
Important Considerations for the Execute Function
When Azure OpenAI returns the selected function along with the generated arguments, the Copilot Toolkit automatically runs the Execute procedure. This means you should not save any data to the database within this function. The Copilot should be a safe environment, keeping the user in the loop, and any data should only be committed to the system upon user approval.
Additionally, note that the Copilot Toolkit runs the Execute function in a [TryFunction] mode. This means that any errors occurring inside the Execute function will not be visible to the user.
Wrapping Up: Implementing the GetAnswer Function
Now that you have implemented the function, you need to integrate everything into the GetAnswer procedure. This procedure is called when the user clicks on the ‘Ask’ action in the Hotel Copilot.
procedure GetAnswer(Question: Text; var Answer: Text)
var
AzureOpenAI: Codeunit "Azure OpenAi";
CopilotSetup: Record "GPT Booking Copilot Setup";
AOAIChatCompletionParams: Codeunit "AOAI Chat Completion Params";
AOAIChatMessages: Codeunit "AOAI Chat Messages";
GetHotelInfo: Codeunit "GPT GetHotelInfo";
GetAvailabilityByCity: Codeunit "GPT GetAvailabilityByCity";
AOAIOperationResponse: Codeunit "AOAI Operation Response";
AOAIFunctionResponse: Codeunit "AOAI Function Response";
begin
if not AzureOpenAI.IsEnabled(Enum::"Copilot Capability"::"GPT Booking Copilot") then
exit;
AzureOpenAI.SetCopilotCapability(Enum::"Copilot Capability"::"GPT Booking Copilot");
AzureOpenAI.SetAuthorization(Enum::"AOAI Model Type"::"Chat Completions", CopilotSetup.GetEndpoint(), CopilotSetup.GetDeployment(), CopilotSetup.GetSecretKey());
AOAIChatCompletionParams.SetTemperature(0);
AOAIChatMessages.AddTool(GetHotelInfo);
AOAIChatMessages.AddTool(GetAvailabilityByCity);
AOAIChatMessages.SetToolChoice('auto');
AOAIChatMessages.SetPrimarySystemMessage(GetSystemMetaprompt());
AOAIChatMessages.AddUserMessage(Question);
AzureOpenAI.GenerateChatCompletion(AOAIChatMessages, AOAIChatCompletionParams, AOAIOperationResponse);
if not AOAIOperationResponse.IsSuccess() then
Error(AOAIOperationResponse.GetError());
if AOAIOperationResponse.IsFunctionCall() then begin
AOAIFunctionResponse := AOAIOperationResponse.GetFunctionResponse();
if AOAIFunctionResponse.IsSuccess() then
Answer := GenerateFinalResponse(AzureOpenAI, AOAIChatMessages, AOAIChatCompletionParams, AOAIFunctionResponse);
end else
Answer := AOAIChatMessages.GetLastMessage();
end;
This code is consistent across all copilots with function support that you implement.
System Message
In the primary system message, describe to Azure OpenAI how it should select functions to call.
You are the hotel booking assistant.
Select one of the following functions to resolve the user query:
1. `GetHotelInfo`: to get information about a hotel.
2. `GetAvailabilityByCity`: to get available hotels in a city.
In case user asks for something else, don''t answer and ask to rephrase the question.
While you could avoid adding a system message since the function descriptions are already defined in tools, better results are achieved by additionally describing this in the system message. It also provides a place to describe what Azure OpenAI should do if the user asks something unsupported.
Generating the Final Response
The GenerateFinalResponse procedure is crucial in determining how to present the function execution result to the user. Here’s an example of the implementation:
local procedure GenerateFinalResponse(AzureOpenAI: Codeunit "Azure OpenAi"; AOAIChatMessages: Codeunit "AOAI Chat Messages"; AOAIChatCompletionParams: Codeunit "AOAI Chat Completion Params"; AOAIFunctionResponse: Codeunit "AOAI Function Response"): Text
var
AOAIOperationResponse: Codeunit "AOAI Operation Response";
begin
case AOAIFunctionResponse.GetFunctionName() of
'GetHotelInfo':
begin
AOAIChatMessages.ClearTools();
AOAIChatMessages.AddToolMessage(AOAIFunctionResponse.GetFunctionId(), AOAIFunctionResponse.GetFunctionName(), AOAIFunctionResponse.GetResult());
AzureOpenAI.GenerateChatCompletion(AOAIChatMessages, AOAIChatCompletionParams, AOAIOperationResponse);
if not AOAIOperationResponse.IsSuccess() then
Error(AOAIOperationResponse.GetError());
exit(AOAIChatMessages.GetLastMessage());
end;
'GetAvailabilityByCity':
exit(AOAIFunctionResponse.GetResult());
end;
end;
Key Considerations
Direct Result Return:
- If the function result is acceptable to return directly to the user, you should do so using
AOAIFunctionResponse.GetResult(). This approach provides faster results and eliminates the need for a second Azure OpenAI call. - This is particularly useful if the output is structured data, like records, that need to be displayed in the prompt dialog.
- If the function result is acceptable to return directly to the user, you should do so using
Generating a Final Answer:
- If a more polished response is needed, use
AOAIChatMessages.AddToolMessageto add the function execution result to the chat history. This allows Azure OpenAI to see the original user query and the function result, enabling it to generate a refined final message. - Before making this call, it’s a good practice to clear the tools using
AOAIChatMessages.ClearTools(). This ensures that Azure OpenAI focuses solely on generating the answer without considering additional tools.
- If a more polished response is needed, use
By carefully choosing between direct result return and generating a final answer, you can optimize the user experience based on the context and the nature of the function’s output.
Final Thoughts
Function calling in the Copilot Toolkit is a powerful tool that allows you to use the internal business logic of Business Central or external APIs to answer user queries and look up data without actually sending it. This enhances natural language models by enabling them to perform actions and retrieve information based on user input, making interactions more intuitive and user-friendly. It provides a seamless way to bridge human language and business logic, improving interactivity and responsiveness.
Additional Resources
There are many other possibilities you can explore with function calling, such as defining ‘magic functions,’ adding context to functions, counting tokens, and more. I highly recommend watching these BC TechDays recordings for more insights:
- Copilot Development: AI-Powered Extensions for Business Central by Dmitry Katson and Horina Serbanescu
- Microsoft Presents: Prompt Engineering and Functional Calling in AL by Darrick Joo and Qasim Ikram
Microsoft Launch Event:
- What’s New: Extending Copilot Using AL Code from the BC Lunch event
Microsoft Learn:
