Function calling

Function calling enables people to take Large Language Models (LLMs) and use the model response to interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call tools. The model then intelligently understands when it needs to do a tool call, and it returns a JSON output which can be fed to another API.

In essence, you define a tool’s API schema, the LLM then generates a JSON object based on the schema, then you use the JSON object as an input to make an API call.

​​ What models support function calling?

There are open-source models which have been fine-tuned to do function calling. When browsing our model catalog, look for models with the function calling property beside it. For example, @hf/nousresearch/hermes-2-pro-mistral-7b is a fine-tuned variant of Mistral 7B that you can use for function calling.

​​ Basic function calling: input schema

With function calling, you define an array of tools with the name, description, and tool arguments. The example below shows how you would pass a tool called getWeather in an inference request to a model.

const response = await env.AI.run("@hf/nousresearch/hermes-2-pro-mistral-7b", {
      messages: [{ role: "user", content: "what is the weather in london?" }],
      tools: [
        {
          name: "getWeather",
          description: "Return the weather for a latitude and longitude",
          parameters: {
            type: "object",
            properties: {
              latitude: {
                type: "string",
                description: "The latitude for the given location",
              },
              longitude: {
                type: "string",
                description: "The longitude for the given location",
              },
            },
            required: ["latitude", "longitude"],
          },
        },
      ],
    });

return new Response(JSON.stringify(response.tool_calls));

The LLM will then return a JSON object with the required arguments and the name of the tool that was called. You can then pass this JSON object to make an API call.

[{"arguments":{"latitude":"51.5074","longitude":"-0.1278"},"name":"getWeather"}]

For a working example on how to do function calling, take a look at our demo app Open external link .