Use fetch() handler

A very common use case is to provide the LLM with the ability to perform API calls via function calling.

In this example the LLM will retrieve the weather forecast for the next 5 days. To do so a getWeather function is defined that is passed to the LLM as tool.

The getWeatherfunction extracts the user’s location from the request and calls the external weather API via the Workers’ Fetch API and returns the result.

Embedded function calling example with fetch()

import { runWithTools } from '@cloudflare/ai-utils';

type Env = {
  AI: Ai;
};

export default {
  async fetch(request, env, ctx) {
    // Define function
    const getWeather = async (args: { numDays: number }) => {
      const { numDays } = args;
      // Location is extracted from request based on
      // https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties
      const lat = request.cf?.latitude
      const long = request.cf?.longitude

      // Interpolate values for external API call
      const response = await fetch(
        `https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${long}&daily=temperature_2m_max,precipitation_sum&timezone=GMT&forecast_days=${numDays}`
      );
      return response.text();
    };
    // Run AI inference with function calling
    const response = await runWithTools(
      env.AI,
      // Model with function calling support
      '@hf/nousresearch/hermes-2-pro-mistral-7b',
      {
        // Messages
        messages: [
          {
            role: 'user',
            content: 'What the weather like the next 5 days? Respond as text',
          },
        ],
        // Definition of available tools the AI model can leverage
        tools: [
          {
            name: 'getWeather',
            description: 'Get the weather for the next [numDays] days',
            parameters: {
              type: 'object',
              properties: {
                numDays: { type: 'numDays', description: 'number of days for the weather forecast' },
              },
              required: ['numDays'],
            },
            // reference to previously defined function
            function: getWeather,
          },
        ],
      }
    );
    return new Response(JSON.stringify(response));
  },
} satisfies ExportedHandler<Env>;