Skip to content
Cloudflare Docs
i

granite-4.0-h-micro

Text Generation • ibm-granite
@cf/ibm-granite/granite-4.0-h-micro

Granite 4.0 instruct models deliver strong performance across benchmarks, achieving industry-leading results in key agentic tasks like instruction following and function calling. These efficiencies make the models well-suited for a wide range of use cases like retrieval-augmented generation (RAG), multi-agent workflows, and edge deployments.

Model Info
Context Window ↗131,000 tokens
Unit Pricing$0.017 per M input tokens, $0.11 per M output tokens

Playground

Try out this model with Workers AI LLM Playground. It does not require any setup or authentication and an instant way to preview and test a model directly in the browser.

Launch the LLM Playground

Usage

Worker - Streaming
TypeScript
export interface Env {
  AI: Ai;
}


export default {
  async fetch(request, env): Promise<Response> {


    const messages = [
      { role: "system", content: "You are a friendly assistant" },
      {
        role: "user",
        content: "What is the origin of the phrase Hello, World",
      },
    ];


    const stream = await env.AI.run("@cf/ibm-granite/granite-4.0-h-micro", {
      messages,
      stream: true,
    });


    return new Response(stream, {
      headers: { "content-type": "text/event-stream" },
    });
  },
} satisfies ExportedHandler<Env>;

Worker
TypeScript
export interface Env {
  AI: Ai;
}


export default {
  async fetch(request, env): Promise<Response> {


    const messages = [
      { role: "system", content: "You are a friendly assistant" },
      {
        role: "user",
        content: "What is the origin of the phrase Hello, World",
      },
    ];
    const response = await env.AI.run("@cf/ibm-granite/granite-4.0-h-micro", { messages });


    return Response.json(response);
  },
} satisfies ExportedHandler<Env>;

Python
Python
import os
import requests


ACCOUNT_ID = "your-account-id"
AUTH_TOKEN = os.environ.get("CLOUDFLARE_AUTH_TOKEN")


prompt = "Tell me all about PEP-8"
response = requests.post(
  f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/ibm-granite/granite-4.0-h-micro",
    headers={"Authorization": f"Bearer {AUTH_TOKEN}"},
    json={
      "messages": [
        {"role": "system", "content": "You are a friendly assistant"},
        {"role": "user", "content": prompt}
      ]
    }
)
result = response.json()
print(result)

curl
Terminal window
curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/ibm-granite/granite-4.0-h-micro \
  -X POST \
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \
  -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}'

Parameters

* indicates a required field

Input

  • 0 object

    • prompt string required min 1

      The input text prompt for the model to generate a response.

    • lora string

      Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model.

    • response_format object

      • type string

      • json_schema

    • raw boolean

      If true, a chat template is not applied and you must adhere to the specific model's expected formatting.

    • stream boolean

      If true, the response will be streamed back incrementally using SSE, Server Sent Events.

    • max_tokens integer default 2000

      The maximum number of tokens to generate in the response.

    • temperature number default 0.6 min 0 max 5

      Controls the randomness of the output; higher values produce more random results.

    • top_p number min 0.001 max 1

      Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses.

    • top_k integer min 1 max 50

      Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises.

    • seed integer min 1 max 9999999999

      Random seed for reproducibility of the generation.

    • repetition_penalty number min 0 max 2

      Penalty for repeated tokens; higher values discourage repetition.

    • frequency_penalty number min -2 max 2

      Decreases the likelihood of the model repeating the same lines verbatim.

    • presence_penalty number min -2 max 2

      Increases the likelihood of the model introducing new topics.

  • 1 object

    • messages array required

      An array of message objects representing the conversation history.

      • items object

        • role string required

          The role of the message sender (e.g., 'user', 'assistant', 'system', 'tool').

        • content string required

          The content of the message as a string.

    • functions array

      • items object

        • name string required

        • code string required

    • tools array

      A list of tools available for the assistant to use.

      • items one of

        • 0 object

          • name string required

            The name of the tool. More descriptive the better.

          • description string required

            A brief description of what the tool does.

          • parameters object required

            Schema defining the parameters accepted by the tool.

            • type string required

              The type of the parameters object (usually 'object').

            • required array

              List of required parameter names.

              • items string

            • properties object required

              Definitions of each parameter.

              • additionalProperties object

                • type string required

                  The data type of the parameter.

                • description string required

                  A description of the expected parameter.

        • 1 object

          • type string required

            Specifies the type of tool (e.g., 'function').

          • function object required

            Details of the function tool.

            • name string required

              The name of the function.

            • description string required

              A brief description of what the function does.

            • parameters object required

              Schema defining the parameters accepted by the function.

              • type string required

                The type of the parameters object (usually 'object').

              • required array

                List of required parameter names.

                • items string

              • properties object required

                Definitions of each parameter.

                • additionalProperties object

                  • type string required

                    The data type of the parameter.

                  • description string required

                    A description of the expected parameter.

    • response_format object

      • type string

      • json_schema

    • raw boolean

      If true, a chat template is not applied and you must adhere to the specific model's expected formatting.

    • stream boolean

      If true, the response will be streamed back incrementally using SSE, Server Sent Events.

    • max_tokens integer default 2000

      The maximum number of tokens to generate in the response.

    • temperature number default 0.6 min 0 max 5

      Controls the randomness of the output; higher values produce more random results.

    • top_p number min 0.001 max 1

      Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses.

    • top_k integer min 1 max 50

      Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises.

    • seed integer min 1 max 9999999999

      Random seed for reproducibility of the generation.

    • repetition_penalty number min 0 max 2

      Penalty for repeated tokens; higher values discourage repetition.

    • frequency_penalty number min -2 max 2

      Decreases the likelihood of the model repeating the same lines verbatim.

    • presence_penalty number min -2 max 2

      Increases the likelihood of the model introducing new topics.

  • 2 object

    • requests array required

      • items one of

        • 0 object

          • prompt string required min 1

            The input text prompt for the model to generate a response.

          • lora string

            Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model.

          • response_format object

            • type string

            • json_schema

          • raw boolean

            If true, a chat template is not applied and you must adhere to the specific model's expected formatting.

          • stream boolean

            If true, the response will be streamed back incrementally using SSE, Server Sent Events.

          • max_tokens integer default 256

            The maximum number of tokens to generate in the response.

          • temperature number default 0.6 min 0 max 5

            Controls the randomness of the output; higher values produce more random results.

          • top_p number min 0.001 max 1

            Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses.

          • top_k integer min 1 max 50

            Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises.

          • seed integer min 1 max 9999999999

            Random seed for reproducibility of the generation.

          • repetition_penalty number min 0 max 2

            Penalty for repeated tokens; higher values discourage repetition.

          • frequency_penalty number min -2 max 2

            Decreases the likelihood of the model repeating the same lines verbatim.

          • presence_penalty number min -2 max 2

            Increases the likelihood of the model introducing new topics.

        • 1 object

          • messages array required

            An array of message objects representing the conversation history.

            • items object

              • role string required

                The role of the message sender (e.g., 'user', 'assistant', 'system', 'tool').

              • content string required

                The content of the message as a string.

          • functions array

            • items object

              • name string required

              • code string required

          • tools array

            A list of tools available for the assistant to use.

            • items one of

              • 0 object

                • name string required

                  The name of the tool. More descriptive the better.

                • description string required

                  A brief description of what the tool does.

                • parameters object required

                  Schema defining the parameters accepted by the tool.

                  • type string required

                    The type of the parameters object (usually 'object').

                  • required array

                    List of required parameter names.

                    • items string

                  • properties object required

                    Definitions of each parameter.

                    • additionalProperties object

                      • type string required

                        The data type of the parameter.

                      • description string required

                        A description of the expected parameter.

              • 1 object

                • type string required

                  Specifies the type of tool (e.g., 'function').

                • function object required

                  Details of the function tool.

                  • name string required

                    The name of the function.

                  • description string required

                    A brief description of what the function does.

                  • parameters object required

                    Schema defining the parameters accepted by the function.

                    • type string required

                      The type of the parameters object (usually 'object').

                    • required array

                      List of required parameter names.

                      • items string

                    • properties object required

                      Definitions of each parameter.

                      • additionalProperties object

                        • type string required

                          The data type of the parameter.

                        • description string required

                          A description of the expected parameter.

          • response_format object

            • type string

            • json_schema

          • raw boolean

            If true, a chat template is not applied and you must adhere to the specific model's expected formatting.

          • stream boolean

            If true, the response will be streamed back incrementally using SSE, Server Sent Events.

          • max_tokens integer default 256

            The maximum number of tokens to generate in the response.

          • temperature number default 0.6 min 0 max 5

            Controls the randomness of the output; higher values produce more random results.

          • top_p number min 0.001 max 1

            Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses.

          • top_k integer min 1 max 50

            Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises.

          • seed integer min 1 max 9999999999

            Random seed for reproducibility of the generation.

          • repetition_penalty number min 0 max 2

            Penalty for repeated tokens; higher values discourage repetition.

          • frequency_penalty number min -2 max 2

            Decreases the likelihood of the model repeating the same lines verbatim.

          • presence_penalty number min -2 max 2

            Increases the likelihood of the model introducing new topics.

Output

  • 0 object

    • id string

      Unique identifier for the completion

    • object string

      Object type identifier

    • created number

      Unix timestamp of when the completion was created

    • model string

      Model used for the completion

    • choices array

      List of completion choices

      • items object

        • index number

          Index of the choice in the list

        • message object

          The message generated by the model

          • role string required

            Role of the message author

          • content string required

            The content of the message

          • reasoning_content string

            Internal reasoning content (if available)

          • tool_calls array

            Tool calls made by the assistant

            • items object

              • id string required

                Unique identifier for the tool call

              • type string required

                Type of tool call

              • function object required

                • name string required

                  Name of the function to call

                • arguments string required

                  JSON string of arguments for the function

        • finish_reason string

          Reason why the model stopped generating

        • stop_reason string

          Stop reason (may be null)

        • logprobs object

          Log probabilities (if requested)

    • usage object

      Usage statistics for the inference request

      • prompt_tokens number 0

        Total number of tokens in input

      • completion_tokens number 0

        Total number of tokens in output

      • total_tokens number 0

        Total number of input and output tokens

    • prompt_logprobs object

      Log probabilities for the prompt (if requested)

  • 1 object

    • id string

      Unique identifier for the completion

    • object string

      Object type identifier

    • created number

      Unix timestamp of when the completion was created

    • model string

      Model used for the completion

    • choices array

      List of completion choices

      • items object

        • index number required

          Index of the choice in the list

        • text string required

          The generated text completion

        • finish_reason string required

          Reason why the model stopped generating

        • stop_reason string

          Stop reason (may be null)

        • logprobs object

          Log probabilities (if requested)

        • prompt_logprobs object

          Log probabilities for the prompt (if requested)

    • usage object

      Usage statistics for the inference request

      • prompt_tokens number 0

        Total number of tokens in input

      • completion_tokens number 0

        Total number of tokens in output

      • total_tokens number 0

        Total number of input and output tokens

  • 2 string

  • 3 object

    • request_id string

      The async request id that can be used to obtain the results.

API Schemas

The following schemas are based on JSON Schema

{
    "type": "object",
    "oneOf": [
        {
            "title": "Prompt",
            "properties": {
                "prompt": {
                    "type": "string",
                    "minLength": 1,
                    "description": "The input text prompt for the model to generate a response."
                },
                "lora": {
                    "type": "string",
                    "description": "Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model."
                },
                "response_format": {
                    "title": "JSON Mode",
                    "type": "object",
                    "properties": {
                        "type": {
                            "type": "string",
                            "enum": [
                                "json_object",
                                "json_schema"
                            ]
                        },
                        "json_schema": {}
                    }
                },
                "raw": {
                    "type": "boolean",
                    "default": false,
                    "description": "If true, a chat template is not applied and you must adhere to the specific model's expected formatting."
                },
                "stream": {
                    "type": "boolean",
                    "default": false,
                    "description": "If true, the response will be streamed back incrementally using SSE, Server Sent Events."
                },
                "max_tokens": {
                    "type": "integer",
                    "default": 2000,
                    "description": "The maximum number of tokens to generate in the response."
                },
                "temperature": {
                    "type": "number",
                    "default": 0.6,
                    "minimum": 0,
                    "maximum": 5,
                    "description": "Controls the randomness of the output; higher values produce more random results."
                },
                "top_p": {
                    "type": "number",
                    "minimum": 0.001,
                    "maximum": 1,
                    "description": "Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses."
                },
                "top_k": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 50,
                    "description": "Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises."
                },
                "seed": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 9999999999,
                    "description": "Random seed for reproducibility of the generation."
                },
                "repetition_penalty": {
                    "type": "number",
                    "minimum": 0,
                    "maximum": 2,
                    "description": "Penalty for repeated tokens; higher values discourage repetition."
                },
                "frequency_penalty": {
                    "type": "number",
                    "minimum": -2,
                    "maximum": 2,
                    "description": "Decreases the likelihood of the model repeating the same lines verbatim."
                },
                "presence_penalty": {
                    "type": "number",
                    "minimum": -2,
                    "maximum": 2,
                    "description": "Increases the likelihood of the model introducing new topics."
                }
            },
            "required": [
                "prompt"
            ]
        },
        {
            "title": "Messages",
            "properties": {
                "messages": {
                    "type": "array",
                    "description": "An array of message objects representing the conversation history.",
                    "items": {
                        "type": "object",
                        "properties": {
                            "role": {
                                "type": "string",
                                "description": "The role of the message sender (e.g., 'user', 'assistant', 'system', 'tool')."
                            },
                            "content": {
                                "type": "string",
                                "description": "The content of the message as a string."
                            }
                        },
                        "required": [
                            "role",
                            "content"
                        ]
                    }
                },
                "functions": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {
                                "type": "string"
                            },
                            "code": {
                                "type": "string"
                            }
                        },
                        "required": [
                            "name",
                            "code"
                        ]
                    }
                },
                "tools": {
                    "type": "array",
                    "description": "A list of tools available for the assistant to use.",
                    "items": {
                        "type": "object",
                        "oneOf": [
                            {
                                "properties": {
                                    "name": {
                                        "type": "string",
                                        "description": "The name of the tool. More descriptive the better."
                                    },
                                    "description": {
                                        "type": "string",
                                        "description": "A brief description of what the tool does."
                                    },
                                    "parameters": {
                                        "type": "object",
                                        "description": "Schema defining the parameters accepted by the tool.",
                                        "properties": {
                                            "type": {
                                                "type": "string",
                                                "description": "The type of the parameters object (usually 'object')."
                                            },
                                            "required": {
                                                "type": "array",
                                                "description": "List of required parameter names.",
                                                "items": {
                                                    "type": "string"
                                                }
                                            },
                                            "properties": {
                                                "type": "object",
                                                "description": "Definitions of each parameter.",
                                                "additionalProperties": {
                                                    "type": "object",
                                                    "properties": {
                                                        "type": {
                                                            "type": "string",
                                                            "description": "The data type of the parameter."
                                                        },
                                                        "description": {
                                                            "type": "string",
                                                            "description": "A description of the expected parameter."
                                                        }
                                                    },
                                                    "required": [
                                                        "type",
                                                        "description"
                                                    ]
                                                }
                                            }
                                        },
                                        "required": [
                                            "type",
                                            "properties"
                                        ]
                                    }
                                },
                                "required": [
                                    "name",
                                    "description",
                                    "parameters"
                                ]
                            },
                            {
                                "properties": {
                                    "type": {
                                        "type": "string",
                                        "description": "Specifies the type of tool (e.g., 'function')."
                                    },
                                    "function": {
                                        "type": "object",
                                        "description": "Details of the function tool.",
                                        "properties": {
                                            "name": {
                                                "type": "string",
                                                "description": "The name of the function."
                                            },
                                            "description": {
                                                "type": "string",
                                                "description": "A brief description of what the function does."
                                            },
                                            "parameters": {
                                                "type": "object",
                                                "description": "Schema defining the parameters accepted by the function.",
                                                "properties": {
                                                    "type": {
                                                        "type": "string",
                                                        "description": "The type of the parameters object (usually 'object')."
                                                    },
                                                    "required": {
                                                        "type": "array",
                                                        "description": "List of required parameter names.",
                                                        "items": {
                                                            "type": "string"
                                                        }
                                                    },
                                                    "properties": {
                                                        "type": "object",
                                                        "description": "Definitions of each parameter.",
                                                        "additionalProperties": {
                                                            "type": "object",
                                                            "properties": {
                                                                "type": {
                                                                    "type": "string",
                                                                    "description": "The data type of the parameter."
                                                                },
                                                                "description": {
                                                                    "type": "string",
                                                                    "description": "A description of the expected parameter."
                                                                }
                                                            },
                                                            "required": [
                                                                "type",
                                                                "description"
                                                            ]
                                                        }
                                                    }
                                                },
                                                "required": [
                                                    "type",
                                                    "properties"
                                                ]
                                            }
                                        },
                                        "required": [
                                            "name",
                                            "description",
                                            "parameters"
                                        ]
                                    }
                                },
                                "required": [
                                    "type",
                                    "function"
                                ]
                            }
                        ]
                    }
                },
                "response_format": {
                    "title": "JSON Mode",
                    "type": "object",
                    "properties": {
                        "type": {
                            "type": "string",
                            "enum": [
                                "json_object",
                                "json_schema"
                            ]
                        },
                        "json_schema": {}
                    }
                },
                "raw": {
                    "type": "boolean",
                    "default": false,
                    "description": "If true, a chat template is not applied and you must adhere to the specific model's expected formatting."
                },
                "stream": {
                    "type": "boolean",
                    "default": false,
                    "description": "If true, the response will be streamed back incrementally using SSE, Server Sent Events."
                },
                "max_tokens": {
                    "type": "integer",
                    "default": 2000,
                    "description": "The maximum number of tokens to generate in the response."
                },
                "temperature": {
                    "type": "number",
                    "default": 0.6,
                    "minimum": 0,
                    "maximum": 5,
                    "description": "Controls the randomness of the output; higher values produce more random results."
                },
                "top_p": {
                    "type": "number",
                    "minimum": 0.001,
                    "maximum": 1,
                    "description": "Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses."
                },
                "top_k": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 50,
                    "description": "Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises."
                },
                "seed": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 9999999999,
                    "description": "Random seed for reproducibility of the generation."
                },
                "repetition_penalty": {
                    "type": "number",
                    "minimum": 0,
                    "maximum": 2,
                    "description": "Penalty for repeated tokens; higher values discourage repetition."
                },
                "frequency_penalty": {
                    "type": "number",
                    "minimum": -2,
                    "maximum": 2,
                    "description": "Decreases the likelihood of the model repeating the same lines verbatim."
                },
                "presence_penalty": {
                    "type": "number",
                    "minimum": -2,
                    "maximum": 2,
                    "description": "Increases the likelihood of the model introducing new topics."
                }
            },
            "required": [
                "messages"
            ]
        },
        {
            "title": "Async Batch",
            "type": "object",
            "properties": {
                "requests": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "oneOf": [
                            {
                                "title": "Prompt",
                                "properties": {
                                    "prompt": {
                                        "type": "string",
                                        "minLength": 1,
                                        "description": "The input text prompt for the model to generate a response."
                                    },
                                    "lora": {
                                        "type": "string",
                                        "description": "Name of the LoRA (Low-Rank Adaptation) model to fine-tune the base model."
                                    },
                                    "response_format": {
                                        "title": "JSON Mode",
                                        "type": "object",
                                        "properties": {
                                            "type": {
                                                "type": "string",
                                                "enum": [
                                                    "json_object",
                                                    "json_schema"
                                                ]
                                            },
                                            "json_schema": {}
                                        }
                                    },
                                    "raw": {
                                        "type": "boolean",
                                        "default": false,
                                        "description": "If true, a chat template is not applied and you must adhere to the specific model's expected formatting."
                                    },
                                    "stream": {
                                        "type": "boolean",
                                        "default": false,
                                        "description": "If true, the response will be streamed back incrementally using SSE, Server Sent Events."
                                    },
                                    "max_tokens": {
                                        "type": "integer",
                                        "default": 256,
                                        "description": "The maximum number of tokens to generate in the response."
                                    },
                                    "temperature": {
                                        "type": "number",
                                        "default": 0.6,
                                        "minimum": 0,
                                        "maximum": 5,
                                        "description": "Controls the randomness of the output; higher values produce more random results."
                                    },
                                    "top_p": {
                                        "type": "number",
                                        "minimum": 0.001,
                                        "maximum": 1,
                                        "description": "Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses."
                                    },
                                    "top_k": {
                                        "type": "integer",
                                        "minimum": 1,
                                        "maximum": 50,
                                        "description": "Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises."
                                    },
                                    "seed": {
                                        "type": "integer",
                                        "minimum": 1,
                                        "maximum": 9999999999,
                                        "description": "Random seed for reproducibility of the generation."
                                    },
                                    "repetition_penalty": {
                                        "type": "number",
                                        "minimum": 0,
                                        "maximum": 2,
                                        "description": "Penalty for repeated tokens; higher values discourage repetition."
                                    },
                                    "frequency_penalty": {
                                        "type": "number",
                                        "minimum": -2,
                                        "maximum": 2,
                                        "description": "Decreases the likelihood of the model repeating the same lines verbatim."
                                    },
                                    "presence_penalty": {
                                        "type": "number",
                                        "minimum": -2,
                                        "maximum": 2,
                                        "description": "Increases the likelihood of the model introducing new topics."
                                    }
                                },
                                "required": [
                                    "prompt"
                                ]
                            },
                            {
                                "title": "Messages",
                                "properties": {
                                    "messages": {
                                        "type": "array",
                                        "description": "An array of message objects representing the conversation history.",
                                        "items": {
                                            "type": "object",
                                            "properties": {
                                                "role": {
                                                    "type": "string",
                                                    "description": "The role of the message sender (e.g., 'user', 'assistant', 'system', 'tool')."
                                                },
                                                "content": {
                                                    "type": "string",
                                                    "description": "The content of the message as a string."
                                                }
                                            },
                                            "required": [
                                                "role",
                                                "content"
                                            ]
                                        }
                                    },
                                    "functions": {
                                        "type": "array",
                                        "items": {
                                            "type": "object",
                                            "properties": {
                                                "name": {
                                                    "type": "string"
                                                },
                                                "code": {
                                                    "type": "string"
                                                }
                                            },
                                            "required": [
                                                "name",
                                                "code"
                                            ]
                                        }
                                    },
                                    "tools": {
                                        "type": "array",
                                        "description": "A list of tools available for the assistant to use.",
                                        "items": {
                                            "type": "object",
                                            "oneOf": [
                                                {
                                                    "properties": {
                                                        "name": {
                                                            "type": "string",
                                                            "description": "The name of the tool. More descriptive the better."
                                                        },
                                                        "description": {
                                                            "type": "string",
                                                            "description": "A brief description of what the tool does."
                                                        },
                                                        "parameters": {
                                                            "type": "object",
                                                            "description": "Schema defining the parameters accepted by the tool.",
                                                            "properties": {
                                                                "type": {
                                                                    "type": "string",
                                                                    "description": "The type of the parameters object (usually 'object')."
                                                                },
                                                                "required": {
                                                                    "type": "array",
                                                                    "description": "List of required parameter names.",
                                                                    "items": {
                                                                        "type": "string"
                                                                    }
                                                                },
                                                                "properties": {
                                                                    "type": "object",
                                                                    "description": "Definitions of each parameter.",
                                                                    "additionalProperties": {
                                                                        "type": "object",
                                                                        "properties": {
                                                                            "type": {
                                                                                "type": "string",
                                                                                "description": "The data type of the parameter."
                                                                            },
                                                                            "description": {
                                                                                "type": "string",
                                                                                "description": "A description of the expected parameter."
                                                                            }
                                                                        },
                                                                        "required": [
                                                                            "type",
                                                                            "description"
                                                                        ]
                                                                    }
                                                                }
                                                            },
                                                            "required": [
                                                                "type",
                                                                "properties"
                                                            ]
                                                        }
                                                    },
                                                    "required": [
                                                        "name",
                                                        "description",
                                                        "parameters"
                                                    ]
                                                },
                                                {
                                                    "properties": {
                                                        "type": {
                                                            "type": "string",
                                                            "description": "Specifies the type of tool (e.g., 'function')."
                                                        },
                                                        "function": {
                                                            "type": "object",
                                                            "description": "Details of the function tool.",
                                                            "properties": {
                                                                "name": {
                                                                    "type": "string",
                                                                    "description": "The name of the function."
                                                                },
                                                                "description": {
                                                                    "type": "string",
                                                                    "description": "A brief description of what the function does."
                                                                },
                                                                "parameters": {
                                                                    "type": "object",
                                                                    "description": "Schema defining the parameters accepted by the function.",
                                                                    "properties": {
                                                                        "type": {
                                                                            "type": "string",
                                                                            "description": "The type of the parameters object (usually 'object')."
                                                                        },
                                                                        "required": {
                                                                            "type": "array",
                                                                            "description": "List of required parameter names.",
                                                                            "items": {
                                                                                "type": "string"
                                                                            }
                                                                        },
                                                                        "properties": {
                                                                            "type": "object",
                                                                            "description": "Definitions of each parameter.",
                                                                            "additionalProperties": {
                                                                                "type": "object",
                                                                                "properties": {
                                                                                    "type": {
                                                                                        "type": "string",
                                                                                        "description": "The data type of the parameter."
                                                                                    },
                                                                                    "description": {
                                                                                        "type": "string",
                                                                                        "description": "A description of the expected parameter."
                                                                                    }
                                                                                },
                                                                                "required": [
                                                                                    "type",
                                                                                    "description"
                                                                                ]
                                                                            }
                                                                        }
                                                                    },
                                                                    "required": [
                                                                        "type",
                                                                        "properties"
                                                                    ]
                                                                }
                                                            },
                                                            "required": [
                                                                "name",
                                                                "description",
                                                                "parameters"
                                                            ]
                                                        }
                                                    },
                                                    "required": [
                                                        "type",
                                                        "function"
                                                    ]
                                                }
                                            ]
                                        }
                                    },
                                    "response_format": {
                                        "title": "JSON Mode",
                                        "type": "object",
                                        "properties": {
                                            "type": {
                                                "type": "string",
                                                "enum": [
                                                    "json_object",
                                                    "json_schema"
                                                ]
                                            },
                                            "json_schema": {}
                                        }
                                    },
                                    "raw": {
                                        "type": "boolean",
                                        "default": false,
                                        "description": "If true, a chat template is not applied and you must adhere to the specific model's expected formatting."
                                    },
                                    "stream": {
                                        "type": "boolean",
                                        "default": false,
                                        "description": "If true, the response will be streamed back incrementally using SSE, Server Sent Events."
                                    },
                                    "max_tokens": {
                                        "type": "integer",
                                        "default": 256,
                                        "description": "The maximum number of tokens to generate in the response."
                                    },
                                    "temperature": {
                                        "type": "number",
                                        "default": 0.6,
                                        "minimum": 0,
                                        "maximum": 5,
                                        "description": "Controls the randomness of the output; higher values produce more random results."
                                    },
                                    "top_p": {
                                        "type": "number",
                                        "minimum": 0.001,
                                        "maximum": 1,
                                        "description": "Adjusts the creativity of the AI's responses by controlling how many possible words it considers. Lower values make outputs more predictable; higher values allow for more varied and creative responses."
                                    },
                                    "top_k": {
                                        "type": "integer",
                                        "minimum": 1,
                                        "maximum": 50,
                                        "description": "Limits the AI to choose from the top 'k' most probable words. Lower values make responses more focused; higher values introduce more variety and potential surprises."
                                    },
                                    "seed": {
                                        "type": "integer",
                                        "minimum": 1,
                                        "maximum": 9999999999,
                                        "description": "Random seed for reproducibility of the generation."
                                    },
                                    "repetition_penalty": {
                                        "type": "number",
                                        "minimum": 0,
                                        "maximum": 2,
                                        "description": "Penalty for repeated tokens; higher values discourage repetition."
                                    },
                                    "frequency_penalty": {
                                        "type": "number",
                                        "minimum": -2,
                                        "maximum": 2,
                                        "description": "Decreases the likelihood of the model repeating the same lines verbatim."
                                    },
                                    "presence_penalty": {
                                        "type": "number",
                                        "minimum": -2,
                                        "maximum": 2,
                                        "description": "Increases the likelihood of the model introducing new topics."
                                    }
                                },
                                "required": [
                                    "messages"
                                ]
                            }
                        ]
                    }
                }
            },
            "required": [
                "requests"
            ]
        }
    ]
}