ai/tool_using/README_en.md

75 lines
3.1 KiB
Markdown

# Tool Invocation
This document will introduce how to use the ChatGLM3-6B for tool invocation. Currently, only the ChatGLM3-6B model supports tool invocation, while the ChatGLM3-6B-Base and ChatGLM3-6B-32K models do not support it.
## Building System Prompt
Here are two examples of tool invocation. First, prepare the description information of the data to be built.
```python
tools = [
{
"name": "track",
"description": "Track the real-time price of a specified stock",
"parameters": {
"type": "object",
"properties": {
"symbol": {
"description": "The stock code that needs to be tracked"
}
},
"required": ['symbol']
}
},
{
"name": "text-to-speech",
"description": "Convert text to speech",
"parameters": {
"type": "object",
"properties": {
"text": {
"description": "The text that needs to be converted into speech"
},
"voice": {
"description": "The type of voice to use (male, female, etc.)"
},
"speed": {
"description": "The speed of the speech (fast, medium, slow, etc.)"
}
},
"required": ['text']
}
}
]
system_info = {"role": "system", "content": "Answer the following questions as best as you can. You have access to the following tools:", "tools": tools}
```
Please ensure that the definition format of the tool is consistent with the example to obtain optimal performance.
## Asking Questions
Note: Currently, the tool invocation of ChatGLM3-6B only supports the `chat` method and does not support the `stream_chat` method.
```python
history = [system_info]
query = "Help me inquire the price of stock 10111"
response, history = model.chat(tokenizer, query, history=history)
print(response)
```
The expected output here is
```json
{"name": "track", "parameters": {"symbol": "10111"}}
```
This indicates that the model needs to call the tool `track`, and the parameter `symbol` needs to be passed in.
## Invoke Tool, Generate Response
Here, you need to implement the logic of calling the tool yourself. Assuming that the return result has been obtained, return the result to the model in json format and get a response.
```python
result = json.dumps({"price": 12412}, ensure_ascii=False)
response, history = model.chat(tokenizer, result, history=history, role="observation")
print(response)
```
Here `role="observation"` indicates that the input is the return value of the tool invocation rather than user input, and it cannot be omitted.
The expected output is
```
Based on your query, after the API call, the price of stock 10111 is 12412.
```
This indicates that this tool invocation has ended, and the model generates a response based on the return result. For more complex questions, the model may need to make multiple tool invocations. At this time, you can judge whether the returned `response` is `str` or `dict` to determine whether the return is a generated response or a tool invocation request.