多模态输入
本页说明对话接口的多模态输入能力。SilvaMux 对多模态内容透传给模型侧,能否处理取决于模型本身。
支持 OpenAI vision 格式,在 messages 的 content 中传入 image_url:
curl https://www.silvamux.com/api/v1/chat/completions \
-H "Authorization: Bearer $SILVAMUX_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "minimax-m3",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "这张图里有什么?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}}
]
}
]
}'
# pip install openai
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["SILVAMUX_API_KEY"],
base_url="https://www.silvamux.com/api/v1",
)
response = client.chat.completions.create(
model="minimax-m3",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "这张图里有什么?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}},
],
}
],
)
print(response.choices[0].message.content)
// npm install openai
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.SILVAMUX_API_KEY,
baseURL: "https://www.silvamux.com/api/v1",
});
const response = await client.chat.completions.create({
model: "minimax-m3",
messages: [
{
role: "user",
content: [
{ type: "text", text: "这张图里有什么?" },
{ type: "image_url", image_url: { url: "https://example.com/cat.jpg" } },
],
},
],
});
console.log(response.choices[0].message.content);
// 仅依赖标准库
package main
import (
"bytes"
"fmt"
"io"
"net/http"
"os"
)
func main() {
payload := []byte(`{
"model": "minimax-m3",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "这张图里有什么?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}}
]
}
]
}`)
req, err := http.NewRequest("POST", "https://www.silvamux.com/api/v1/chat/completions", bytes.NewReader(payload))
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("SILVAMUX_API_KEY"))
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
fmt.Println(string(body))
}
image_url.url 可以是公网 URL,或 base64 编码(data:image/jpeg;base64,...)。- 平台不解析也不剥离图片内容,原样转发给模型侧。
- 是否支持图片输入由模型决定(如部分多模态模型支持,纯文本模型会报错)。
支持 OpenAI audio 格式,在 content 中传入 input_audio:
{
"type": "input_audio",
"input_audio": {"data": "<base64>", "format": "wav"}
}
- 平台透传,能否处理由模型决定。
- 音频 token 按普通输入 token 计价(无独立多模态单价)。