火山方舟
火山方舟
文档指南
请输入
- 文档首页
火山方舟调用多模态理解图片理解
复制全文
下载 pdf图片理解
文档反馈
问问助手部分大模型具备图片视觉理解能力，支持本地文件和图片 URL 方式传入图片，适用于图片描述、分类、视觉定位等场景。
说明
快速开始
通过图片 URL 方式传入模型快速体验图片理解效果，Responses API 示例代码如下。
输入 | 输出预览 |
支持输入图片的模型系列是哪个？ |
|
curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Authorization: Bearer $ARK_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"input": [
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"
},
{
"type": "input_text",
"text": "支持输入图片的模型系列是哪个？"
}
]
}
]
}'
import os
from arkruntime import Ark
api_key = os.getenv('ARK_API_KEY')
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=api_key,
)
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"
},
{
"type": "input_text",
"text": "Which model series supports image input?"
},
],
}
]
)
for item in response.output:
if item.type == "reasoning":
for summary in item.summary:
if summary.type == "summary_text":
print(f"思考：{summary.text}")
elif item.type == "message":
for content in item.content:
if content.type == "output_text":
print(f"回答：{content.text}")
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
client := arkruntime.NewClientWithApiKey(
// Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
inputMessage := responses.InputItem{
OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleUser),
Content: responses.NewContentItemArrayMessageContent(
[]responses.ContentItem{
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png")})},
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "Which model series supports image input?"})},
},
),
},
),
}
resp, err := client.CreateResponses(ctx, &responses.ResponsesRequest{
Model: "doubao-seed-2-1-pro-260628",
Input: responses.NewInputItemArrayResponsesInput([]responses.InputItem{inputMessage}),
})
if err != nil {
fmt.Printf("response error: %v\n", err)
return
}
fmt.Println(resp)
}
package com.ark.example;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
public class demo {
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
ArkService arkService = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
ResponsesRequest request = ResponsesRequest.builder()
.model("doubao-seed-2-1-pro-260628")
.input(ResponsesInput.ofList(java.util.Arrays.asList(ItemEasyMessage.builder().role(MessageRole.USER).content(
MessageContent.ofList(java.util.Arrays.asList(ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png").build(), ContentItemText.builder().text("Which model series supports image input?").build()))
).build())))
.build();
Response resp = arkService.createResponse(request);
System.out.println(resp);
arkService.shutdownExecutor();
}
}
import os
from openai import OpenAI
api_key = os.getenv('ARK_API_KEY')
client = OpenAI(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=api_key,
)
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"
},
{
"type": "input_text",
"text": "Which model series supports image input?"
},
],
}
]
)
for item in response.output:
if item.type == "reasoning":
for summary in item.summary:
if summary.type == "summary_text":
print(f"思考：{summary.text}")
elif item.type == "message":
for content in item.content:
if content.type == "output_text":
print(f"回答：{content.text}")
支持模型
- 请参见视觉理解能力。
API 接口
- Responses API：支持图片作为输入进行分析。支持 File ID 方式进行图片理解，使用方式参见 Files API 上传（推荐）。
- Chat API：支持图片作为输入进行分析。支持 File ID 方式进行图片理解，使用方式参见 Files API 上传（推荐）。
图片传入方式
支持的图片传入方式如下：
- 本地文件上传：
- Files API 上传（推荐）：通过 Files API 上传图片文件，文件大小不能超过 512 MB。
- Base64 编码传入：适用于图片文件体积较小的场景，单张图片小于 10 MB，请求体不能超过 64 MB。
- 图片 URL 传入：适用于图片文件已存在公网可访问 URL 的场景，单张图片小于 10 MB。
说明
Chat API 是无状态的，如需模型对同一张图片进行多轮理解，则每次请求时都需传入该图片信息。
本地文件上传
Files API 上传（推荐）
建议优先采用文件路径方式上传本地文件，该方式可以支持最大 512MB 文件的处理。通过 Files API 上传图片并待文件状态变为 active 后，Responses API 和 Chat API 均支持通过 file_id 方式传入图片。
以下示例先调用 Files API 上传本地文件并等待处理完成，再通过 File ID 调用 Responses API 或 Chat API 进行图片分析：
- 如果需要实时获取分析内容，或者要规避复杂任务引发的客户端超时失败问题，可采用流式输出的方式，使用方式可参见示例代码。
- 支持直接使用 Files API 上传本地文件，具体请参见文件输入(Files API)。
- Responses API 示例代码：
import os
from pathlib import Path
from arkruntime import Ark
client = Ark(
base_url="https://ark.cn-beijing.volces.com/api/v3",
api_key=os.environ.get("ARK_API_KEY"),
)
file_path = Path(os.getenv("FILE_PATH", "/Users/doc/demo.png"))
uploaded_file = client.files.create(file=file_path, purpose="user_data")
client.files.wait_for_processing(uploaded_file.id)
response = client.responses.create(
model=os.getenv("MODEL_ID", "doubao-seed-2-1-pro-260628"),
input=[
{
"role": "user",
"content": [
{"type": "input_image", "file_id": uploaded_file.id},
{"type": "input_text", "text": "请描述图片的主要内容。"},
],
}
],
)
print(response)
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/file"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
ctx := context.Background()
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
filePath := os.Getenv("FILE_PATH")
if filePath == "" {
filePath = "/Users/doc/demo.png"
}
data, err := os.Open(filePath)
if err != nil {
panic(err)
}
defer data.Close()
uploaded, err := client.UploadFile(ctx, &file.FileCreateRequest{Purpose: file.PurposeUserData}, data)
if err != nil {
panic(err)
}
uploaded, err = client.WaitForFileProcessing(ctx, uploaded.ID, arkruntime.WaitForFileProcessingOptions{})
if err != nil || uploaded.Status != file.StatusActive {
panic(fmt.Sprintf("file processing failed: %v, status=%s", err, uploaded.Status))
}
media := responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(
responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, FileID: responses.NewOptString(uploaded.ID)},
)}
text := responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(
responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "请描述图片的主要内容。"},
)}
message := responses.InputItem{OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleUser),
Content: responses.NewContentItemArrayMessageContent([]responses.ContentItem{media, text}),
},
)}
modelID := os.Getenv("MODEL_ID")
if modelID == "" {
modelID = "doubao-seed-2-1-pro-260628"
}
response, err := client.CreateResponses(ctx, &responses.ResponsesRequest{
Model: modelID,
Input: responses.NewInputItemArrayResponsesInput([]responses.InputItem{message}),
})
if err != nil {
panic(err)
}
fmt.Println(response)
}
import com.volcengine.ark.runtime.models.file.FileCreateRequest;
import com.volcengine.ark.runtime.models.file.FileObject;
import com.volcengine.ark.runtime.models.file.Purpose;
import com.volcengine.ark.runtime.models.file.Status;
import com.volcengine.ark.runtime.models.responses.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.io.File;
import java.util.Arrays;
public class Main {
public static void main(String[] args) {
ArkService service = ArkService.builder()
.apiKey(System.getenv("ARK_API_KEY"))
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
String filePath = System.getenv().getOrDefault("FILE_PATH", "/Users/doc/demo.png");
String modelId = System.getenv().getOrDefault("MODEL_ID", "doubao-seed-2-1-pro-260628");
FileObject uploaded = service.uploadFile(
FileCreateRequest.builder().purpose(Purpose.USER_DATA).build(),
new File(filePath)
);
uploaded = service.waitForFileProcessing(uploaded.getId());
if (uploaded.getStatus() != Status.ACTIVE) {
throw new IllegalStateException("File processing failed: " + uploaded.getStatus());
}
ContentItem media = ContentItemImage.builder()
.type(ContentItemType.INPUT_IMAGE)
.fileId(uploaded.getId())
.build();
ContentItem text = ContentItemText.builder()
.type(ContentItemType.INPUT_TEXT)
.text("请描述图片的主要内容。")
.build();
ResponsesRequest request = ResponsesRequest.builder()
.model(modelId)
.input(ResponsesInput.ofList(Arrays.asList(
ItemEasyMessage.builder()
.role(MessageRole.USER)
.content(MessageContent.ofList(Arrays.asList(media, text)))
.build()
)))
.build();
Response response = service.createResponse(request);
System.out.println(response);
service.shutdownExecutor();
}
}
- Chat API 示例代码：
import os
from pathlib import Path
from arkruntime import Ark
client = Ark(
base_url="https://ark.cn-beijing.volces.com/api/v3",
api_key=os.environ.get("ARK_API_KEY"),
)
file_path = Path(os.getenv("FILE_PATH", "/Users/doc/demo.png"))
uploaded_file = client.files.create(
file=file_path,
purpose="user_data",
)
client.files.wait_for_processing(uploaded_file.id)
completion = client.chat.completions.create(
model=os.getenv("MODEL_ID", "doubao-seed-2-1-pro-260628"),
messages=[
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"file_id": uploaded_file.id}},
{"type": "text", "text": "请描述图片的主要内容。"},
],
}
],
)
print(f"id={completion.id}")
print(completion.choices[0].message.content)
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
"github.com/volcengine/ark-runtime-go/arkruntime/model/file"
)
func main() {
ctx := context.Background()
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
filePath := os.Getenv("FILE_PATH")
if filePath == "" {
filePath = "/Users/doc/demo.png"
}
data, err := os.Open(filePath)
if err != nil {
panic(err)
}
defer data.Close()
uploaded, err := client.UploadFile(ctx, &file.FileCreateRequest{
Purpose: file.PurposeUserData,
}, data)
if err != nil {
panic(err)
}
uploaded, err = client.WaitForFileProcessing(
ctx,
uploaded.ID,
arkruntime.WaitForFileProcessingOptions{},
)
if err != nil || uploaded.Status != file.StatusActive {
panic(fmt.Sprintf("file processing failed: %v, status=%s", err, uploaded.Status))
}
media := chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{FileID: chat.NewOptString(uploaded.ID)})})}
text := chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(
chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "请描述图片的主要内容。"},
)}
message := chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(
chat.ChatCompletionRequestUserMessage{
Role: chat.ChatCompletionRequestUserMessageRoleUser,
Content: chat.NewOptNilChatCompletionMessageContent(chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(
[]chat.ChatCompletionContentPart{media, text},
)),
},
)}
modelID := os.Getenv("MODEL_ID")
if modelID == "" {
modelID = "doubao-seed-2-1-pro-260628"
}
response, err := client.CreateChatCompletion(ctx, &chat.ChatCompletionRequest{
Model: modelID,
Messages: []chat.ChatCompletionRequestMessage{message},
})
if err != nil {
panic(err)
}
fmt.Printf("id=%s\n%s\n", response.ID, response.Choices[0].Message.Content)
}
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.models.file.FileCreateRequest;
import com.volcengine.ark.runtime.models.file.FileObject;
import com.volcengine.ark.runtime.models.file.PreprocessConfigs;
import com.volcengine.ark.runtime.models.file.Purpose;
import com.volcengine.ark.runtime.models.file.Status;
import com.volcengine.ark.runtime.models.file.Video;
import com.volcengine.ark.runtime.service.ArkService;
import java.io.File;
import java.util.Arrays;
public class Main {
public static void main(String[] args) {
ArkService service = ArkService.builder()
.apiKey(System.getenv("ARK_API_KEY"))
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
String filePath = System.getenv().getOrDefault("FILE_PATH", "/Users/doc/demo.png");
String modelId = System.getenv().getOrDefault("MODEL_ID", "doubao-seed-2-1-pro-260628");
FileObject uploaded = service.uploadFile(
FileCreateRequest.builder()
.purpose(Purpose.USER_DATA)
.build(),
new File(filePath)
);
uploaded = service.waitForFileProcessing(uploaded.getId());
if (uploaded.getStatus() != Status.ACTIVE) {
throw new IllegalStateException("File processing failed: " + uploaded.getStatus());
}
ChatCompletionContentPart media = ChatCompletionContentPartImage.builder().type(ChatCompletionContentPartType.IMAGE_URL).imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().fileId(uploaded.getId()).build())).build();
ChatCompletionContentPart text = ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("请描述图片的主要内容。")
.build();
ChatCompletionRequestMessage message = ChatCompletionRequestUserMessage.builder()
.role(ChatCompletionRequestMessageType.USER)
.content(ChatCompletionMessageContent.builder()
.value(Arrays.asList(media, text))
.build())
.build();
ChatCompletionRequest request = ChatCompletionRequest.builder()
.model(modelId)
.messages(Arrays.asList(message))
.build();
ChatCompletionResponse response = service.createChatCompletion(request);
System.out.println("id=" + response.getId());
System.out.println(response.getChoices().get(0).getMessage().getContent());
service.shutdownExecutor();
}
}
Base64 编码传入
将本地文件转换为 Base64 编码字符串，然后提交给大模型。该方式适用于图片文件体积较小的情况，单张图片小于 10 MB，请求体不能超过 64MB。（Responses API 和 Chat API 都支持该方式。）
注意
将图片文件转换为Base64编码字符串，然后遵循data:{mime_type};base64,{base64_data}格式拼接，传入模型。
- {mime_type}：文件的媒体类型，需要与文件格式mime_type对应。支持的图片格式详细见图片格式说明。
- {base64_data}：文件经过Base64编码后的字符串。
... model="doubao-seed-2-1-pro-260628", messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": f"data:image/png;base64,{base64_image}" } }, { "type": "text", "text": "Which model series supports image input?" } ] } ] ... | ... model="doubao-seed-2-1-pro-260628", input=[ { "role": "user", "content": [ { "type": "input_image", "image_url": f"data:image/png;base64,{base64_image}" }, { "type": "input_text", "text": "Which model series supports image input?" } ] } ] ... |
- Responses API 示例代码：
BASE64_IMAGE=$(base64 < demo.png) && curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d @- <<EOF
{
"model": "doubao-seed-2-1-pro-260628",
"input": [
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "data:image/png;base64,$BASE64_IMAGE"
},
{
"type": "input_text",
"text": "Which model series supports image input?"
}
]
}
]
}
EOF
import os
from arkruntime import Ark
import base64
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key = os.getenv('ARK_API_KEY')
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=api_key,
)
# Convert local files to Base64-encoded strings.
def encode_file(file_path):
with open(file_path, "rb") as read_file:
return base64.b64encode(read_file.read()).decode('utf-8')
base64_file = encode_file("/Users/doc/demo.png") # 替换为您本地图片的实际路径
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": f"data:image/png;base64,{base64_file}"
},
{
"type": "input_text",
"text": "Which model series supports image input?"
},
],
}
]
)
print(response)
package main
import (
"context"
"encoding/base64"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
// Convert local files to Base64-encoded strings.
fileBytes, err := os.ReadFile("/Users/doc/demo.png") // 替换为您本地图片的实际路径
if err != nil {
fmt.Printf("read file error: %v\n", err)
return
}
base64File := base64.StdEncoding.EncodeToString(fileBytes)
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
inputMessage := responses.InputItem{
OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleUser),
Content: responses.NewContentItemArrayMessageContent(
[]responses.ContentItem{
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString(fmt.Sprintf("data:image/png;base64,%s", base64File))})},
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "Which model series supports image input?"})},
},
),
},
),
}
resp, err := client.CreateResponses(ctx, &responses.ResponsesRequest{
Model: "doubao-seed-2-1-pro-260628",
Input: responses.NewInputItemArrayResponsesInput([]responses.InputItem{inputMessage}),
})
if err != nil {
fmt.Printf("response error: %v\n", err)
return
}
fmt.Println(resp)
}
package com.ark.sample;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Base64;
import java.io.IOException;
public class demo {
private static String encodeFile(String filePath) throws IOException {
byte[] fileBytes = Files.readAllBytes(Paths.get(filePath));
return Base64.getEncoder().encodeToString(fileBytes);
}
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
ArkService arkService = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
// Convert local files to Base64-encoded strings.
String base64Data = "";
try {
base64Data = "data:image/png;base64," + encodeFile("/Users/demo.png");
} catch (IOException e) {
System.err.println("encode error: " + e.getMessage());
}
ResponsesRequest request = ResponsesRequest.builder()
.model("doubao-seed-2-1-pro-260628")
.input(ResponsesInput.ofList(java.util.Arrays.asList(ItemEasyMessage.builder().role(MessageRole.USER).content(
MessageContent.ofList(java.util.Arrays.asList(ContentItemImage.builder().imageUrl(base64Data).build(), ContentItemText.builder().text("Which model series supports image input?").build()))
).build())))
.build();
Response resp = arkService.createResponse(request);
System.out.println(resp);
arkService.shutdownExecutor();
}
}
import os
from openai import OpenAI
import base64
api_key = os.getenv('ARK_API_KEY')
client = OpenAI(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=api_key,
)
# Convert local files to Base64-encoded strings.
def encode_file(file_path):
with open(file_path, "rb") as read_file:
return base64.b64encode(read_file.read()).decode('utf-8')
base64_file = encode_file("/Users/doc/demo.png") # 替换为您本地图片的实际路径
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": f"data:image/png;base64,{base64_file}",
},
{
"type": "input_text",
"text": "Which model series supports image input?"
},
],
}
]
)
print(response)
- Chat API 示例代码：
BASE64_IMAGE=$(base64 < demo.png) && curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d @- <<EOF
{
"model": "doubao-seed-2-1-pro-260628",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,$BASE64_IMAGE"
}
},
{
"type": "text",
"text": "Which model series supports image input?"
}
]
}
],
"max_tokens": 300
}
EOF
import base64
import os
# Install SDK: pip install arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
# 定义方法将指定路径图片转为Base64编码
def encode_image(image_path):
with open(image_path, "rb") as image_file:
return base64.b64encode(image_file.read()).decode('utf-8')
# 需传给大模型的图片
image_path = "demo.png"
# 将图片转为Base64编码
base64_image = encode_image(image_path)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260628",
messages=[
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
# 需注意：传入Base64编码遵循格式 data:image/<IMAGE_FORMAT>;base64,{base64_image}：
# PNG图片："url": f"data:image/png;base64,{base64_image}"
# JPEG图片："url": f"data:image/jpeg;base64,{base64_image}"
# WEBP图片："url": f"data:image/webp;base64,{base64_image}"
"url": f"data:image/<IMAGE_FORMAT>;base64,{base64_image}"
},
},
{
"type": "text",
"text": "Which model series supports image input?",
},
],
}
],
)
print(completion.choices[0])
package main
import (
"context"
"encoding/base64"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
)
func main() {
// 读取本地图片文件
imageBytes, err := os.ReadFile("demo.png") // 替换为实际图片路径
if err != nil {
fmt.Printf("读取图片失败: %v\n", err)
return
}
base64Image := base64.StdEncoding.EncodeToString(imageBytes)
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
// The base URL for model invocation .
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
req := &chat.ChatCompletionRequest{
// Replace with Model ID
Model: "doubao-seed-2-1-pro-260628",
Messages: []chat.ChatCompletionRequestMessage{
chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(chat.ChatCompletionRequestUserMessage{Role: chat.ChatCompletionRequestUserMessageRoleUser, Content: chat.NewOptNilChatCompletionMessageContent(chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(
[]chat.ChatCompletionContentPart{
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString(fmt.Sprintf("data:image/png;base64,%s", base64Image))})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "Which model series supports image input?"})},
},
))})},
},
}
resp, err := client.CreateChatCompletion(ctx, req)
if err != nil {
fmt.Printf("standard chat error: %v\n", err)
return
}
fmt.Println(resp.Choices[0].Message.Content)
}
package com.ark.sample;
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.models.chat.ChatCompletionContentPart.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import okhttp3.ConnectionPool;
import okhttp3.Dispatcher;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Base64;
import java.io.IOException;
public class Sample {
static String apiKey = System.getenv("ARK_API_KEY");
static ConnectionPool connectionPool = new ConnectionPool(5, 1, TimeUnit.SECONDS);
static Dispatcher dispatcher = new Dispatcher();
static ArkService service = ArkService.builder()
.dispatcher(dispatcher)
.connectionPool(connectionPool)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
// The base URL for model invocation .
.apiKey(apiKey)
.build();
// Base64编码方法
private static String encodeImage(String imagePath) throws IOException {
byte[] imageBytes = Files.readAllBytes(java.nio.file.Paths.get(imagePath));
return Base64.getEncoder().encodeToString(imageBytes);
}
public static void main(String[] args) throws Exception {
List<ChatCompletionRequestMessage> messagesForReqList = new ArrayList<>();
// 本地图片路径（替换为实际路径）
String imagePath = "demo.png";
// 生成Base64数据URL
String base64Data = "data:image/png;base64," + encodeImage(imagePath);
// 构建消息内容（修复内容部分构建方式）
List<ChatCompletionContentPart> contentParts = new ArrayList<>();
// 图片部分使用builder模式
contentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url(base64Data).build()))
.build());
// 文本部分使用builder模式
contentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("Which model series supports image input?")
.build());
// 创建消息
messagesForReqList.add(ChatCompletionRequestUserMessage.builder()
.role(ChatCompletionRequestMessageType.USER)
.content(ChatCompletionMessageContent.ofList(contentParts))
.build());
ChatCompletionRequest req = ChatCompletionRequest.builder()
.model("doubao-seed-2-1-pro-260628") //Replace with Model ID .
.messages(messagesForReqList)
.maxTokens(300)
.build();
service.createChatCompletion(req)
.getChoices()
.forEach(choice -> System.out.println(choice.getMessage().getContent()));
// shutdown service after all requests are finished
service.shutdownExecutor();
}
}
图片 URL 传入
如果图片已存在公网可访问URL，可以在请求中直接填入图片的公网URL，单张图片不能超过 10 MB。（Responses API 和 Chat API 都支持该方式。）
说明
如果使用 URL，建议使用火山引擎TOS（对象存储）存储图片并生成访问链接，不仅能保证图片的稳定存储，还能利用方舟与TOS的内网通信优势，有效降低模型回复的时延和公网流量费用。
... model="doubao-seed-2-1-pro-260628", messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png" } }, { "type": "text", "text": "Which model series supports image input?" } ] } ] ... | ... model="doubao-seed-2-1-pro-260628", input=[ { "role": "user", "content": [ { "type": "input_image", "image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png" }, { "type": "input_text", "text": "Which model series supports image input?" } ] } ] ... |
- Responses API 示例代码：[快速开始](https://ark.volcengine.com/region:cn-beijing/docs/ark/image-understanding#18cf565a){target="_self"}
- Chat API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"messages": [
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"}},
{"type": "text", "text": "Which model series supports image input?"}
]
}
],
"max_tokens": 300
}'
import os
# Install SDK: pip install arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260628",
messages=[
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"}},
{"type": "text", "text": "Which model series supports image input?"},
],
}
],
)
print(completion.choices[0])
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
)
func main() {
client := arkruntime.NewClientWithApiKey(
//Use os.Getenv to get ARK_API_KEY
os.Getenv("ARK_API_KEY"),
// The base URL for model invocation
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
// Create a context background
ctx := context.Background()
// Construct the content of the message
contentParts := []chat.ChatCompletionContentPart{
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png")})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "Which model series supports image input?"})},
}
// Construct chat, specify model and message
req := &chat.ChatCompletionRequest{
// Replace with Model ID
Model: "doubao-seed-2-1-pro-260628",
Messages: []chat.ChatCompletionRequestMessage{
chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(chat.ChatCompletionRequestUserMessage{Role: chat.ChatCompletionRequestUserMessageRoleUser, Content: chat.NewOptNilChatCompletionMessageContent(chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(contentParts))})},
},
}
// Send chat, store result in resp and any possible error in err
resp, err := client.CreateChatCompletion(ctx, req)
if err!= nil {
fmt.Printf("standard chat error: %v\n", err)
return
}
// Print response
fmt.Println(resp.Choices[0].Message.Content)
}
package com.ark.sample;
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.models.chat.ChatCompletionContentPart.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import okhttp3.ConnectionPool;
import okhttp3.Dispatcher;
public class MultiImageSample {
static String apiKey = System.getenv("ARK_API_KEY");
static ConnectionPool connectionPool = new ConnectionPool(5, 1, TimeUnit.SECONDS);
static Dispatcher dispatcher = new Dispatcher();
static ArkService service = ArkService.builder()
.dispatcher(dispatcher)
.connectionPool(connectionPool)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
// The base URL for model invocation .
.apiKey(apiKey)
.build();
public static void main(String[] args) throws Exception {
List<ChatCompletionRequestMessage> messagesForReqList = new ArrayList<>();
// Construct the content of the message
List<ChatCompletionContentPart> contentParts = new ArrayList<>();
// Use builder mode for the image
contentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png").build()))
.build());
// Use builder mode for text
contentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("Which model series supports image input?")
.build());
// Create message
messagesForReqList.add(ChatCompletionRequestUserMessage.builder()
.role(ChatCompletionRequestMessageType.USER)
.content(ChatCompletionMessageContent.ofList(contentParts))
.build());
ChatCompletionRequest req = ChatCompletionRequest.builder()
.model("doubao-seed-2-1-pro-260628") //Replace with Model ID .
.messages(messagesForReqList)
.build();
service.createChatCompletion(req)
.getChoices()
.forEach(choice -> System.out.println(choice.getMessage().getContent()));
// shutdown service after all requests are finished
service.shutdownExecutor();
}
}
使用场景
多图输入
API 可支持接受和处理多个图像输入，这些图像可通过图片可访问 URL 或图片转为 Base64 编码后输入，模型将结合所有传入的图像中的信息来回答问题。
- Responses API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Authorization: Bearer $ARK_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"input": [
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png"
},
{
"type": "input_text",
"text": "支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？"
}
]
}
]
}'
import os
from arkruntime import Ark
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=os.getenv('ARK_API_KEY')
)
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png"
},
{
"type": "input_text",
"text": "支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？"
}
]
}
]
)
print(response.output)
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
inputMessage := responses.InputItem{
OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleUser),
Content: responses.NewContentItemArrayMessageContent(
[]responses.ContentItem{
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png")})},
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png")})},
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？"})},
},
),
},
),
}
resp, err := client.CreateResponses(ctx, &responses.ResponsesRequest{
Model: "doubao-seed-2-1-pro-260628",
Input: responses.NewInputItemArrayResponsesInput([]responses.InputItem{inputMessage}),
})
if err != nil {
fmt.Printf("response error: %v\n", err)
return
}
fmt.Println(resp)
}
package com.ark.sample;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
public class demo {
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
ArkService arkService = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
ResponsesRequest request = ResponsesRequest.builder()
.model("doubao-seed-2-1-pro-260628")
.input(ResponsesInput.ofList(java.util.Arrays.asList(ItemEasyMessage.builder().role(MessageRole.USER).content(
MessageContent.ofList(java.util.Arrays.asList(ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png").build(), ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png").build(), ContentItemText.builder().text("支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？").build()))
).build())))
.build();
Response resp = arkService.createResponse(request);
System.out.println(resp);
arkService.shutdownExecutor();
}
}
- Chat API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"messages": [
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"}},
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png"}},
{"type": "text", "text": "支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？"}
]
}
],
"max_tokens": 300
}'
import os
# Install SDK: pip install arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260628",
messages=[
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"}},
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png"}},
{"type": "text", "text": "支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？"},
],
}
],
)
print(completion.choices[0])
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
)
func main() {
client := arkruntime.NewClientWithApiKey(
//Use os.Getenv to get ARK_API_KEY
os.Getenv("ARK_API_KEY"),
// The base URL for model invocation
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
// Create context background
ctx := context.Background()
// Construct message, including 2 images and a text
contentParts := []chat.ChatCompletionContentPart{
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png")})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png")})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？"})},
}
// Construct chat request
req := &chat.ChatCompletionRequest{
// Replace with Model ID
Model: "doubao-seed-2-1-pro-260628",
Messages: []chat.ChatCompletionRequestMessage{
chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(chat.ChatCompletionRequestUserMessage{Role: chat.ChatCompletionRequestUserMessageRoleUser, Content: chat.NewOptNilChatCompletionMessageContent(chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(contentParts))})},
},
MaxTokens: chat.NewOptInt32(int32(300)), // Set max output token count
}
// Send the chat completion request
resp, err := client.CreateChatCompletion(ctx, req)
if err!= nil {
fmt.Printf("standard chat error: %v\n", err)
return
}
// Print response
fmt.Println(resp.Choices[0].Message.Content)
}
package com.ark.sample;
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.models.chat.ChatCompletionContentPart.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import okhttp3.ConnectionPool;
import okhttp3.Dispatcher;
public class MultiImageSample {
static String apiKey = System.getenv("ARK_API_KEY");
static ConnectionPool connectionPool = new ConnectionPool(5, 1, TimeUnit.SECONDS);
static Dispatcher dispatcher = new Dispatcher();
static ArkService service = ArkService.builder()
.dispatcher(dispatcher)
.connectionPool(connectionPool)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
// The base URL for model invocation
.apiKey(apiKey)
.build();
public static void main(String[] args) throws Exception {
List<ChatCompletionRequestMessage> messagesForReqList = new ArrayList<>();
// Construct content of the message
List<ChatCompletionContentPart> contentParts = new ArrayList<>();
// Use builder mode for the first image
contentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png").build()))
.build());
//
contentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_2.png").build()))
.build());
contentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("支持输入图片的模型系列是哪个？同时，豆包应用场景有哪些？")
.build());
messagesForReqList.add(ChatCompletionRequestUserMessage.builder()
.role(ChatCompletionRequestMessageType.USER)
.content(ChatCompletionMessageContent.ofList(contentParts))
.build());
ChatCompletionRequest req = ChatCompletionRequest.builder()
.model("doubao-seed-2-1-pro-260628") //Replace with Model ID
.messages(messagesForReqList)
.maxTokens(300)
.build();
service.createChatCompletion(req)
.getChoices()
.forEach(choice -> System.out.println(choice.getMessage().getContent()));
// shutdown service after all requests are finished
service.shutdownExecutor();
}
}
控制图片理解的精细度
控制图片理解的精细度（指对画面的精细）： image_pixel_limit 、detail 字段，2个字段若同时配置，则生效逻辑如下：
- 生效前提：图片像素范围在 [196, 36,000,000] px，否则直接报错。
- 生效优先级：image_pixel_limit 高于 detail 字段，即同时配置 detail 与 image_pixel_limit 字段时，生效 image_pixel_limit 字段配置。
- 缺省时生效：image_pixel_limit 字段的 min_pixels / max_pixels 字段未设置，则使用 detail 默认值配置所对应的值。具体范围参见通过 detail 字段（图片理解）。
下面分别介绍如何通过 detail 、 image_pixel_limit 控制视觉理解的精度。
通过 detail 字段（图片理解）
通过detail参数来控制模型理解图片的精细度， 不同模型支持的 detail 模式、token 用量、图片像素区间如下：
方式一：
说明
doubao-seed-2.0 及以后模型 detail 默认值为 high，单图固定 1280 个 tokens，在不牺牲效果的同时消耗的 tokens 更少。
detail模式 | doubao-seed-2.0 及以后模型 detail 默认值为 high | |
| 单图token范围 | 图片像素区间 |
low | [1, 1280] | [1764, 2257920] |
high | 1280 | 2257920 |
xhigh | [1280, 5120] | [2257920, 9031680] |
- detail 为 low 时，图片处理速度会提高，适合图片本身细节较少或者只需模型理解图片大致信息或者对速度有要求的场景。
- detail 为 high 或 xhigh 时，模型可感知图片更多的细节，但是图片处理速度会降低，适合图像像素值高且需关注细节信息的场景，如街道地图分析等。
图片缩放规则：不在指定模式对应的图片像素区间时，方舟会等比例缩放至范围内。
方式二：
DeepSeek-v4.1-flash 模型在图片理解支持的 detail 模式及缩放规则如下：
DeepSeek-V4.1-Flash 模型在图片理解的使用限制如下：
使用限制 | 要求 |
单图大小 | 50 MB |
总请求体大小 | 64 MB |
单请求中图片张数 | 3600 张 |
单个请求中通过 URL 方式传入图片文件的总大小 | 200 MB |
代码示例：
- Responses API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Authorization: Bearer $ARK_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"input": [
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png",
"detail": "high"
},
{
"type": "input_text",
"text": "Which model series supports image input?"
}
]
}
]
}'
import os
from arkruntime import Ark
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=os.getenv('ARK_API_KEY')
)
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png",
"detail": "high"
},
{
"type": "input_text",
"text": "Which model series supports image input?"
}
]
}
]
)
print(response.output)
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
inputMessage := responses.InputItem{
OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleUser),
Content: responses.NewContentItemArrayMessageContent(
[]responses.ContentItem{
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"), Detail: responses.NewOptImageDetail(responses.ImageDetailHigh)})},
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "Which model series supports image input?"})},
},
),
},
),
}
resp, err := client.CreateResponses(ctx, &responses.ResponsesRequest{
Model: "doubao-seed-2-1-pro-260628",
Input: responses.NewInputItemArrayResponsesInput([]responses.InputItem{inputMessage}),
})
if err != nil {
fmt.Printf("response error: %v\n", err)
return
}
fmt.Println(resp)
}
package com.ark.sample;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
public class demo {
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
ArkService arkService = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
ResponsesRequest request = ResponsesRequest.builder()
.model("doubao-seed-2-1-pro-260628")
.input(ResponsesInput.ofList(java.util.Arrays.asList(ItemEasyMessage.builder().role(MessageRole.USER).content(
MessageContent.ofList(java.util.Arrays.asList(ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png").detail(ImageDetail.HIGH).build(), ContentItemText.builder().text("Which model series supports image input?").build()))
).build())))
.build();
Response resp = arkService.createResponse(request);
System.out.println(resp);
arkService.shutdownExecutor();
}
}
- Chat API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"messages": [
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png","detail": "high"}},
{"type": "text", "text": "Which model series supports image input?"}
]
}
]
}'
import os
# Install SDK: pip install arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260628",
messages=[
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png","detail": "high"}},
{"type": "text", "text": "Which model series supports image input?"},
],
}
],
)
print(completion.choices[0])
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
)
func main() {
client := arkruntime.NewClientWithApiKey(
//Use os.Getenv to get ARK_API_KEY
os.Getenv("ARK_API_KEY"),
// The base URL for model invocation
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
// Construct a context background
ctx := context.Background()
// Message content
contentParts := []chat.ChatCompletionContentPart{
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png"), Detail: chat.NewOptImageDetail(chat.ImageDetailHigh)})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "Which model series supports image input?"})},
}
req := &chat.ChatCompletionRequest{
// Replace with Model ID
Model: "doubao-seed-2-1-pro-260628",
Messages: []chat.ChatCompletionRequestMessage{
chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(chat.ChatCompletionRequestUserMessage{Role: chat.ChatCompletionRequestUserMessageRoleUser, Content: chat.NewOptNilChatCompletionMessageContent(chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(contentParts))})},
},
MaxTokens: chat.NewOptInt32(int32(300)), // Max output token
}
resp, err := client.CreateChatCompletion(ctx, req)
if err!= nil {
fmt.Printf("standard chat error: %v\n", err)
return
}
fmt.Println(resp.Choices[0].Message.Content)
}
package com.ark.sample;
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.models.chat.ChatCompletionContentPart.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import okhttp3.ConnectionPool;
import okhttp3.Dispatcher;
public class MultiImageSample {
static String apiKey = System.getenv("ARK_API_KEY");
static ConnectionPool connectionPool = new ConnectionPool(5, 1, TimeUnit.SECONDS);
static Dispatcher dispatcher = new Dispatcher();
static ArkService service = ArkService.builder()
.dispatcher(dispatcher)
.connectionPool(connectionPool)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
// The base URL for model invocation .
.apiKey(apiKey)
.build();
public static void main(String[] args) throws Exception {
List<ChatCompletionRequestMessage> messagesForReqList = new ArrayList<>();
List<ChatCompletionContentPart> contentParts = new ArrayList<>();
contentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png").detail(ImageDetail.fromValue("high")).build()))
.build());
contentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("Which model series supports image input?")
.build());
messagesForReqList.add(ChatCompletionRequestUserMessage.builder()
.role(ChatCompletionRequestMessageType.USER)
.content(ChatCompletionMessageContent.ofList(contentParts))
.build());
ChatCompletionRequest req = ChatCompletionRequest.builder()
.model("doubao-seed-2-1-pro-260628") //Replace with Model ID .
.messages(messagesForReqList)
.maxTokens(300)
.build();
service.createChatCompletion(req)
.getChoices()
.forEach(choice -> System.out.println(choice.getMessage().getContent()));
// shutdown service after all requests are finished
service.shutdownExecutor();
}
}
通过 image_pixel_limit 结构体
控制传入给方舟的图像像素大小范围，如果不在此范围，则会等比例放大或者缩小至该范围内，后传给模型进行理解。你可通过 image_pixel_limit 结构体，精细控制模型可理解的图片像素多少。
对应结构体如下：
说明
max_pixels 为图片最大像素，min_pixels 为图片最小像素。
...
"image_pixel_limit": {
"max_pixels": 3014080,
"min_pixels": 3136
}
...
示例代码如下：
Java SDK、 Go SDK 不支持此字段。
- Responses API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Authorization: Bearer $ARK_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"input": [
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png",
"image_pixel_limit": {
"max_pixels": 3014080,
"min_pixels": 3136
}
},
{
"type": "input_text",
"text": "Which model series supports image input?"
}
]
}
]
}'
import os
from arkruntime import Ark
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=os.getenv('ARK_API_KEY')
)
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png",
"image_pixel_limit": {
"max_pixels": 3014080,
"min_pixels": 3136,
}
},
{
"type": "input_text",
"text": "Which model series supports image input?"
}
]
}
]
)
print(response.output)
- Chat API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"messages": [
{
"role": "user",
"content": [
{"type": "image_url","image_url": {"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png","image_pixel_limit": {"max_pixels": 3014080,"min_pixels": 3136}}},
{"type": "text", "text": "Which model series supports image input?"}
]
}
],
"max_tokens": 300
}'
import os
# Install SDK: pip install arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260628",
messages=[
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/ark_demo_img_1.png",
"image_pixel_limit": {
"max_pixels": 3014080,
"min_pixels": 3136,
},
},
},
{"type": "text", "text": "Which model series supports image input?"},
],
}
],
)
print(completion.choices[0])
图文混排
支持灵活地传入提示词和图片信息的方式，你可任意调整传入图片和文本的顺序，以及在system message或者User message传入图文信息。模型会根据顺序返回处理信息的结果，示例如下。
说明
图文混排场景，图文顺序可能影响模型输出效果，若结果不符预期，可调整顺序。当多图+一段文字时，建议将文字放在图片之后。
- Responses API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/responses \
-H "Authorization: Bearer $ARK_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"input": [
{
"role": "system",
"content": [
{
"type": "input_text",
"text": "下面人物是目标人物"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png"
},
{
"type": "input_text",
"text": "请确认下面图片中是否含有目标人物"
}
]
},
{
"role": "user",
"content": [
{
"type": "input_text",
"text": "图片1中是否含有目标人物"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png"
},
{
"type": "input_text",
"text": "图片2中是否含有目标人物"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png"
}
]
}
]
}'
import os
from arkruntime import Ark
client = Ark(
base_url='https://ark.cn-beijing.volces.com/api/v3',
api_key=os.getenv('ARK_API_KEY')
)
response = client.responses.create(
model="doubao-seed-2-1-pro-260628",
input=[
{
"role": "system",
"content": [
{
"type": "input_text",
"text": "下面人物是目标人物"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png"
},
{
"type": "input_text",
"text": "请确认下面图片中是否含有目标人物"
}
]
},
{
"role": "user",
"content": [
{
"type": "input_text",
"text": "图片1中是否含有目标人物"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png"
},
{
"type": "input_text",
"text": "图片2中是否含有目标人物"
},
{
"type": "input_image",
"image_url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png"
}
]
}
]
)
print(response.output)
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/responses"
)
func main() {
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
ctx := context.Background()
systemMessage := responses.InputItem{
OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleSystem),
Content: responses.NewContentItemArrayMessageContent(
[]responses.ContentItem{
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "下面人物是目标人物"})},
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png")})},
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "请确认下面图片中是否含有目标人物"})},
},
),
},
),
}
userMessage := responses.InputItem{
OneOf: responses.NewItemEasyMessageInputItemSum(
responses.ItemEasyMessage{
Role: responses.NewOptMessageRole(responses.MessageRoleUser),
Content: responses.NewContentItemArrayMessageContent(
[]responses.ContentItem{
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "图片1中是否含有目标人物"})},
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png")})},
responses.ContentItem{OneOf: responses.NewContentItemTextContentItemSum(responses.ContentItemText{Type: responses.ContentItemTextTypeInputText, Text: "图片2中是否含有目标人物"})},
responses.ContentItem{OneOf: responses.NewContentItemImageContentItemSum(responses.ContentItemImage{Type: responses.ContentItemImageTypeInputImage, ImageURL: responses.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png")})},
},
),
},
),
}
resp, err := client.CreateResponses(ctx, &responses.ResponsesRequest{
Model: "doubao-seed-2-1-pro-260628",
Input: responses.NewInputItemArrayResponsesInput([]responses.InputItem{systemMessage, userMessage}),
})
if err != nil {
fmt.Printf("response error: %v\n", err)
return
}
fmt.Println(resp)
}
package com.ark.sample;
import com.volcengine.ark.runtime.service.ArkService;
import com.volcengine.ark.runtime.models.responses.*;
public class demo {
public static void main(String[] args) {
String apiKey = System.getenv("ARK_API_KEY");
ArkService arkService = ArkService.builder().apiKey(apiKey)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
.build();
ResponsesRequest request = ResponsesRequest.builder()
.model("doubao-seed-2-1-pro-260628")
.input(ResponsesInput.ofList(java.util.Arrays.asList(ItemEasyMessage.builder().role(MessageRole.SYSTEM).content(
MessageContent.ofList(java.util.Arrays.asList(ContentItemText.builder().text("下面人物是目标人物").build(), ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png").build(), ContentItemText.builder().text("请确认下面图片中是否含有目标人物").build()))
).build(), ItemEasyMessage.builder().role(MessageRole.USER).content(
MessageContent.ofList(java.util.Arrays.asList(ContentItemText.builder().text("图片1中是否含有目标人物").build(), ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png").build(), ContentItemText.builder().text("图片2中是否含有目标人物").build(), ContentItemImage.builder().imageUrl("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png").build()))
).build()))
).build();
Response resp = arkService.createResponse(request);
System.out.println(resp);
arkService.shutdownExecutor();
}
}
- Chat API 示例代码：
curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ARK_API_KEY" \
-d '{
"model": "doubao-seed-2-1-pro-260628",
"messages": [
{
"role": "system",
"content": [
{"type": "text", "text": "下面人物是目标人物"},
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png"
}
},
{"type": "text", "text": "请确认下面图片中是否含有目标人物"}
]
},
{
"role": "user",
"content": [
{"type": "text", "text": "图片1中是否含有目标人物"},
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png"
}
},
{"type": "text", "text": "图片2中是否含有目标人物"},
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png"
}
}
]
}
],
"max_tokens": 300
}'
import os
# Install SDK: pip install arkruntime
from arkruntime import Ark
client = Ark(
# The base URL for model invocation
base_url="https://ark.cn-beijing.volces.com/api/v3",
# Get API Key: https://ark.volcengine.com/region:cn-beijing/apikey
api_key=os.getenv('ARK_API_KEY'),
)
completion = client.chat.completions.create(
# Replace with Model ID
model = "doubao-seed-2-1-pro-260628",
messages=[
{
"role": "system",
"content": [
{"type": "text", "text": "下面人物是目标人物"},
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png"
},
},
{"type": "text", "text": "请确认下面图片中是否含有目标人物"},
],
},
{
"role": "user",
"content": [
{"type": "text", "text": "图片1中是否含有目标人物"},
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png"
},
},
{"type": "text", "text": "图片2中是否含有目标人物"},
{
"type": "image_url",
"image_url": {
"url": "https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png"
},
},
],
},
],
)
print(completion.choices[0].message.content)
package main
import (
"context"
"fmt"
"os"
"github.com/volcengine/ark-runtime-go/arkruntime"
"github.com/volcengine/ark-runtime-go/arkruntime/model/chat"
)
func main() {
client := arkruntime.NewClientWithApiKey(
os.Getenv("ARK_API_KEY"),
// The base URL for model invocation
arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
)
// Create a context, typically used to pass request context information, such as timeouts and cancellations
ctx := context.Background()
// Build the system message content
systemContentParts := []chat.ChatCompletionContentPart{
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "下面人物是目标人物"})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png")})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "请确认下面图片中是否含有目标人物"})},
}
// Build the user message content
userContentParts := []chat.ChatCompletionContentPart{
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "图片1中是否含有目标人物"})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png")})})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartTextChatCompletionContentPartSum(chat.ChatCompletionContentPartText{Type: chat.ChatCompletionContentPartTextTypeText, Text: "图片2中是否含有目标人物"})},
chat.ChatCompletionContentPart{OneOf: chat.NewChatCompletionContentPartImageChatCompletionContentPartSum(chat.ChatCompletionContentPartImage{Type: chat.ChatCompletionContentPartImageTypeImageURL, ImageURL: chat.NewChatCompletionContentPartImageImageUrlChatCompletionContentPartImageImageURL(chat.ChatCompletionContentPartImageImageUrl{URL: chat.NewOptString("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png")})})},
}
// Build a chat completion request and set the model and message content
req := &chat.ChatCompletionRequest{
// Replace with Model ID
Model: "doubao-seed-2-1-pro-260628",
Messages: []chat.ChatCompletionRequestMessage{
chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestSystemMessageChatCompletionRequestMessageSum(chat.ChatCompletionRequestSystemMessage{Role: chat.ChatCompletionRequestSystemMessageRoleSystem, Content: chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(systemContentParts)})},
chat.ChatCompletionRequestMessage{OneOf: chat.NewChatCompletionRequestUserMessageChatCompletionRequestMessageSum(chat.ChatCompletionRequestUserMessage{Role: chat.ChatCompletionRequestUserMessageRoleUser, Content: chat.NewOptNilChatCompletionMessageContent(chat.NewChatCompletionContentPartArrayChatCompletionMessageContent(userContentParts))})},
},
MaxTokens: chat.NewOptInt32(int32(300)),
}
// Send the chat completion request, store the result in resp, and store any possible errors in err
resp, err := client.CreateChatCompletion(ctx, req)
if err!= nil {
fmt.Printf("standard chat error: %v\n", err)
return
}
fmt.Println(resp.Choices[0].Message.Content)
}
package com.ark.sample;
import com.volcengine.ark.runtime.models.chat.*;
import com.volcengine.ark.runtime.models.chat.ChatCompletionContentPart.*;
import com.volcengine.ark.runtime.service.ArkService;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import okhttp3.ConnectionPool;
import okhttp3.Dispatcher;
public class MultiImageSample {
static String apiKey = System.getenv("ARK_API_KEY");
static ConnectionPool connectionPool = new ConnectionPool(5, 1, TimeUnit.SECONDS);
static Dispatcher dispatcher = new Dispatcher();
static ArkService service = ArkService.builder()
.dispatcher(dispatcher)
.connectionPool(connectionPool)
.baseUrl("https://ark.cn-beijing.volces.com/api/v3")
// The base URL for model invocation
.apiKey(apiKey)
.build();
public static void main(String[] args) throws Exception {
List<ChatCompletionRequestMessage> messagesForReqList = new ArrayList<>();
// Build the system message content
List<ChatCompletionContentPart> systemContentParts = new ArrayList<>();
systemContentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("下面人物是目标人物")
.build());
systemContentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/target.png").build()))
.build());
systemContentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("请确认下面图片中是否含有目标人物")
.build());
// Create the system message
messagesForReqList.add(ChatCompletionRequestSystemMessage.builder()
.role(ChatCompletionRequestMessageType.SYSTEM)
.content(ChatCompletionMessageContent.ofList(systemContentParts))
.build());
// Build the user message content
List<ChatCompletionContentPart> userContentParts = new ArrayList<>();
userContentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("图片1中是否含有目标人物")
.build());
userContentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_01.png").build()))
.build());
userContentParts.add(ChatCompletionContentPartText.builder()
.type(ChatCompletionContentPartType.TEXT)
.text("图片2中是否含有目标人物")
.build());
userContentParts.add(ChatCompletionContentPartImage.builder()
.type(ChatCompletionContentPartType.IMAGE_URL)
.imageUrl(ChatCompletionContentPartImageImageUrl2.ofChatCompletionContentPartImageImageUrl(ChatCompletionContentPartImageImageUrl.builder().url("https://ark-project.tos-cn-beijing.volces.com/doc_image/scene_02.png").build()))
.build());
// Create user message
messagesForReqList.add(ChatCompletionRequestUserMessage.builder()
.role(ChatCompletionRequestMessageType.USER)
.content(ChatCompletionMessageContent.ofList(userContentParts))
.build());
ChatCompletionRequest req = ChatCompletionRequest.builder()
.model("doubao-seed-2-1-pro-260628") //Replace with Model ID
.messages(messagesForReqList)
.maxTokens(300)
.build();
service.createChatCompletion(req)
.getChoices()
.forEach(choice -> System.out.println(choice.getMessage().getContent()));
// shutdown service after all requests are finished
service.shutdownExecutor();
}
}
视觉定位（Visual Grounding）
GUI任务处理
使用说明
说明
处理完图片/视频后，文件会从方舟服务器删除。方舟不会保留你提交的图片、视频以及文本信息等用户数据来训练模型。
图片像素说明
- 传入图片像素要求如下，超出限制后会直接报错。
- 宽 > 14px 且高 > 14px
- 宽*高范围：[196px, 36000000px]
- 宽高比范围：[1/150, 150]
- 图片预处理：
- 根据使用的模型、设置的 detail 模式，将图片等比例缩放至相应的范围（具体见通过 detail 字段（图片理解）），可降低模型响应时延及 token 消耗。
图片 token 用量说明
doubao-seed-2.0 及后续模型估算逻辑：
min(image_width * image_height ÷ 1764, max_image_tokens)
以传入 doubao-seed-2.0 模型的单图 token 最大值为 1280 为例，计算图片消耗的 token 数的逻辑如下：
- 图片尺寸为 1280 px × 960 px：理解这张图消耗的 token 为1280×960÷1764=697，该值小于 1280，根据公式计算消耗 token 数为 697。
- 图片尺寸为 2560 px × 1440 px：理解这张图消耗的 token 为2560×1440÷1764=2090，该值大于 1280，根据公式计算消耗 token 数为 1280。
- 这种情况会对图片进行压缩，即图片会丢失部分细节。譬如字体很小的图片，模型可能会无法识别文字内容。
图片数量说明
单次请求传入图片数量受限于模型上下文窗口。当输入过长，触发模型上下文窗口，信息会被截断。
举例说明：
- 当图片总像素值大，使用的模型上下文窗口为 32k token，每张图片转为 1312 token ，单次请求可传入的图片数量为 32000 ÷ 1312 = 24张。
- 当图片总像素值小，使用的模型上下文窗口为 32k token，每张图片转为 256 token，单次请求可传入的数量为 32000 ÷ 256 = 125 张。
说明
模型回复的质量，受输入图片信息量影响。过多的图片会导致模型回复质量下滑，请合理控制单次请求传入图片的数量。
图片文件容量
- 非 DeepSeek-V4.1-Flash 模型在图片文件的使用限制如下：
- 使用 URL 方式传入图片，单张图片不能超过 10MB。
- 使用 Base64 编码传入图片，单张图片不能超过 10MB，请求体不能超过 64MB。
- 使用文件路径传入图片，图片不能超过 512 MB。
- DeepSeek-V4.1-Flash 模型在图片文件的使用限制参见：通过 detail 字段（图片理解）。
图片格式说明
支持的图片格式如下表，注意文件后缀匹配图片格式，即图片文件扩展名（URL传入时）、图片格式声明（Base64 编码传入时）需与图片实际信息一致。
图片格式 | 文件扩展名 | 内容格式 Content Type |
JPEG | .jpg, .jpeg | image/jpeg |
PNG | .png | image/png |
GIF | .gif | image/gif |
WEBP | .webp | image/webp |
BMP | .bmp | image/bmp |
TIFF | .tiff, .tif | image/tiff |
ICO | .ico | image/x-icon |
DIB | .dib | image/bmp |
ICNS | .icns | image/x-icns |
SGI | .sgi | image/sgi |
JPEG2000 | .j2c, .j2k, .jp2, .jpc, .jpf, .jpx | image/jp2 |
HEIC | .heic | image/heic doubao-1.5-vision-pro及以后模型支持 |
HEIF | .heif | image/heic doubao-1.5-vision-pro及以后模型支持 |
说明
- 上传文件至对象存储时设置，详情请参见 文档。
- 传入 Base64编码时使用：Base64 编码输入。
- 图片格式需小写。
- TIFF、 SGI、ICNS、JPEG2000 几种格式图片，需保证和元数据对齐，如在对象存储中正确设置文件元数据，否则会解析失败，详细请参见 使用视觉理解模型时，报错InvalidParameter？
API 参数字段说明
以下字段视觉理解暂不支持。
- 不支持设置频率惩罚系数，无 frequency_penalty 字段。
- 不支持设置存在惩罚系数，presence_penalty 字段。
- 不支持为单个请求生成多个返回，无 n 字段。
常见问题
最近更新时间：2026.09.22 16:52:16
这个页面对您有帮助吗？
有用
有用
无用
无用