[Bug]: assert len(indices) == len(inputs) with Qwen/Qwen2-VL-2B-Instruct
#9128
Comments
|
You should follow the prompt template as shown on their HuggingFace repo. The easiest way is to use |
|
If I were to use |
|
Please read the examples on their HuggingFace repo. The format of |
|
Modified my code: from vllm import LLM, SamplingParams
from PIL import Image
if __name__ == "__main__":
vllm_engine = LLM("Qwen/Qwen2-VL-2B-Instruct")
sampling_params = SamplingParams(max_tokens=120)
num_images = 3
messages = [{"role": "user", "content": []}]
for _ in range(num_images):
new_image = {"type": "image", "image": Image.new("RGB", (224, 224))}
messages[0]["content"].append(new_image)
messages[0]["content"].append({"type": "text", "text": "Describe this image."})
outputs = vllm_engine.chat(messages, sampling_params)
print(outputs)Now it leads to: [rank0]: Traceback (most recent call last):
[rank0]: File "/home/sayak/diffusers/check_video_vllm.py", line 32, in <module>
[rank0]: outputs = vllm_engine.chat(messages, sampling_params)
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/llm.py", line 556, in chat
[rank0]: conversation, mm_data = parse_chat_messages(
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 487, in parse_chat_messages
[rank0]: sub_messages = _parse_chat_message_content(msg, mm_tracker)
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 440, in _parse_chat_message_content
[rank0]: result = _parse_chat_message_content_parts(
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 402, in _parse_chat_message_content_parts
[rank0]: raise NotImplementedError(f"Unknown part type: {part_type}")
[rank0]: NotImplementedError: Unknown part type: image |
for _ in range(num_images):
new_image = {"type": "image", "image": Image.new("RGB", (224, 224))}
messages[0]["content"].append(new_image)Instead of |
|
Alright. I will try that and report here but I think #9128 (comment) is misleading then. I did follow the exact "messages" formatting. |
Sorry, I missed the slight difference in the image format. Hope that everything is cleared up now! |
|
I tested Qwen2-VL-7B-Instruct work fine, but tested Qwen2-VL-72B-Instruct-GPTQ-Int4 error! |
|
I tested Qwen2-VL-2B-Instruct working well! |
Can you check whether both repos have a correctly defined chat template? |
|
With def encode_image(image):
buffered = io.BytesIO()
image.save(buffered, format="JPEG")
image_bytes = buffered.getvalue()
return base64.b64encode(image_bytes).decode("utf-8")
num_images = 3
messages = [{"role": "user", "content": []}]
for _ in range(num_images):
base64_image = encode_image(Image.new("RGB", (224, 224)))
new_image = {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}}
messages[0]["content"].append(new_image)
messages[0]["content"].append({"type": "text", "text": "Describe this image."})I get: [rank0]: Traceback (most recent call last):
[rank0]: File "/home/sayak/diffusers/check_video_vllm.py", line 41, in <module>
[rank0]: outputs = vllm_engine.chat(messages, sampling_params)
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/llm.py", line 556, in chat
[rank0]: conversation, mm_data = parse_chat_messages(
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 487, in parse_chat_messages
[rank0]: sub_messages = _parse_chat_message_content(msg, mm_tracker)
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 440, in _parse_chat_message_content
[rank0]: result = _parse_chat_message_content_parts(
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 392, in _parse_chat_message_content_parts
[rank0]: mm_parser.parse_image(image_url["url"])
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 276, in parse_image
[rank0]: placeholder = self._tracker.add("image", image)
[rank0]: File "/home/sayak/vllm/vllm/entrypoints/chat_utils.py", line 205, in add
[rank0]: raise ValueError(
[rank0]: ValueError: At most 1 image(s) may be provided in one request.What am I missing out on? |
Please check our docs on using VLMs. There is a section on how to input multiple images per prompt. |
|
I am following https://docs.vllm.ai/en/stable/models/vlm.html and I am still really not sure what I am missing out here. A bit more specificity in your replies would be appreciated. |
You need to set |
Your current environment
The output of `python collect_env.py`
Model Input Dumps
No response
🐛 Describe the bug
Trying to run:
Leads to:
Before submitting a new issue...
The text was updated successfully, but these errors were encountered: