Unable to run with latest vllm version

by suneetk - opened Dec 3, 2025

Dec 3, 2025

(APIServer pid=87) Traceback (most recent call last):
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/builds-v0/.tmpkjMLaP/bin/vllm", line 10, in <module>
(APIServer pid=87)     sys.exit(main())
(APIServer pid=87)              ^^^^^^
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=87)     args.dispatch_function(args)
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=87)     uvloop.run(run_server(args))
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=87)     return __asyncio.run(
(APIServer pid=87)            ^^^^^^^^^^^^^^
(APIServer pid=87)   File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=87)     return runner.run(main)
(APIServer pid=87)            ^^^^^^^^^^^^^^^^
(APIServer pid=87)   File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=87)     return self._loop.run_until_complete(task)
(APIServer pid=87)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=87)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=87)     return await main
(APIServer pid=87)            ^^^^^^^^^^
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1385, in run_server
(APIServer pid=87)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1404, in run_server_worker
(APIServer pid=87)     async with build_async_engine_client(
(APIServer pid=87)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=87)     return await anext(self.gen)
(APIServer pid=87)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 172, in build_async_engine_client
(APIServer pid=87)     async with build_async_engine_client_from_engine_args(
(APIServer pid=87)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=87)     return await anext(self.gen)
(APIServer pid=87)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 198, in build_async_engine_client_from_engine_args
(APIServer pid=87)     vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=87)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1739, in create_engine_config
(APIServer pid=87)     config = VllmConfig(
(APIServer pid=87)              ^^^^^^^^^^^
(APIServer pid=87)   File "/home/ubuntu/.cache/uv/archive-v0/_BAPhTxgb2PFk0XjVv2H-/lib/python3.12/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=87)     s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=87) pydantic_core._pydantic_core.ValidationError: 2 validation errors for VllmConfig
(APIServer pid=87) scale_dtype
(APIServer pid=87)   Extra inputs are not permitted [type=extra_forbidden, input_value=None, input_type=NoneType]
(APIServer pid=87)     For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
(APIServer pid=87) zp_dtype
(APIServer pid=87)   Extra inputs are not permitted [type=extra_forbidden, input_value=None, input_type=NoneType]
(APIServer pid=87)     For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
[WARN] <orchestrator-8B> ExitError >> exit status 1, exit code: 1

cpatonn

cyankiwi org Dec 3, 2025

Hi @suneetk , thank you for letting me know. Please redownload the config.json file, and replace the old one. It should work now.

suneetk

Dec 5, 2025

Now, there is a new error

(APIServer pid=975) Traceback (most recent call last):
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/builds-v0/.tmpt0PQqW/bin/vllm", line 10, in <module>
(APIServer pid=975)     sys.exit(main())
(APIServer pid=975)              ^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=975)     args.dispatch_function(args)
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=975)     uvloop.run(run_server(args))
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=975)     return __asyncio.run(
(APIServer pid=975)            ^^^^^^^^^^^^^^
(APIServer pid=975)   File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
(APIServer pid=975)     return runner.run(main)
(APIServer pid=975)            ^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=975)     return self._loop.run_until_complete(task)
(APIServer pid=975)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=975)     return await main
(APIServer pid=975)            ^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1385, in run_server
(APIServer pid=975)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1404, in run_server_worker
(APIServer pid=975)     async with build_async_engine_client(
(APIServer pid=975)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=975)     return await anext(self.gen)
(APIServer pid=975)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 172, in build_async_engine_client
(APIServer pid=975)     async with build_async_engine_client_from_engine_args(
(APIServer pid=975)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=975)     return await anext(self.gen)
(APIServer pid=975)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 213, in build_async_engine_client_from_engine_args
(APIServer pid=975)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=975)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 223, in from_vllm_config
(APIServer pid=975)     return cls(
(APIServer pid=975)            ^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 114, in __init__
(APIServer pid=975)     tokenizer = init_tokenizer_from_config(self.model_config)
(APIServer pid=975)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/tokenizers/registry.py", line 227, in init_tokenizer_from_config
(APIServer pid=975)     return get_tokenizer(
(APIServer pid=975)            ^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/tokenizers/registry.py", line 191, in get_tokenizer
(APIServer pid=975)     tokenizer = TokenizerRegistry.get_tokenizer(
(APIServer pid=975)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/tokenizers/registry.py", line 86, in get_tokenizer
(APIServer pid=975)     return item.from_pretrained(*args, **kwargs)
(APIServer pid=975)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/vllm/tokenizers/hf.py", line 84, in from_pretrained
(APIServer pid=975)     tokenizer = AutoTokenizer.from_pretrained(
(APIServer pid=975)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1156, in from_pretrained
(APIServer pid=975)     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
(APIServer pid=975)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2113, in from_pretrained
(APIServer pid=975)     return cls._from_pretrained(
(APIServer pid=975)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2359, in _from_pretrained
(APIServer pid=975)     tokenizer = cls(*init_inputs, **init_kwargs)
(APIServer pid=975)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/models/qwen2/tokenization_qwen2_fast.py", line 120, in __init__
(APIServer pid=975)     super().__init__(
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/tokenization_utils_fast.py", line 178, in __init__
(APIServer pid=975)     super().__init__(**kwargs)
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1472, in __init__
(APIServer pid=975)     self._set_model_specific_special_tokens(special_tokens=self.extra_special_tokens)
(APIServer pid=975)   File "/home/ubuntu/.cache/uv/archive-v0/DNxrC1LzeJJ5pCahzoD3O/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1210, in _set_model_specific_special_tokens
(APIServer pid=975)     self.SPECIAL_TOKENS_ATTRIBUTES = self.SPECIAL_TOKENS_ATTRIBUTES + list(special_tokens.keys())
(APIServer pid=975)                                                                            ^^^^^^^^^^^^^^^^^^^
(APIServer pid=975) AttributeError: 'list' object has no attribute 'keys'
[WARN] <orchestrator-8B> ExitError >> exit status 1, exit code: 1

cpatonn

cyankiwi org Dec 5, 2025

Thanks for letting me know again. Please redownload the .json files.

It is tested to work now :)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment