Skip to content

Avoid blocking the event loop with requests.get in async update_model_type#5055

Open
kratos0718 wants to merge 1 commit into
xorbitsai:mainfrom
kratos0718:fix/blocking-requests-in-async-update-model-type
Open

Avoid blocking the event loop with requests.get in async update_model_type#5055
kratos0718 wants to merge 1 commit into
xorbitsai:mainfrom
kratos0718:fix/blocking-requests-in-async-update-model-type

Conversation

@kratos0718

Copy link
Copy Markdown

What

update_model_type is an async actor method, but it calls the synchronous requests.get(url, timeout=30) directly:

async def update_model_type(self, model_type: str):
    ...
    response = requests.get(url, timeout=30)

A blocking call inside a coroutine freezes the actor's event loop for the entire duration of the request — up to the 30s timeout while downloading model configurations from model.xinference.io. During that window the worker actor cannot service any other request.

Fix

Run the blocking request in a worker thread with asyncio.to_thread(), so the event loop stays responsive:

response = await asyncio.to_thread(requests.get, url, timeout=30)

This matches the pattern already used throughout worker.py (e.g. await asyncio.to_thread(...) appears in several other places in this module). asyncio is already imported at the top of the file. No behavior change other than no longer blocking the loop.

…_type

update_model_type is an async actor method, but it called the synchronous
requests.get() directly, which blocks the actor's event loop for up to the
30s timeout while downloading model configurations from the remote API.
During that window the worker cannot service any other request.

Run the blocking call in a thread via asyncio.to_thread(), matching the
pattern already used throughout this module.
@XprobeBot XprobeBot added this to the v2.x milestone Jun 19, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates xinference/core/worker.py to run the blocking requests.get call inside asyncio.to_thread when downloading JSON from a remote API. This prevents the synchronous network request from freezing the actor's event loop. There are no review comments, and I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants