Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can not use torch.compile with dynamic=Ture when using multi-threads #126024

Open
CaoE opened this issue May 12, 2024 · 1 comment
Open

Can not use torch.compile with dynamic=Ture when using multi-threads #126024

CaoE opened this issue May 12, 2024 · 1 comment
Labels
module: dynamic shapes oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@CaoE
Copy link
Collaborator

CaoE commented May 12, 2024

馃悰 Describe the bug

When running a model processed by torch.compile in a multi-threaded environment, the following error will be reported:
RuntimeError: Detected that you are using FX to symbolically trace a dynamo-optimized function. This is not supported at the moment.
There is a case where the same model is run in multiple threads:

model = torch.compile(model, dynamic=True)

thread = threading.Thread(
    target=run_sharing_model,
    args=(model, dataloader, ...)
)

Is this error expected and is it possible to support this case?

Error logs

RuntimeError: Detected that you are using FX to symbolically trace a dynamo-optimized function. This is not supported at the moment.

Minified repro

No response

Versions

torch 2.4.0.dev20240417+cpu

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang

@ezyang
Copy link
Contributor

ezyang commented May 12, 2024

Probably same root cause as #118387

@bdhirsh bdhirsh added triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module module: dynamic shapes labels May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: dynamic shapes oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

3 participants