Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Transformer backend error on iGPU #1971

Closed
fakezeta opened this issue Apr 8, 2024 · 0 comments · Fixed by #1972
Closed

bug: Transformer backend error on iGPU #1971

fakezeta opened this issue Apr 8, 2024 · 0 comments · Fixed by #1972
Assignees
Labels
bug Something isn't working unconfirmed

Comments

@fakezeta
Copy link
Collaborator

fakezeta commented Apr 8, 2024

LocalAI version:
latest master-sycl docker image

Describe the bug
After the bump for intel-extension-for-transformers version 1.4 transformer library does no more work on system without dGPU sinche the package does not support iGPU like 1.3.2

Opened also an issue upstream intel/intel-extension-for-transformers#1463

optimum[openvino] is not impacted by this beaviour and should continue to work as before

To Reproduce
run a transformer model without dGPU Hardware

Logs

1:51PM DBG GRPC Service state dir: /tmp/go-processmanager1406127051
1:51PM DBG GRPC Service Started
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr /usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   warn(
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr /usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   warnings.warn(
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr 2024-04-08 13:51:23,531 - datasets - INFO - PyTorch version 2.1.0a0+cxx11.abi available.
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr Traceback (most recent call last):
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   File "/build/backend/python/transformers/transformers_server.py", line 26, in <module>
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr     from intel_extension_for_transformers.transformers.modeling import AutoModelForCausalLM
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   File "/usr/local/lib/python3.10/dist-packages/intel_extension_for_transformers/transformers/__init__.py", line 56, in <module>
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr     from .modeling import (
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   File "/usr/local/lib/python3.10/dist-packages/intel_extension_for_transformers/transformers/modeling/__init__.py", line 22, in <module>
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr     from .modeling_auto import (AutoModel, AutoModelForCausalLM,
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   File "/usr/local/lib/python3.10/dist-packages/intel_extension_for_transformers/transformers/modeling/modeling_auto.py", line 87, in <module>
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr     if is_ipex_available() and get_gpu_family() != "no_gpu":
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr   File "/usr/local/lib/python3.10/dist-packages/intel_extension_for_transformers/tools/utils.py", line 63, in get_gpu_family
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr     assert False, "Unsupported GPU device: {}".format(name)
1:51PM DBG GRPC(fakezeta/Starling-LM-7B-beta-openvino-int8-127.0.0.1:45665): stderr AssertionError: Unsupported GPU device: Intel(R) UHD Graphics 770

Additional context
I'm working on a PR to handle the error hoping it will be fixed upstream.

@fakezeta fakezeta added bug Something isn't working unconfirmed labels Apr 8, 2024
@fakezeta fakezeta self-assigned this Apr 8, 2024
@fakezeta fakezeta mentioned this issue Apr 8, 2024
1 task
@fakezeta fakezeta linked a pull request Apr 8, 2024 that will close this issue
1 task
mudler pushed a commit that referenced this issue Apr 8, 2024
fixes regression #1971 introduced by intel_extension_for_transformers==1.4
mudler pushed a commit that referenced this issue Apr 21, 2024
…mpts options (#2090)

* fix regression #1971

fixes regression #1971 introduced by intel_extension_for_transformers==1.4

* UseTokenizerTemplate and StopPrompt

Implementation of use_tokenizer_template and stopwords options
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working unconfirmed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant