Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WASI][AOT] LLVM ERROR: out of memory #101533

Open
pavelsavara opened this issue Apr 25, 2024 · 11 comments
Open

[WASI][AOT] LLVM ERROR: out of memory #101533

pavelsavara opened this issue Apr 25, 2024 · 11 comments
Assignees
Labels
arch-wasm WebAssembly architecture area-Codegen-AOT-mono area-Codegen-LLVM-mono disabled-test The test is disabled in source code against the issue Known Build Error Use this to report build issues in the .NET Helix tab os-wasi Related to WASI variant of arch-wasm
Milestone

Comments

@pavelsavara
Copy link
Member

pavelsavara commented Apr 25, 2024

Note both LLVM 17 and 18 have the same problem with AOT of Invariant.Tests
Log

C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang -cc1 version 18.1.2 based upon LLVM 18.1.2 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]

Log

C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang -cc1 version 17.0.6 based upon LLVM 17.0.6 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=655571
Build error leg or test failing:

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "LLVM ERROR: out of memory",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=655571
Error message validated: [LLVM ERROR: out of memory]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/25/2024 9:35:23 AM UTC

Report

Build Definition Test Pull Request
678721 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
678685 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #102362
678603 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #102357
678577 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
678427 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #102351
671701 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
670895 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
665970 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
663948 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101800
664120 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101461
664107 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
664022 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101828
663982 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
663755 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101806
663692 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101818
663727 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101732
663696 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101773
663695 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101819
663649 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
663686 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
663591 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101277
663579 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101814
663535 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101800
663468 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100951
663429 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101640
663401 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101806
663389 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101808
663382 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101807
663336 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101762
663301 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101802
663287 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101781
663262 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100951
663268 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101800
663222 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101797
663200 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101624
662930 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101761
663164 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101640
663140 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101716
663076 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100951
662957 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101762
662949 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101767
662938 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100951
662942 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101684
662830 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
662542 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101761
662574 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100619
662347 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101624
662312 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #99490
662296 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101753
662281 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
662229 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101732
662225 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101151
662174 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101773
662158 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101761
662154 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101277
662112 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101451
662100 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101753
662046 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101732
661923 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101151
662033 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101767
661965 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101766
661299 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101745
661869 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101624
661888 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101716
661873 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101762
661816 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101717
661799 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101443
661671 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
661662 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100266
661068 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101712
661154 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101715
661633 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101151
661543 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101743
661229 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101735
661485 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101151
661476 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101755
661442 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101743
661457 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101732
661428 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101753
661405 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100030
661384 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101752
661374 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101751
661304 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101674
661263 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
661247 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101740
661214 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101277
661111 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution
660990 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101631
660981 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101624
660967 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101212
660955 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101451
660937 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101725
661006 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101693
661010 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101693
661019 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101318
660940 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101725
660999 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100266
660849 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101151
660861 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #101674
660836 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #100619
Displaying 100 of 370 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 5 370
@pavelsavara pavelsavara added area-Codegen-AOT-mono Known Build Error Use this to report build issues in the .NET Helix tab os-wasi Related to WASI variant of arch-wasm labels Apr 25, 2024
@pavelsavara pavelsavara added this to the 9.0.0 milestone Apr 25, 2024
@SingleAccretion
Copy link
Contributor

If this is with the 'vanilla' clang from WASI SDK (it looks to be like that from the logs), this may be WebAssembly/wasi-sdk#326.

@lewing
Copy link
Member

lewing commented Apr 25, 2024

If this is with the 'vanilla' clang from WASI SDK (it looks to be like that from the logs), this may be WebAssembly/wasi-sdk#326.

It is. The first recorded failure was in #100993 and the second was in a rolling build with that commit in it so I wonder if we're hitting a new path (for mono) that is triggering that failure. The other possibility is some trimming changes that may have exposed it.

cc @radekdoulik @tannergooding

@lewing
Copy link
Member

lewing commented Apr 25, 2024

:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Failed to compile C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.CoreLib.dll.bc -> C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.CoreLib.dll.o [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>C:\Windows\System32\chcp.com 65001 1>nul  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>setlocal [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>set errorlevel=dummy  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>set errorlevel=  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>"C:\helix\work\correlation\build\\wasi-sdk\\bin\clang.exe" "@C:\helix\work\correlation\build\microsoft.netcore.app.runtime.wasi-wasm\runtimes\wasi-wasm\native\src\wasi-default.rsp" "@C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\wasi-compile-bc.rsp" -c -o "C:\Users\ContainerAdministrator\AppData\Local\Temp\tmp1tki23.tmp" "C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.CoreLib.dll.bc"  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang version 17.0.6 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error :  (in-process) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error :  "C:/helix/work/correlation/build/wasi-sdk/bin/clang.exe" -cc1 -triple wasm32-unknown-wasi -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name System.Private.CoreLib.dll.bc -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility=hidden -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=C:/helix/work/workitem/e/publish -resource-dir C:/helix/work/correlation/build/wasi-sdk/lib/clang/17 -fdebug-compilation-dir=C:/helix/work/workitem/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -o "C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmp1tki23.tmp" -x ir "C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\System.Private.CoreLib.dll.bc" [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang -cc1 version 17.0.6 based upon LLVM 17.0.6 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Stack dump: [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0.	Program arguments: C:\\helix\\work\\correlation\\build\\\\wasi-sdk\\\\bin\\clang.exe @C:\\helix\\work\\correlation\\build\\microsoft.netcore.app.runtime.wasi-wasm\\runtimes\\wasi-wasm\\native\\src\\wasi-default.rsp @C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\wasi-compile-bc.rsp -c -o C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmp1tki23.tmp C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\System.Private.CoreLib.dll.bc [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 1.	Code generation [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 2.	Releasing pass 'Assumption Cache Tracker' [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Exception Code: 0xC000001D [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00304C15 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x7280C902 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x002E6D34 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x001EF0F6 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x001EF22E <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x001F0C32 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x001EDF1A <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x11FEECD0 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00800C8F <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00A5BE58 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang: error: clang frontend command failed due to signal (use -v to see invocation) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang version 17.0.6 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs. [took 15.23s] [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Failed to compile C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.Xml.dll.bc -> C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.Xml.dll.o [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>C:\Windows\System32\chcp.com 65001 1>nul  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>setlocal [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>set errorlevel=dummy  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>set errorlevel=  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : C:\helix\work\workitem\e\publish>"C:\helix\work\correlation\build\\wasi-sdk\\bin\clang.exe" "@C:\helix\work\correlation\build\microsoft.netcore.app.runtime.wasi-wasm\runtimes\wasi-wasm\native\src\wasi-default.rsp" "@C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\wasi-compile-bc.rsp" -c -o "C:\Users\ContainerAdministrator\AppData\Local\Temp\tmpekto3q.tmp" "C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.Xml.dll.bc"  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang version 17.0.6 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error :  (in-process) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error :  "C:/helix/work/correlation/build/wasi-sdk/bin/clang.exe" -cc1 -triple wasm32-unknown-wasi -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name System.Private.Xml.dll.bc -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility=hidden -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=C:/helix/work/workitem/e/publish -resource-dir C:/helix/work/correlation/build/wasi-sdk/lib/clang/17 -fdebug-compilation-dir=C:/helix/work/workitem/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -o "C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmpekto3q.tmp" -x ir "C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\System.Private.Xml.dll.bc" [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang -cc1 version 17.0.6 based upon LLVM 17.0.6 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Stack dump: [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0.	Program arguments: C:\\helix\\work\\correlation\\build\\\\wasi-sdk\\\\bin\\clang.exe @C:\\helix\\work\\correlation\\build\\microsoft.netcore.app.runtime.wasi-wasm\\runtimes\\wasi-wasm\\native\\src\\wasi-default.rsp @C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\wasi-compile-bc.rsp -c -o C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmpekto3q.tmp C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\System.Private.Xml.dll.bc [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 1.	Code generation [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 2.	Running pass 'Function Pass Manager' on module 'C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\System.Private.Xml.dll.bc'. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 3.	Running pass 'WebAssembly Explicit Locals' on function '@System.Private.Xml_System_Xml_Xsl_IlGen_XmlILOptimizerVisitor_VisitAdd_System_Xml_Xsl_Qil_QilBinary' [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Exception Code: 0xC000001D [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00304C15 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x7280C902 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x002E6D34 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x002E8DE4 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00B25963 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00B25CD8 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00B25C42 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x00B25BD4 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x004596FA <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x028B7400 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : 0x0280328C <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang: error: clang frontend command failed due to signal (use -v to see invocation) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang version 17.0.6 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(832,5): error : clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs. [took 15.62s] [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]

@lewing lewing added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' arch-wasm WebAssembly architecture labels Apr 25, 2024
@lewing
Copy link
Member

lewing commented Apr 25, 2024

see also #99150

Copy link
Contributor

Tagging subscribers to this area: @lambdageek, @steveisok
See info in area-owners.md if you want to be subscribed.

@steveisok
Copy link
Member

@fanyang-mono this is likely instrinics related

@tannergooding
Copy link
Member

tannergooding commented Apr 26, 2024

The first recorded failure was in #100993 and the second was in a rolling build with that commit in it so I wonder if we're hitting a new path (for mono) that is triggering that failure

Ah, that's interesting. CI for that PR was run several times, with the very last one just being a rerun for extra precaution before P4 and all of the CI failures had existing issues attributed to them (in this case, it looks like it was attributed to #99150).

Given the test the OP is referencing, it doesn't look like the issue had shown up in the prior CI runs either. At a glance: https://github.com/dotnet/runtime/runs/24118131821 (which is on top of 11b53b2) doesn't have S.C.SpecializedTests failing vs https://github.com/dotnet/runtime/runs/24176375150 (which is on top of 40bc2d8) which did. There's an 80 commit difference made to main between those two: 11b53b2...40bc2d8 (but notably no change to the PR code itself, the same was true for the commit from several days before that which likewise just merged main into the PR to get up to date test results)

Is this one not consistently failing perhaps?

@lambdageek
Copy link
Member

lambdageek commented Apr 26, 2024

I remember Zoltan turned on dead code elimination in mono when AOTing llvm bitcode. Was that only for browser-wasm, or all llvm? is it possible we didn't turn it on for wasi?

@lewing
Copy link
Member

lewing commented May 2, 2024

Instead of worrying about what caused it to start failing recently the bundled clang is definitely still 32bit (as @SingleAccretion
pointed out) and that seems to be underlying cause so lets update our wasi-sdk packages with a proper clang and use those asap. @radekdoulik can you please take care of that.

lewing added a commit to lewing/runtime that referenced this issue May 2, 2024
The clang in the wasi-sdk is a 32bit exe and it is
failing to do aot compilation correctly so disable
the lane until that is fixed

dotnet#101533
@lewing lewing added disabled-test The test is disabled in source code against the issue in-pr There is an active PR which will close this issue when it is merged labels May 2, 2024
@lewing
Copy link
Member

lewing commented May 2, 2024

opened #101827 to disable the failing bits in the meantime

lewing added a commit to lewing/runtime that referenced this issue May 2, 2024
The clang in the wasi-sdk is a 32bit exe and it is
failing to do aot compilation correctly so disable
the lane until that is fixed

dotnet#101533
@lewing lewing closed this as completed in 9e96675 May 3, 2024
@lewing lewing reopened this May 3, 2024
@lewing
Copy link
Member

lewing commented May 3, 2024

reopening to reenable the disabled tests and verify the fix

24-Hour Hit Count 7-Day Hit Count 1-Month Count
33 248 358

@lewing lewing removed in-pr There is an active PR which will close this issue when it is merged blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' labels May 3, 2024
michaelgsharp pushed a commit to michaelgsharp/runtime that referenced this issue May 9, 2024
The clang in the wasi-sdk is a 32bit exe and it is
failing to do aot compilation correctly so disable
the lane until that is fixed

dotnet#101533
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-wasm WebAssembly architecture area-Codegen-AOT-mono area-Codegen-LLVM-mono disabled-test The test is disabled in source code against the issue Known Build Error Use this to report build issues in the .NET Helix tab os-wasi Related to WASI variant of arch-wasm
Projects
None yet
Development

No branches or pull requests

7 participants