Skip to content

Known issues

Suwat Ch edited this page Apr 26, 2023 · 25 revisions

SiteExtension installation timeout in China region

By default, Kudu is using nuget V2 api version and, unfortunately, nuget does not have services replicated in the China region leading connectivity issue and timeout. The mitigation is to configure SCM to use nuget V3 api version by adding appSettings SCM_SITEEXTENSIONS_FEED_URL = https://api.nuget.org/v3/index.json to the App Service.

New Roslyn based compiler maxes out CPU, slowing down everything

Recently, we updated the .NET build process to use the new VS 2015 toolset. The primary reason is to give access to new C# 6 compiler features like string interpolation.

Unfortunately, we found an issue with compiler that can in some cases cause it to max out the CPU, and that can continue even after the build is complete. Seethis issue.

We are actively discussing with the compiler team, and they will hopefully address the issue. But in the meantime, there are workarounds that can be used to avoid it.

Disable the 'shared compilation' mode of the new compiler

By default, the compiler starts a server process (vbcscompiler.exe), and sends work to it. This server keeps running across compilations. The bug is related to how this server communicate via a named pipe.

You can easily disable this behavior by adding the following App Settings in the Azure portal:

SCM_BUILD_ARGS=/p:UseSharedCompilation=false

Go back to the older compiler

As an alternative, if you are not using any VS 2015 features, you can back to the previous compiler. Add the following App Settings in the Azure portal:

MSBUILD_PATH=D:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe

You may get occasional "Couldn't reserve space for cygwin's heap" errors when git pushing

This is tracked by https://github.com/projectkudu/kudu/issues/878.

The full error looks like this:

remote:       0 [main] us 0 init_cheap: VirtualAlloc pointer is null, Win32 error 487
remote: AllocationBase 0x0, BaseAddress 0x68570000, RegionSize 0x4A0000, State 0x10000
remote: D:\Program Files (x86)\Git\bin\sh.exe: *** Couldn't reserve space for cygwin's heap, Win32 error 0
To https://xxx.scm.azurewebsites.net:443/xxx.git
 * [new branch]      master -> master

It seems to be a general msysgit issue that people run into occasionally (not related to Azure Web Apps). e.g. see various references to it. From our experience, this happens at most once on a given server, so I suspect you won't hit this continually (though you may get moved to a different server later).

The workaround is to make a dummy commit (e.g. using --allow-empty so you don't have to change any files) and push again. If you want to keep your repo clean, you can then reset that commit and force push.

Our hope is that this issue is fixed in the newest msysgit (version 2.5.x). Right now, Kudu is still using an older build (1.8.4). Unless we find some issues with using the newer build, we're planning to move to it before the end of 2015, and hopefully this will take care of this.

Request time out after 4 mins

The http requests coming for an App Service will first arrive at the load balancer (aka. FrontEnd) before getting forwarded to the worker machine. There is a hardcoded timeout of 4 mins on load balancer. Meaning, once the request is forwarded if there is a period of 4 mins (or 240 secs) without response writing back, the load balance will timeout the request with 502 to caller. The worker may not be aware about this and continue to execute the request till completion (200 status in IISLog). To workaround this timeout, an App Service must keep writing the response back and avoid this 4 mins gap of idle response.

NullRepositoy: ChangeSetId(...) does not match ...

This error may happen during ZipDeploy to an App Service. The common cause was an App Service was setup to deploy from git at some point but not disconnect from git completely or successfully. As a result, any other deployment mechanisms (such as ZipDeploy) will fail with that error to prevent multiple conflicting deployment sources to the same App Service.

To workaround,

  • Azure Portal Deployment Center and disconnect the deployment from an App Service.
  • Remove DEPLOYMENT_BRANCH appsettings from App Service
  • Delete d:\home\site\repository directory as well as d:\home\site\deployments\settings.xml file.

Invoke-RestMethod -Method POST failed with 403

Invoke-RestMethod -Method POST (as well as Invoke-WebRequest), by default, sends Content-Type: application/x-www-form-urlencoded and User-Agent: Mozilla/5.0, ...   This is a browser FORM POST scenario and, to prevent CSRF, Referer header is required (browsers always send this header) and must match the current HOST.  Invoke-RestMethod does not send Referer header; hence, the request is rejected.

To workaround - either of the below would do.

  • Use PUT instead of POST method.
  • Set different User-Agent other than those of known browser User-Agents (Mozilla/.. among other things).
  • Set request header Content-Type to match the content. For instance, Zipdeploy's content-type should be application/octet-stream or application/json depending on the payload.
  • Set request header Referer: https://samehostname
Clone this wiki locally