Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gitlab CI Jobs randomly fail #49

Open
Flash1232 opened this issue Jan 22, 2021 · 3 comments
Open

Gitlab CI Jobs randomly fail #49

Flash1232 opened this issue Jan 22, 2021 · 3 comments
Labels
missing information Further information is requested not our bug Bugs from external software/configuration

Comments

@Flash1232
Copy link

Flash1232 commented Jan 22, 2021

Did you ever encounter weird phenomena like these:

Running with gitlab-runner 13.8.0 (775dd39d)
  on newest xxxxxx
Resolving secrets
00:00
Preparing the "docker" executor
...
Preparing environment
00:01
Running on runner-xxxxxxxx-project-169-concurrent-0 via c9efa222138c...
Getting source from Git repository
00:04
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/<group>/<project>/.git/
fatal: shallow file has changed since we read it
Cleaning up file based variables
00:02
ERROR: Job failed: exit code 1

?

It happens randomly to different jobs (mostly linux and windows export stages), sometimes files go missing, sometimes I get this. It's frustrating. I'm running a docker-in-docker gitlab-runner setup.

EDIT: I am running a docker-in-docker runner configuration and set concurrent=4 in the runner toml file so there should never be more jobs than this setting allows. I was able to workaround this issue by adding retries to the jobs inside the .gitlab-ci.yml file. Unsure if this is a GitLab, Godot command line or godot-ci -specific bug.

@Calinou Calinou added the bug Something isn't working label Jan 22, 2021
@TRy-Dev
Copy link

TRy-Dev commented Apr 16, 2021

I started getting these errors for Windows build recently as well.

There's a link to https://api.itch.io/wharf/builds/... in the log, here's it's contents:

{
  "errors": [
    "method not supported"
  ],
  "details": "The HTTP method you used (GET) does not work with this API endpoint"
}

@nicktheway
Copy link

I run into the same issue today, for me it turned out that I was using 2 gitlab-runner services on the same runner config:

  • One that I'd started after registering a runner using gitlab-runner run
  • And another running in the background from the gitlab-runner.service. I didn't think I would have that one since I never explicitly run gitlab-runner install/start.

Also noticed that, something weird was going on since even though I had set concurrent = 1, two jobs were being picked up at the same time on my custom runner.

I figured it out when, finally, I tried to create the service seeing that it was already there 🤦‍♂️

The issue was resolved after killing the manually started runner ^ ^

TLDR
If you have manually started a runner (eg. using gitlab-runner run) make sure that there is no other runner instance running against the same config (could be a manual or a service one - eg if systemctl status gitlab-runner.service shows the service as active)

@abarichello abarichello added missing information Further information is requested not our bug Bugs from external software/configuration and removed bug Something isn't working labels Sep 21, 2021
@exarkun
Copy link

exarkun commented Oct 5, 2021

This also happens even if you don't accidentally have to gitlab-runner instances running using the same config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
missing information Further information is requested not our bug Bugs from external software/configuration
Projects
None yet
Development

No branches or pull requests

6 participants