Support http protocol for adding applications

[davidstauffer]

We need to support for customer engagements the HTTP protocol to connect to git provider.
Customers are using HTTP protocol through a VPN.

So that customers can add and manage applications via GitOps.

[LutzLange]

The Amgen firewall blocks ssh traffic to gitlab.com. This prevents us from adding applications from the WGE UI.

[bigkevmcd]

@LutzLange How do they authenticate the HTTPS connections? because it's read-only and internal, they don't?

[LutzLange]

Good question. I don't think this is an issue though, right?

Authentication should run through the GITLAB_TOKEN, should it not? https://docs.gitlab.com/ee/security/token_overview.html
I can ask.

[bigkevmcd]

@LutzLange not necessarily...for example, https:// URLs are read-only without authentication on GitHub (which is why they are the default if you're not authenticated/not allowed to push).

I'm not totally sure about GitLab, but I'd suspect it's the same

[bigkevmcd]

@LutzLange the reason I ask, is to ensure that this isn't going to become another mTLS issue...

[LutzLange]

They will need write access as well for creating pull requests. I'll ask how they authenticate. SSL verification should be something else here, right. This should usually work through the ca-bundle that is on the machine you make the call from. Or am I missing something?

[foot]

• EE already does everything over HTTPS
• Core adds apps via gitClone (which it might force onto ssh right now?)
• Needs a GITHUB token for flux to use somehow, how does this work? Does flux use your PAT directly or uses it to create a new TOKEN (? possible?)

[LutzLange]

@foot This was regarding adding applications. That would be the Core part as well, right?

We are talking about GitLab here and this would possibly be a GitLab Token

[foot]

Needs a GITHUB token for flux to use somehow, how does this work? Does flux use your PAT directly or uses it to create a new TOKEN (? possible?)

Did a quick test this seems to be the case, saves GITHUB_TOKEN into the flux-system secret.

[foot]

1. gitops install
   • save GITHUB_TOKEN from env into a secret and use this to sync a private repo over https (flux support this w/ --token-auth)
2. Adding a new app from the CLI:
   • do what we did in 1.
3. Add a new app from the UI (can use the PAT but can't actually grab the PAT)
   • Is this possible?????

[LutzLange]

@foot we need this to work with gitlab in this case. Also with onprem gitlab.

[LutzLange]

Amgen confirms that they use an GitLab_Token as env var.

[foot]

• Change to gitops install to allow running flux bootstrap --token-auth
• Change to gitops add app: either
  • Update gitClone to use https and get the token somehow?
  • re-write all the gitClones to use github/gitlab apis? (like EE).

[jpellizzari]

Related:

• Cloning a repo relies on settings in the user's environment #1100
• Spike: Unify git operations (choose between go_git and go-git-providers)  #837

[jpellizzari]

Refinement decision: need a spike on this. Spike outcome:

Enumerate all of the work that needs to be done to support this. Also let's draw a picture maybe?

Notes on possible AC:

• Add fields to allow for a secret name to be specified
• Respect HTTPS protocol as a settings (see Support HTTPS installations #1276)
• Avoid adding deploy keys to a repo in a case where the HTTP(S) protocol is provided