runtime-local apis for project deployment #4909

pjain1 · 2024-05-15T20:15:40Z

This PR adds 3 new APIs to support auto-deploy from UI.

DeployValidation - It does basic validation and returns a response which can indicate further action to be taken by the client
1. is_authenticated - If false then client should redirect to /auth to get user signed in, other fields in the response should be ignored, after login the validation should be done again to get latest info. A redirect query can be set with url that the runtime should redirect to after login
2. is_github_connected - User does have not rill git app installed, so redirect user to git_grant_access_url which is the next field.
3. git_grant_access_url - git access url
4. git_user_name - git user name, also the default git org for the user.
5. git_user_orgs - git orgs that user is part of. Currently not used, we use the default org which is equivalent to git username
6. is_git_repo - is the rill project folder a git repo ? meaning does it have .git folder. If this is true, then in first version we should just ask to continue in cli.
7. git_url - first git remote of the git repo, only if is_git_repo is true.
8. uncommited_changes - if its already a git repo then are there any uncommitted changes that are not pushed to remote git. In first version this is not used as we only allow deploying from folder that are not git repo
9. rill_org_exists_as_git_user_name - If a rill org already exists as the git_user_name, this is useful because if no orgs exists for user then we auto create an org with git_user_name, so if this is true then ask user for a new org name. This is applicable only for new user those who don’t have any org. For existing user see next option.
10. rill_user_orgs - User can choose a rill org to deploy to.
11. local_project_name - Folder name of local project from where rill start is done
PushToGit - This is only used when is_git_repo is false and all other validation passes. This will create a git repo in the existing project folder from where rill start is done, auto commit all changes and push the project to git org named git_user_name in a project named the folder name of the local project.
Deploy - Should be done when all DeployValidation passes and it accepts two parameters rill_org and rill_project_name . We will ask user to enter these, can be auto populated as per DeployValidation response.

pjain1 · 2024-05-22T13:20:47Z

Added few more fields to DeployValidationResponse -
1. is_user_app_installed - If rill git app is installed on users personal account, may not be needed if user is deploying from another git organization just is_github_connected should be true
2. user_app_permission - if user has provided read or write access to the rill git app on their personal account
3. git_remote_found - if is_git_repo is true then if there exists a github remote
4. is_repo_access_granted - when the local project has git remote meaning is_git_repo is true and git_remote_found is also true then user should have give access to repo by authorizing app for the repo, if not redirect to git_grant_access_url
5. organizations_with_app- List of orgs on which user has installed the rill app along with permission given. If user choses to push to one of these or deploy to one of this then we should - first have app installed and write permission for push and read permission for deploying. This list will not contain users' personal account name, git_user_name and user_app_permission can be used for this.
uncommitted_changes is change to enum and now have UNCOMMITTED_CHANGES_UNSPECIFIED to indicate we do not know the status so user should ensure repo is in sync with remote.
PushToGit API also accepts org and repo name, this should only be done when git_remote_found is false. These are optional - if not set then default with user' personal org with name as local project name. If set then org should have git app installed with write permission, can be ideally checked with organizations_with_app field of DeployValidationResponse
DeployResponse now returns frontend_url which is the dashboard url and user can be redirected to it after api returns successfully

pjain1 · 2024-05-22T14:26:30Z

One issue I am seeing currently is that gitClient.Organizations.List(ctx, user.GithubUsername, nil) does not list all my orgs however when I explicitly do s.admin.Github.AppClient().Apps.FindOrganizationInstallation(ctx, "<org>") for that org it works. Fixed

proto/rill/admin/v1/api.proto

proto/rill/local/v1/api.proto

admin/server/github.go

begelundmuller · 2024-06-06T09:12:46Z

cli/cmd/deploy/deploy.go

+
+	repoOwner := ""
+	if len(candidateOrgs) == 0 {
+		ch.PrintfWarn("\nRill app does not have permissions to create Github repository on any of your accounts. Visit %q to grant access.\n", pollRes.GrantAccessUrl)


Suggested change

ch.PrintfWarn("\nRill app does not have permissions to create Github repository on any of your accounts. Visit %q to grant access.\n", pollRes.GrantAccessUrl)

ch.PrintfWarn("\nRill does not have permissions to create a repository on your Github account. Visit this URL to grant access: %s\n", pollRes.GrantAccessUrl)

begelundmuller · 2024-06-06T09:17:50Z

cli/cmd/deploy/deploy.go

+		return nil
+	} else if len(candidateOrgs) == 1 {
+		repoOwner = candidateOrgs[0]
+		ok, err = cmdutil.ConfirmPrompt(fmt.Sprintf("Rill has write access to %q Github account. Do you want to use this account?", repoOwner), "", true)


Suggested change

ok, err = cmdutil.ConfirmPrompt(fmt.Sprintf("Rill has write access to %q Github account. Do you want to use this account?", repoOwner), "", true)

ok, err = cmdutil.ConfirmPrompt(fmt.Sprintf("Rill will create a new repository in the Github account %q. Do you want to continue?, repoOwner), "", true)

begelundmuller · 2024-06-06T09:18:15Z

cli/cmd/deploy/deploy.go

 		if err != nil {
 			return err
 		}
+		if !ok {
+			ch.PrintfWarn("\nIf you want to deploy to another Github account, Please visit %q to grant access.\n", pollRes.GrantAccessUrl)


Suggested change

ch.PrintfWarn("\nIf you want to deploy to another Github account, Please visit %q to grant access.\n", pollRes.GrantAccessUrl)

ch.PrintfWarn("\nIf you want to deploy to another Github account, visit this URL to grant access: %s\n", pollRes.GrantAccessUrl)

begelundmuller · 2024-06-06T09:18:46Z

cli/cmd/deploy/deploy.go

+	} else {
+		repoOwner, err = cmdutil.SelectPrompt("Rill has write access to following Github accounts. Please select one", candidateOrgs, candidateOrgs[0])
+		if err != nil {
+			ch.PrintfWarn("\nIf you want to deploy to another Github account, Please visit %q to grant access.\n", pollRes.GrantAccessUrl)


Suggested change

ch.PrintfWarn("\nIf you want to deploy to another Github account, Please visit %q to grant access.\n", pollRes.GrantAccessUrl)

ch.PrintfWarn("\nIf you want to deploy to another Github account, visit this URL to grant access: %s\n", pollRes.GrantAccessUrl)

begelundmuller · 2024-06-06T09:20:08Z

cli/cmd/deploy/deploy.go

+			return nil
+		}
+	} else {
+		repoOwner, err = cmdutil.SelectPrompt("Rill has write access to following Github accounts. Please select one", candidateOrgs, candidateOrgs[0])


Suggested change

repoOwner, err = cmdutil.SelectPrompt("Rill has write access to following Github accounts. Please select one", candidateOrgs, candidateOrgs[0])

repoOwner, err = cmdutil.SelectPrompt("Select a Github account for the new repository", candidateOrgs, candidateOrgs[0])

begelundmuller · 2024-06-06T09:21:41Z

cli/pkg/local/server.go

+		}
+	}
+
+	// TODO check if any project exists with name localProjectName in any of the users rill org


I don't think this is important, but what do you think?

begelundmuller · 2024-06-06T09:22:05Z

cli/pkg/local/server.go

+	githubRepo, _, err := githubClient.Repositories.Create(ctx, githubOrg, &github.Repository{Name: &repoName, DefaultBranch: &defaultBranch})
+	if err != nil {
+		if !strings.Contains(err.Error(), "name already exists") {
+			// TODO should we retry with new name, append a number to repoName like -1, -2 etc


Yes, I think that was actually also requested by Nishant

begelundmuller · 2024-06-06T09:22:19Z

cli/pkg/local/server.go

+		GithubUrl:        ghURL,
+	})
+	if err != nil {
+		// TODO if project with same name exists - should we retry with new name, append a number to repoName like -1, -2 etc


Yeah, same as comment above

begelundmuller · 2024-06-06T09:23:10Z

proto/rill/admin/v1/api.proto

+  // DEPRECATED: Use organization_installation_permissions instead.
+  repeated string organizations = 5 [deprecated = true];
+  GithubPermission user_installation_permission = 6;
+  map<string, GithubPermission> organization_installation_permissions = 7;


Suggested change

// DEPRECATED: Use organization_installation_permissions instead.

repeated string organizations = 5 [deprecated = true];

GithubPermission user_installation_permission = 6;

map<string, GithubPermission> organization_installation_permissions = 7;

GithubPermission user_installation_permission = 6;

map<string, GithubPermission> organization_installation_permissions = 7;

// DEPRECATED: Use organization_installation_permissions instead.

repeated string organizations = 5 [deprecated = true];

begelundmuller · 2024-06-06T09:26:23Z

proto/rill/local/v1/api.proto

+  rill.admin.v1.GithubPermission github_app_user_permission = 6; // if unspecified then github app not installed on user account
+  map<string, rill.admin.v1.GithubPermission> github_organization_installation_permissions = 7;


Is there a reason to use "app" in the first name and "installation" in the second? Otherwise, could consider making the names more similar, e.g. github_user_permissions and github_organization_permissions.

runtime-local apis for project deployment

0628d83

pjain1 marked this pull request as draft May 15, 2024 20:15

pjain1 requested a review from begelundmuller May 16, 2024 09:44

pjain1 marked this pull request as ready for review May 16, 2024 09:44

add local project name to resp

023b268

pjain1 marked this pull request as draft May 16, 2024 18:34

ericpgreen2 mentioned this pull request May 20, 2024

UI-driven project deployment #4930

Draft

pjain1 added 3 commits May 20, 2024 23:37

Merge branch 'main' into deploy_flow

085ac01

Merge branch 'main' into deploy_flow

8be0d95

handle more cases

301e0f8

fixes

99e34f4

pjain1 requested a review from k-anshul May 22, 2024 14:23

pjain1 marked this pull request as ready for review May 22, 2024 14:24

pjain1 added 2 commits May 23, 2024 18:44

fix user installation issue, list accounts with app access

e5f035a

comment

7c377eb

begelundmuller requested changes May 27, 2024

View reviewed changes

pjain1 added 2 commits May 29, 2024 02:17

review comments

3a87332

backwards compatibility

13b9364

pjain1 requested a review from begelundmuller May 28, 2024 21:01

pjain1 added 2 commits May 29, 2024 21:10

fixes

be424bc

lint

f0d9d34

begelundmuller requested changes Jun 6, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime-local apis for project deployment #4909

runtime-local apis for project deployment #4909

pjain1 commented May 15, 2024 •

edited

pjain1 commented May 22, 2024 •

edited

pjain1 commented May 22, 2024 •

edited

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

begelundmuller Jun 6, 2024

	ch.PrintfWarn("\nRill app does not have permissions to create Github repository on any of your accounts. Visit %q to grant access.\n", pollRes.GrantAccessUrl)
	ch.PrintfWarn("\nRill does not have permissions to create a repository on your Github account. Visit this URL to grant access: %s\n", pollRes.GrantAccessUrl)

	ok, err = cmdutil.ConfirmPrompt(fmt.Sprintf("Rill has write access to %q Github account. Do you want to use this account?", repoOwner), "", true)
	ok, err = cmdutil.ConfirmPrompt(fmt.Sprintf("Rill will create a new repository in the Github account %q. Do you want to continue?, repoOwner), "", true)

	ch.PrintfWarn("\nIf you want to deploy to another Github account, Please visit %q to grant access.\n", pollRes.GrantAccessUrl)
	ch.PrintfWarn("\nIf you want to deploy to another Github account, visit this URL to grant access: %s\n", pollRes.GrantAccessUrl)

	repoOwner, err = cmdutil.SelectPrompt("Rill has write access to following Github accounts. Please select one", candidateOrgs, candidateOrgs[0])
	repoOwner, err = cmdutil.SelectPrompt("Select a Github account for the new repository", candidateOrgs, candidateOrgs[0])

		rill.admin.v1.GithubPermission github_app_user_permission = 6; // if unspecified then github app not installed on user account
		map<string, rill.admin.v1.GithubPermission> github_organization_installation_permissions = 7;

runtime-local apis for project deployment #4909

Are you sure you want to change the base?

runtime-local apis for project deployment #4909

Conversation

pjain1 commented May 15, 2024 • edited

pjain1 commented May 22, 2024 • edited

pjain1 commented May 22, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pjain1 commented May 15, 2024 •

edited

pjain1 commented May 22, 2024 •

edited

pjain1 commented May 22, 2024 •

edited