Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

git show-toplevel should use caching #2344

Open
tjstansell opened this issue Oct 31, 2022 · 1 comment · May be fixed by #2668
Open

git show-toplevel should use caching #2344

tjstansell opened this issue Oct 31, 2022 · 1 comment · May be fixed by #2668
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@tjstansell
Copy link
Contributor

I've been debugging slow startup times for terragrunt and one of the things I've noticed is that the various terragrunt functions that resolve paths relative to the git root all run git rev-parse --show-toplevel. Here's a look at the timing coming out of terragrunt (after i built a custom build to enable microsecond timestamps):

root@d3069cc15eed:/app/prd/accounts/saws2-dev# /app/terragrunt graph-dependencies --terragrunt-log-level=trace 2>tg.out
digraph {
	"/app/prd/accounts/saws2-dev" ;
	"/app/prd/accounts/saws2-dev" -> "/app/prd/org";
	"us-east-1" ;
	"us-east-1" -> "/app/prd/accounts/saws2-dev";
	"us-east-2" ;
	"us-east-2" -> "/app/prd/accounts/saws2-dev";
	"us-west-1" ;
	"us-west-1" -> "/app/prd/accounts/saws2-dev";
	"us-west-2" ;
	"us-west-2" -> "/app/prd/accounts/saws2-dev";
	"us-west-2/main" ;
	"us-west-2/main" -> "us-west-2";
	"/app/prd/org" ;
}
root@d3069cc15eed:/app/prd/accounts/saws2-dev# cat tg.out | awk 'BEGIN{p=0}{if ($0 ~ /^time=/) {split($1,a,":"); n=a[3]; printf "\n%0.3f %s",(n-p),$0; p=n}else{printf "%s",$0}}' | grep 'git '
0.012 time=2022-10-31T17:59:45.975Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev]
0.015 time=2022-10-31T17:59:45.991Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev]
0.011 time=2022-10-31T17:59:46.014Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev]
0.012 time=2022-10-31T17:59:46.090Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev/us-east-1]
0.013 time=2022-10-31T17:59:46.162Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev/us-east-2]
0.012 time=2022-10-31T17:59:46.228Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev/us-west-1]
0.015 time=2022-10-31T17:59:46.301Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev/us-west-2]
0.011 time=2022-10-31T17:59:46.380Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/accounts/saws2-dev/us-west-2/main]
0.009 time=2022-10-31T17:59:46.441Z level=debug msg=git show-toplevel result:  prefix=[/app/prd/org]
root@d3069cc15eed:/app/prd/accounts/saws2-dev# time git rev-parse --show-toplevel
/app

real	0m0.014s
user	0m0.000s
sys	0m0.004s

The git root directory should be static and should only need to be evaluated once, then individual relative directory paths calculated from that. This particular example output is from one of many accounts. Looking at a total runtime across our entire repo, this gets called 679 times and adds roughly 8 seconds to terragrunt startup time and could be easily eliminated.
@denis256 denis256 added enhancement New feature or request good first issue Good for newcomers labels Oct 31, 2022
@RaphSku
Copy link

RaphSku commented Aug 14, 2023

I'd be interested in working on that enhancement

@RaphSku RaphSku linked a pull request Aug 16, 2023 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
Terragrunt Roadmap
Status: To do
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Performance enhancement of GitTopLevelDir in shell pkg RaphSku/terragrunt
3 participants
@tjstansell @denis256 @RaphSku