Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building docs takes >40 minutes on master #557

Closed
effigies opened this issue Aug 16, 2020 · 7 comments
Closed

Building docs takes >40 minutes on master #557

effigies opened this issue Aug 16, 2020 · 7 comments

Comments

@effigies
Copy link
Member

Can we reuse prior builds to avoid building all old versions every time?

@effigies effigies changed the title Building docs takes >20 minutes on master Building docs takes >40 minutes on master Aug 16, 2020
@oesteban oesteban added this to the 1.3.1 milestone Sep 11, 2020
@oesteban
Copy link
Member

oesteban commented Dec 7, 2020

Yeah, this is getting very bad. I believe sphinxcontrib-versioning should actually not build old releases if they are already built - however it is clearly not working.

@effigies
Copy link
Member Author

Is there any advantage to keeping old commits around? I suspect it's gh-pages that's making clones take 1.7 minutes.

@oesteban
Copy link
Member

I think that's one problem - probably solvable with a shallow clone that only fetches master and tags.

I don't think sphinxcontrib-versioning is building old commits, I think it is just building all tags.

@effigies
Copy link
Member Author

But there's no real advantage to keeping history. I would suggest just squashing everything down to a single commit periodically (or even on every build) so that garbage collection can do its work.

@effigies
Copy link
Member Author

History squashed with #622. Still need to build off of the last commit to save time.

@oesteban
Copy link
Member

oesteban commented May 7, 2021

I just added the bug label because I think we are not reusing the existing documents in the gh-pages branch, and hence all the versions are built all the times.

I'm not confident the sphinxcontrib-versioning will reuse them (the package is pretty buggy), but, it will certainly will not if they are just not available :D

oesteban added a commit to celprov/niworkflows that referenced this issue Mar 1, 2022
This new protocol should give an end to the expensive builds on circle CI,
as old versions do not need to be rebuilt with every tag / push to master.

After squashing the history again, and probably cleaning up git objects that
are not used, this should effectively reduce the checkout time by a lot.

Resolves: nipreps#557.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants