Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Stats): allow viewing stats without CLI #17

Open
maxpatiiuk opened this issue Feb 10, 2024 · 1 comment
Open

feat(Stats): allow viewing stats without CLI #17

maxpatiiuk opened this issue Feb 10, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@maxpatiiuk
Copy link
Owner

maxpatiiuk commented Feb 10, 2024

The stats generation process relies heavily on having access to the git executable (using the simple-git library).
However, that is not the most user-friendly way, and cuts off this feature from many who are not aware of it, or who are not comfortable with a CLI.
Investigate if there is a way to compute the repository stats from the extension.

GitHub's API should allow getting the list of tags
Getting a list of files and date edited for each file is possible, but would be really expensive (my personal text-hoarder-store has 10k files already - that's A LOT of API calls)

Other options:

@maxpatiiuk maxpatiiuk added the enhancement New feature or request label Feb 10, 2024
@maxpatiiuk
Copy link
Owner Author

Update:
Was able to use a compression stream to download gzip file and uncompress it
The resulting file is a .tar archive - was able to unpack that using a few libraries
However, the archive does not preserve the modified date for files, nor does it preserve the .git folder, making stats extraction harder.
The .zip github endpoint similarly returns an archive without modified dates preserved

The git tree and git blob github API endpoints can return many entires at once, but they don't include create/modified date
The git commit endpoint contains created/modified date but would require making an API call per each commit (10k requests in case of my personal text-hoarder-store)
I do not want to abuse github's api

There is isomorphic-git, a web-based git client, but due to github's cors issues, it requires a proxy server to download a repository - I don't want that burden.


Solutions:

  • Use github's .tar.gz endpoint and provide limited stats (without tags or created dates). I can still get the year created as it's part of the file path, and it happens to be the most important date. And then the web page would tell users that more stats are available in the CLI and give them docs for the CLI
  • Keep it simple for now - I definitely won't be using the limited web-based stats when a cli is available. Plus, a CLI would provide other valuable features like file pre-processing for TTS so CLI would be useful. And the maintenance burden of having web-based stats and cli stats is extra overhead.

Thus, I will push my code for working with .tar.gz to a separate branch for now, and instead create good docs for working with the CLI. The extension does require some technical knowledge after all, since it's dealing with github repositories, and that was my intention from the beginning - that's the target audience I am going for, as there are already plenty of "reader mode" or "pocket" type extensions that very closed ecosystem and limited or even paid, but don't require technical knowledge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: 🆕 New
Development

No branches or pull requests

1 participant