fix: improve git file discovery with untracked and removed files

[xabinapal]

Closes #1647.

Uses two distinct git ls-files commands to obtain a full list of lintable files, including those not yet added to staging area, both new files and removed files.

[ssbarnea]

I will need some extra feedback for this change from others as I have bit of mixed feelings.

For example, if someone is producing temp files that are untracked and they forgot to add the to gitignore, the linter will start looking at them. Believe me or not, I always encounter projects that forget to do so, usually they install collections/roles someone inside the git tree.

It could have being cool to have it a single command.

[xabinapal]

IMHO, no linter should have different behaviour depending on whether it is executed on a git-enabled repository or not. At most, .gitignore can be taken into account (if I don't want a file to be commited, why would I want to lint it?) but, except that, the subset of lintable files should be the same in both scenarios to avoid confusion between users.

Maybe I'm being too naive, but I expect that if somebody forgot to gitignore some paths, those paths will end up being commited and linted anyway, manually or in a CI pipeline.

What bothers me is that, when developing a complex collection with many files, I refactor them in a regular basis: changing filenames or creating new files to group related tasks in different ways. Having to git add those files every time is too error prone.

However, if this is finally not accepted, if you ask me, the removed files section should be considered no matter what.

[ssbarnea]

@xabinapal Focus on fixing the current broken checks. I cannot really approve PR that is not passing HA. I am sure that one way or another we will merge this, I do find your change benefic.

In fact the original reason why we started using git to identify the files is related to gitignore. We did not want to force people to maintain a two list of excludes, one for git and once for the linter. Parsing the .gitignore file is far from easy, so we ended up reusing git. At that time some people complained about not supporting other VCS but the reality is that they are so few, that they can get away using the fallback find instead.

Shortly, I do support your change, I almost daily make a mistake related to that bug, but CI is reporting back.

[xabinapal]

Sorry to ask @ssbarnea but do I need to fix anything? I see all checks have passed after your merge. Also, I've just run them in my own fork without that merge commit and they are also passing, I don't know why they did fail before.

[ssbarnea]

@xabinapal Thanks! I hope to see more fixed from you.