glob paths to ignore files using .gitignore

[jayvdb]

Feature Description

glob paths set using env should not include files which are excluded by .gitignore.

c.f. rust-lang/cargo#2063

Describe The Solution You'd Like

Files excluded via .gitignore are excluded by default from env globs.

Or env globs have include/exclude overrides and docs highlight that 'hidden' files may be included unless exclude is used.

Code Sample

https://github.com/rust-lang/cargo/blob/58a961314437258065e23cb6316dfc121d96fb71/src/cargo/sources/path.rs uses https://crates.io/crates/ignore

https://github.com/jhspetersson/fselect/tree/a1d17018f70c79af17df458f7a2878fb422d7da8/src/ignore includes support for .dockerignore and .hgignore

[sagiegurari]

can you explain your use case and provide a makefile example of what you are doing that requires this?
in many tasks i actually work on files that are in the ignore (meaning files in the target folder), so i'm trying to understand better.

[jayvdb]

My use case atm is selecting files to be linted, specifically Python and QML files.

[sagiegurari]

can you show an example task toml? so I'll understand how you trigger that? because i'm trying to understand how you pull the files to lint in the first place.

in rust for example, these tools are project aware and you don't need to pass anything

[sagiegurari]

@jayvdb can you provide an example of how the usage would be?
how would you use it in a task? how would you see it defined?

[jayvdb]

We are not trying to use this for rust tools. It is other languages, and tools, that we need to do processing of in steps intermingled with the rust build process, as the rust code depends on them.

[sagiegurari]

@jayvdb i got that point already when you said linting py files.
what i'm asking is how you expect it to look like as a user? how will you actually use it.

the solution you requested is:

Files excluded via .gitignore are excluded by default from env globs.

but there is no env globals in cargo-make so your solution is not clear for me.
give me an example of a task definition that meets your needs so i'll have better understanding.

[jayvdb]

sorry env globs should be read as just globs (anywhere the globbing code currently is in effect) - not "globals".

[sagiegurari]

again there is no glob handling in cargo-make at the moment and i didn't get any feedback fromyou on how you see a solution.
so for now, if you reinstall cargo-make you will have a new duckscript command gitignore_path_array which will return an array of paths (its based on the ignore crate used by cargo as well).
you can than loop over those paths, filter by type or by file and so on, and call your command with the paths as args.

[sagiegurari]

@jayvdb i'm thinking of an internal cargo make way to defining env vars with globs and respecting gitignore.
something like

[env]
MY_C_FILES = { glob = "./src/**/*.c", ignore_type="git", include_files = true, include_dirs = false}

[tasks.print-c-files]
command = "echo"
args = ["@@split(MY_C_FILES,;)"]

Might take me a bit, but once done, you will have the ability to run commands on sets of files without any scripting.

[jayvdb]

Thanks for gitignore_path_array. That is a great help.

The proposed builtin support for globbing looks great, however I dont understand the @@split - I guess that is invoking duckscript split ?

I guess then we should also have the separator be supplied with the glob MY_C_FILES = { glob = "./src/**/*.c", separator = ";" ... }.

IMO the separator default for multi-entry env vars should be \n, if duckscript can support that.

[sagiegurari]

no the split is not related to duckscript.
there are built in functions in cargo-make that enable you to manipulate the command arguments dynamically.
see: https://github.com/sagiegurari/cargo-make#usage-functions-split

list env vars in cargo-make use the ';' separator, and than with the split by it, you will get many arguments (one per path) to the command.

[sagiegurari]

@jayvdb this is now available in the dev branch 0.32.17 can you please verify it works well for your needs?
Simple example:

[env]
PY_FILES = { glob = "./src/**/*.py", include_files = true, include_dirs = false, ignore_type = "git" }

[tasks.echo_py]
command = "echo"
args = ["@@split(PY_FILES,;)"]

[sagiegurari]

this is now officially released