Mepo style for local name of repos

[amdasilva]

The current default, hard coded in components.yaml, is to have remote repo noted with a @ prefix, e.g.,

ecbuild:
local: ./@cmake/@ecbuild
remote: ../ecbuild.git
tag: geos/v1.0.6

Some of us would much rather have @ as a suffix so that the @ never gets typed because of tab completions. The suggestion is to change the syntax of the components.yaml file to something like:

ecbuild:
local: ./%cmake/%ecbuild
remote: ../ecbuild.git
tag: geos/v1.0.6

where %denotes an external repo. The following command line option is added to mepo:

-s, --style naked|prefix/suffix|verbose

allowing the local name to have user selectable styles:

naked: ./cmake/ecbuild
prefix: ./@cmake/@ecbuild
suffix: ./cmake@/ecbuild@
verbose: ./cmake (repo)/ecbuild (repo)

For convenience, this choice could be encoded in $HOME/.mepoconfig along with other mepo specific aliases.

    Arlindo

[mathomp4]

@amdasilva How necessary is the verbose method? Passing quoted strings to subprocess to retain the space seems to be quite exciting. Some people say to use shell=True but I really do not want to do that because of security considerations. I'll keep trying to figure it out, but I might have to re-write a good chunk of mepo to support it as we use split a lot.

[tclune]

@mathomp4 can I ask you to elaborate a bit? Presumably where we use split it is not on spaces, as we currently don't use spaces. The spaces in this case should all be in the middle of a string and normal operations should not be stripping them. I suppose the os utilities that construct paths might have a bias against embedded spaces as using them on the command line requires an escape?

I do agree that this seems to be lower priority than the other cases.

[tclune]

@amdasilva Can we leave the "raw" repo indicator as @ instead of %? By doing this, we would avoid any need to change the existing fixtures (except AeroApp)? I suppose it is possible that this change to mepo will allow the older configuration.yaml files to still function, in which case I withdraw my concern. Otherwise it is a hassle to coordinate the various changes.

[mathomp4]

@amdasilva Can we leave the "raw" repo indicator as @ instead of %? By doing this, we would avoid any need to change the existing fixtures (except AeroApp)? I suppose it is possible that this change to mepo will allow the older configuration.yaml files to still function, in which case I withdraw my concern. Otherwise it is a hassle to coordinate the various changes.

@tclune @amdasilva Actually, we don't need to change anything at all. I have some code that doesn't care how the components.yaml is set up. This is probably more of a quirk of mepo (and I say requirement) but we know that we are checking out the repo in the last node of the local: key. So, the last node of the value of the local: key is the "location" of the repo. So, I made up some code that can just use that last node and figure things out. Seems to work, though a bit slow.

[mathomp4]

@mathomp4 can I ask you to elaborate a bit? Presumably where we use split it is not on spaces, as we currently don't use spaces. The spaces in this case should all be in the middle of a string and normal operations should not be stripping them. I suppose the os utilities that construct paths might have a bias against embedded spaces as using them on the command line requires an escape?

I do agree that this seems to be lower priority than the other cases.

@tclune The issue is that mepo current does things like:

    def clone(self, version, recurse):
        cmd = 'git clone '
        if recurse:
            cmd += '--recurse-submodules '
        cmd += '--branch {} --quiet {} {}'.format(version, self.__remote, self.__local)
        shellcmd.run(cmd.split())

We run split on a command string rather than constructing from lists (which gets very tedious and hard to read). I am going to look at rewriting all the places in the code we run shell commands but it is a LOT since mepo is essentially just running shell commands.

Since we run split, we kill spaces. I've looked at quoting with shlex, but it's not doing what I want. I'll keep looking at it, but I am not a real Python guru. I'm trying and I'll keep trying, but spaces in paths look to be fun.

[mathomp4]

Note also: I know you want a .mepoconfig but that will probably be a separate update. I don't want to do two things at once for this. I'll look at adding the style, first then I'll see what I can do for a mepoconfig.