Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: deterministic behaviour of cache installation in prerun #1660

Closed
wants to merge 7 commits into from

Conversation

xabinapal
Copy link
Contributor

@xabinapal xabinapal commented Jul 12, 2021

Ensure cache directory and environment are populated from project_dir and not from current working directory in prerun process.

Fixes: #1654

@ssbarnea
Copy link
Member

@xabinapal I see you have two open PRs that seem to address the same bug. Can you close one?

@ssbarnea ssbarnea added the bug label Jul 13, 2021
@xabinapal
Copy link
Contributor Author

@ssbarnea I see them as two different issues. This one fixes prerun not using the project_dir option and #1661 fixes the guessing of the project_dir (not only during prerun) to fallback to CWD. If you prefer a single PR, I can add that commit to this one.

@ssbarnea
Copy link
Member

There are several problems with this change:

  • the symlink utility function added had no docstring explaining its purpose and even after reading its content I find its signature at least weird as it does receive only one argument and it inferst the other one from tool options, that does not go well with the reusability of the prerun module, which is reused by molecule.
  • there is no real need to pass custom directory location to ansible-galaxy install command because that command will respect what is configured in ansible configured collection/role paths. Probably you seen that we alter these variables in order to inject our custom cache location at the beginning of the path. That ensures that ansible-galaxy command will install them at out desired location, without us having to pass it as an argument.

Keep in mind that we should avoid using options inside prerun module. Instead we should pass things like project_dir on each method. That is because there is no config when used in molecule.

@xabinapal
Copy link
Contributor Author

The symlink function was a refactor of already existing code that was duplicated twice, just to enforce DRY: https://github.com/ansible-community/ansible-lint/blob/master/src/ansiblelint/prerun.py#L409-L412 and https://github.com/ansible-community/ansible-lint/blob/master/src/ansiblelint/prerun.py#L295-L298

I changed it a little bit to check if link_path is not a symlink and raise a friendlier error than the default exception. Admittedly, giving it a second thought, it could be more readable with EAFP. Also, I forgot to add the docstring, thanks for noticing. Refactoring this function could be like this:

def _symlink_galaxy_install(link_path: pathlib.Path) -> None:
    """Helper function to symlink project directory in a custom location"""
    target = str(pathlib.Path(options.project_dir).absolute())
    try:
        if os.readlink(str(link_path.absolute())) != target:
            link_path.unlink()
        else:
            return
    except FileNotFoundError:
        pass
    except OSError:
        raise RuntimeError(
            "Can't replace non-symlink path '%s'" % link_path.absolute()
        ) from None
    link_path.symlink_to(target, target_is_directory=True)

Regarding the usage of options inside this function, I agree that it’s not the cleanest way of doing things. I noticed that options.project_dir and options.cache_dir are used extensively throughout the entire file and not being passed as arguments, so I just followed that pattern. If required, I could refactor this function adding a second parameter to it or, if required, I could try to refactor the entire file.

On the other hand, I don’t really understand what you mean with the second problem. I’ve not changed neither the install_collection function nor theinstall_requirement one, the only ones that launch ansible-galaxy install.

@webknjaz webknjaz added the new Triage required label Sep 13, 2021
@ssbarnea
Copy link
Member

ssbarnea commented Oct 1, 2021

I need to close this because it is in direct conflict with #1666 as we are moving all the bootstraping of the testing environment out of the linter, so we can use it with molecule and also to enable use of execution environments. If the problem is still valid after we make this change, we will have to alter ansible-compat library.

@ssbarnea ssbarnea closed this Oct 1, 2021
@ssbarnea ssbarnea removed the new Triage required label Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Current working directory affects execution environment generated on prerun
3 participants