Support extending files_to_watch

[soundofspace]

Language modules should be able to extend files to watch. Not sure how this would fit into the current setup, but goal would be that they could register files that are important for direnv and direnvrc script to watch.

Examples would be yarn.lock, requirements.txt... Currently these are only updated when a new shell is opened and might leave the current env in a old/unexpected state. Current workaround is modifying one of the files it watches eg add whitespace to devenv.nix to make sure a refresh is triggered, but this is easy to forget and error prone.

[sandydoo]

Language modules should be able to extend files to watch

Cool idea! We could do something similar to how we handle imports.

devenv/direnvrc

Lines 118 to 122 in 6ee4b6b

	if [[ -f "$flake_dir/.devenv/imports.txt" ]]; then
	for file in $(cat "$flake_dir/.devenv/imports.txt"); do
	files_to_watch+=("$file")
	done
	fi

Current workaround is modifying one of the files it watches

You can watch any file you like with direnv's watch_file. Add this to your .envrc:

watch_file yarn.lock requirements.txt

use devenv

eg add whitespace to devenv.nix to make sure a refresh is triggered

N.B. touch devenv.nix is a quick way to do the same.

[soundofspace]

@sandydoo yea had a similar idea like you did, just not sure how language modules would exposes files to watch, this would somehow require devenv to have already parsed nix files.

You can watch any file you like with direnv's watch_file. Add this to your .envrc:

True, and this will trigger a refresh, but this if statement won't trigger, so doesn't really do much

if [[ ! -e "$profile_rc" || "$need_update" == "1" ]];
then
  log_status "updated devenv shell cache"
else
  log_status "using cached devenv shell"
fi

N.B. touch devenv.nix is a quick way to do the same.

Thought of that as well, but main problem is that this doesn't get checked into git, hence doesn't work for teammembers (which is the main issue).

[sandydoo]

True, and this will trigger a refresh, but this if statement won't trigger, so doesn't really do much

Ah, I forgot we had caching! Maybe nix_direnv_watch_file?

yea had a similar idea like you did, just not sure how language modules would exposes files to watch, this would somehow require devenv to have already parsed nix files.

Bit of a chicken and egg problem. I think there's 3 things that need to happen:

1. Export the list of files to watch. This can either be an env var or a file in .devenv. enterShell is one place to do it. This list will be created after running devenv print-dev-env.
2. Watch files on initial shell creation. Fetch the list of files and watch_file them. This ensures we reload the shell when those files are touched after the first ever evaluation. Maybe this can be made conditional on whether this is the first ever run?
3. Watch files on subsequent shell reloads. Before running use devenv, we read the list of files and add them to files_to_watch.

[soundofspace]

Overlooked that nix_direnv_watch_file was available in envrc, using that before use devenv already works like a charm, thanks for that.

[soundofspace]

I was thinking of implementing it like this:

• Export DEVENV_EXTRA_FILES_TO_WATCH as env variable in nix config (language modules can extend this)
• And use like this (pseudo code)

nix_direnv_watch_file "${files_to_watch[@]}"

local layout_dir profile_rc
layout_dir=$(direnv_layout_dir)
profile_rc="${layout_dir}/devenv-profile$(_nix_argsum_suffix "${files_to_watch[@]}").rc"


if [[ -e "$profile_rc" ]];
  DEVENV_EXTRA_FILES_TO_WATCH=$(read_from $profile_rc)
  nix_direnv_watch_file $DEVENV_EXTRA_FILES_TO_WATCH
fi

local need_update=0
local file=
for file in "${nix_watches[@]}"; do
  if [[ "$file" -nt "$profile_rc" ]]; then
    need_update=1
    log_status "$file changed, reloading"
    break
  fi
done

if [[ ! -e "$profile_rc" || "$need_update" == "1" ]];
then
  # We need to update our cache
  local env
  if ! env=$("${DEVENV_BIN}" print-dev-env); then
    log_error "failed to build the devenv environment. devenv.nix may contain errors. see above."
    exit 0
  fi

  if [[ ! -e "$profile_rc" ]]; then
    DEVENV_EXTRA_FILES_TO_WATCH=$(read_from $env)
    nix_direnv_watch_file $DEVENV_EXTRA_FILES_TO_WATCH
  fi

  echo "$env" > "$profile_rc"
  log_status "updated devenv shell cache"
  _nix_import_env "$profile_rc"
else
  log_status "using cached devenv shell"
  _nix_import_env "$profile_rc"
fi

I'm willing to look into PR if you agree or have a better way

[sandydoo]

I put down some ideas in #1186. Seems to work!

The above should also work. I guess you want to grep out the variable from the script? Only thing to keep in mind is that we need to watch any new files added to the list. The second [[ ! -e "$profile_rc" ]] check would prevent that.

[soundofspace]

Yea I was wrong to assume that any change to files_to_watch would create a new checksum hence a new profile_rc, so would indeed miss some cases.