Shiny app does not load if repository unavailable while all libraries are cached

[sybrohee]

A few weeks ago, we had a major issue on our professional apps as the posit package manager (https://packagemanager.posit.com) was experiencing a major issue for a few hours.

This was a big problem for us as some of our apps, specifically those using BioConductor libraries were not able to load properly. We somehow solved the problem temporarily by specifying another mirror (i.e. https://cloud.r-project.org) in the renv.lock file.

This is the kind of output I had two weeks ago :

Error: package 'BiocManager' is not available
Traceback (most recent calls last):
15: renv::restore()
14: renv_bioconductor_init(library = library)
13: renv_bioconductor_init_biocmanager(library)
12: install("BiocManager", library = library)
11: retrieve(packages)
10: handler(package, renv_retrieve_impl(package))
 9: renv_retrieve_impl(package)
 8: withCallingHandlers(renv_available_packages_latest(package), 
        error = function(err) stopf("package '%s' is not available", 
            package))
 7: renv_available_packages_latest(package)
 6: stopf("package '%s' is not available", package)
 5: stop(sprintf(fmt, ...), call. = call.)
 4: .handleSimpleError(function (err) 
    {
        ...
    }, "package 'BiocManager' is not available", base::quote(NULL))
 3: h(simpleError(msg, call))
 2: stopf("package '%s' is not available", package)
 1: stop(sprintf(fmt, ...), call. = call.)

However, I still don't understand why this issue arise (as BiocManager is in cache) and how I can make sure that will not happen again.

1. Why does the app has to connect to an external repo if it has everything stored in cache?
2. Why does it only affect applications making use of BioConductor libraries (I tend to incriminate BiocManager, not BioConductor itself).
3. Would it be possible to specify more than one URL for the repositories defined in the renv.lock file? It would be something like this :

      {
        "Name": "CRAN",
        "URL": ["https://packagemanager.posit.co/cran/latest",  "https://cloud.r-project.org"]
      }

Many thanks for your help.
Regards.

[kevinushey]

However, I still don't understand why this issue arise (as BiocManager is in cache) and how I can make sure that will not happen again. Why does the app has to connect to an external repo if it has everything stored in cache?

I believe this occurs because renv is trying to install the latest-available version of the BiocManager package, but in order to determine what that version actually is, we need to reach out to the active package repositories to ask what versions are available.

A better solution for renv here may be to use the version of BiocManager recorded in the lockfile (if any) -- that would allow us to bypass the need to resolve the required version.

For background, renv uses the BiocManager package for figuring out things like the Bioconductor version appropriate for use with the current version of R, and the repositories to be used when installing packages from Bioconductor.

Why does it only affect applications making use of BioConductor libraries (I tend to incriminate BiocManager, not BioConductor itself).

I'm less certain in this regard, but I believe this was ultimately due to an issue on the package manager side. Since BiocManager couldn't be retrieved, and renv uses BiocManager as part of the flow for installing packages from Bioconductor, that led to this issue.

Would it be possible to specify more than one URL for the repositories defined in the renv.lock file? It would be something like this :

While you can't associate multiple repositories with a single name, you can define multiple repositories in R like so:

options(repos = c(PPM = "https://packagemanager.posit.co/cran/latest", CRAN = "https://cloud.r-project.org"))

If you later call renv::snapshot(), then you will see the following field in the lockfile:

    "Repositories": [
      {
        "Name": "PPM",
        "URL": "https://packagemanager.posit.co/cran/latest"
      },
      {
        "Name": "CRAN",
        "URL": "https://cloud.r-project.org"
      }
    ]

and renv will try each of these repositories in turn when attempting to install packages which are not available in the cache, and which should be retrieved from a package repository. This would effectively allow you to fall back to the default CRAN repository if something was not available from PPM for some reason.

[sybrohee]

Dear Kevin,

Thank you for your fast answer. I will test it as soon as possible.

[sybrohee]

@kevinushey
Dear Kevin,

We tested your idea of specifying two different repositories and it seems to work as expected. Many thanks again.

However, as of today, the bioconductor archive was not working anymore we ran into another problems as renv absolutely wanted to connect to the Archive... an again, our shiny app were stuck.

Interestingly, forcing the use of a new version of BiocManager solved the problem but we still do not really understand why. Doing this, we saw that In the renv.lock file, some bioconductor repositories were removed (BioCsoft, BioCsann, BioCexp, ,..).

What would really be great is to have :

• a way not to connect to anything external provided everything is in cache
• in case an external repository is not available, the timeout should almost be immediate (or at least, we could set ut).

Thanks a lot for your help