Paradox when pool is in stopped pools data structure but is fully started

[mulkieran]

We can engineer this state by:

1. Corrupting Stratis LUKS metadata so that a device is recognized as a Stratis device but has no valid encryption information to unlock a pool.
2. Restarting stratisd.

stratisd detects that there is no valid encryption information and puts the pool into stopped pools data structure. But, because the pool was not taken down previously, it is fully started and functional. If it were torn down, then it would not be possible to just restart it, due to the invalid LUKS info. It seems like it would not help the user to tear it down, on the other hand, stratisd is reporting the fully functional pool as stopped. We believe that stratisd attempts to teardown the pool, but can't if there are filesystems present on the pool, because they hold the underlying pool devices, and the teardown of the thinpool device, for example, will return a failure, probably an EBUSY.

Probably, in the best case, we should put the pool into a reduced availability state and send out a warning. We've proposed NO_IPC_REQUESTS, because that will allow adjustments of the pool if filesystems are written to, etc.