Potential panic when OrderMatters = false and connection lost whilst handler in progress

[MattBrittan]

From PR #504 - logging here so this is more visible (note that the PR will not be accepted as it stands because it sends the ACK before the handler terminates).

When OrderMatters = false there is a potential panic due to this line (panic will occur here or here); in summary the sequence of events is:

1. Publish received and go routine started to call handler .
2. Connection drops (or Disconnect called), internalConnLost calls c.stopCommsWorkers() which (via a chain) results in the closure of ackChan
3. Message handler returns and ackFunc(ackChan, client.persist, message is called which attempts to send to the closed ackChan resulting in a panic.

I made major changes for @1.3.0 to try and remove a range of potential deadlocks and will have introduced this issue at that point (@1.2 did not close incomingPubChan which was not really ideal either as it will have left hanging goRoutines). From the comment

// WARNING the function returned must not be called if the comms routine is shutting down or not running
// (it needs outgoing comms in order to send the acknowledgement). Currently this is only called from
// matchAndDispatch which will be shutdown before the comms are"

I can see that I spotted the risk but missed the fact that matchAndDispatch is doing exactly what I was warning about! At the time usage of OrderMatters = false was fairly low so most tests do not set this option; I have encouraged the use of this option since due to the number of issues logged that turn out to be caused by long running message handlers blocking keep alives.

I think the only viable solution is going to be to drop ACK calls if the connection has been lost but note that it is important that matchAndDispatch shuts down cleanly/quickly even if handlers are still running.

[MattBrittan]

I have created a PR that tests for, and should resolve, this issue; would appreciate any comments on this (not all that happy with the way this works but think it will reliably solve the issue without breaking anyone's code). Will leave this as a PR for a week in case someone has time to review.

[MattBrittan]

As there was no feedback I have gone ahead and merged the PR; would appreciate confirmation that @master resolves this issue and does not introduce any further problems.

[MattBrittan]

Closing as v1.3.5 is now out with the fix.