New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RuntimeException - Unable to write output. #762
Comments
I guess the below already happened when restarting the machine but just checking to be sure. Did you:
Did you upgrade Horizon as well? Did you republish the assets? |
Hi, I restarted the whole server this morning and I have not seen the error again since. So basically all those three were restarted. Horizon is now upgraded to v3.6.1 and I republished the assets just in case. But assets should not matter in this case, since the error is coming from the supervisor console output? If this pops again, is there anything I can do provide additional information? Seems like the Horizon log does not have anything related to this, most likely because it is an exception and it is written in the normal Laravel log. |
Not that I know of. Definitely keep an eye out on log files like nginx, php-fpm, laravel log. |
Anyone else having the same issue. In my case this was a case of free disk space being filled. The exception was different in Bugsnag than in the Laravel log which revealed the culprit. [2020-02-13 18:34:43] production.ERROR: curl_exec(): write of 8192 bytes failed with errno=28 No space left on device {"exception":"[object] (ErrorException(code: 0): curl_exec(): write of 8192 bytes failed with errno=28 No space left o$ |
Hi guys, I am experiencing the same issue intermittently. Seems to have only started in the last couple of days. Horizon Version: 4.0.1 |
Hi there guys, this is happening more and more frequently on my side and I am really having trouble figuring out exactly why. I've included a stack trace below if that helps at all.
|
Yup, happening frequently on my side as well,, not able to figure the root cause at the moment |
My latest theory is that it happens if you manually throw an exception from a queued job, but I am busy adding some more logging to confirm this. |
@robtesch You might be onto something. The jobs that create this error do indeed have manually thrown exceptions in them. |
Well, @lasselehtinen that theory is out the window now unfortunately as i've just got the same error happening on a simple notification class with no manual exception throwing. The new theory is, it's simply whenever there's any kind of exception within a queued job. It's quite frustrating because the actual exception itself is hidden, and we only get to see the "Unable to write output" exception. For what it's worth, I have investigated the case of free disk space being filled, and that is certainly not the case here. System has plenty of disk space, ram, cpu etc available (consistently running at or below 50% usage for all above). |
I've had this recently happening too, still on Laravel 6 with Horizon 3. But it was the first time this happen (twice in a short time frame):
For all I know, the job in question successfully finished but it's parent supervisor was not present anymore. So in my case this is a "one time thing" and I didn't further pursue it. |
I'm having the same issue. Running Laravel 7.0 on PHP 7.4 Happening for both my Scheduler and my Queue Listener. Super odd. I restart them, and it will work for a while no problem. A day or so passes, and it's back to failing again. No idea what could be causing this but will post back if I figure anything out. |
I'll reopen but appreciating any help with this. |
Has anyone reported this to Symfony? |
Yeah, I honestly have no solid solution to this, just happened to run across this. I'll say that I am running on an IIS server for this particular one, so I'm not positive its not somehow caused by the environment it's running in. Just found it odd that others were having this same odd issue. |
I have not reported to Symfony no, it is at the end of the day a Symfony component that is failing to write so would make sense. I can throw something over on their side of things though. |
@JacobBennett (and anyone else on this thread): Nicolas Grekas asked me if you could try with different symfony/console versions to see if the problem was introduced in a specific version. He also said that opening up an issue on the Symfony repo is the best choice. Especially given the comment below at the exception that triggers it: I'll leave this open to follow up. |
Nicolas also said that if a PHP Notice is triggered it would be nice to see what happens there (see the |
@taylorotwell since nobody else appears to have done so yet, i've now submitted this one to Symfony. |
Hey @driesvints I could try doing different console versions but the problem is that it never fails immediately, only after running for a period of time. I have alerts set up to message me as soon as it happens though so will see what we get. I think I might just remove that Also, full disclosure, our business is pivoting along with the rest of the globe to make sure we can handle all the challenges being introduced during this COVID-19 pandemic. This issue for me, while annoying, has a rather simple solution for the time being which is to restart my workers. It's likely I may not be able to prioritize this in the current week but I will get back to this as I can. |
Someone noted on the Symfony issue that the problem is with the latest PHP 7.4 release. So I'm going to close this as it has nothing to do with Horizon itself. Best to follow up on either the Symfony or PHP issue report. |
Thanks @driesvints |
Looks like a fix is in the works symfony/symfony#36222 |
For those following this issue, the fix for this was released on following Symfony releases. So running composer update should update symfony/process. Release v3.4.39 |
Description:
We did an simultaneous upgrade from CentOS 7 to 8 and from PHP 7.3 to 7.4 and we started to get
Symfony\Component\Console\Exception\RuntimeException - Unable to write output
exceptions from the Horizon Supervisor. Sometimes restarting Horizon or rebooting the machine fixes the issue for a while. I have no idea what might be causing this.Stacktrace:
The text was updated successfully, but these errors were encountered: