Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process open fd table is full #38

Open
traufeisen opened this issue Mar 29, 2018 · 3 comments
Open

process open fd table is full #38

traufeisen opened this issue Mar 29, 2018 · 3 comments
Assignees

Comments

@traufeisen
Copy link

During sendRequest the Client checks for the existence of a curl handle in the handle property.
If there is one, it calls curl_reset on it and continues with that handle.
The only way to destroy a curl handle is with the destructor of the Client.

The problem with this is how curl_resetworks.
This function is only resetting the options, but not the underlying socket.
From the curl manual

It does not change the following information kept in the handle: live connections, the Session ID cache, the DNS cache, the cookies and shares.

In long running scripts this leads to more and more open sockets in CLOSE_WAIT status.
Depending on the remote servers.
Until all allowed file descriptors are in use.

The workaround is to construct a new Client object for every request, so the destructor gets called when a request is finished.
A better way would be to explicitely close the handle with curl_close after each request and use a new one for the next request.

@mekras mekras self-assigned this Mar 29, 2018
@mekras
Copy link
Collaborator

mekras commented Mar 29, 2018

OK, I'll check this in a few days.

@bagaskarawg
Copy link

Seems like this issue is what caused this problem to arise in our case:
mmap cache can't open ... - Too many open files (pid 4690)

We are currently using this package which is provided by default by this package: https://github.com/geocoder-php/GeocoderLaravel. We are currently processing quite a bunch of request of around 10,000 requests each day and it stopped at around 400-500 requests, after searching for a solution online, I've found that it was due to the ulimit -n value which is currently set to 1024.

@pdeans
Copy link

pdeans commented Jun 1, 2021

In case this helps anyone else experiencing this issue, we solved this error in our application by setting the Connection: close header on the Client requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants