why so many open file descriptors to the same file?

[withinboredom]

Description

I noticed a large jump in open file descriptors after installing Typesense. I notice that each thread creates it's own file descriptor for all files it manages (logs, etc) instead of sharing descriptors. This is over 16k open file descriptors on our server, not including socket connections. To put it in perspective, the only process using more file descriptors is netdata which has nearly every /dev and /proc file open for monitoring.

Steps to reproduce

1. Install typesense.

Expected Behavior

For files to be opened once and shared between threads.

Actual Behavior

Each thread manages its own open files, and there are a lot of them.

Metadata

Typesense Version: 26

OS: Linux

[kishorenc]

@withinboredom

How are you checking for open files? If you are using lsof, that will duplicate fds across all threads even if only a single thread is opening a file. Instead you should check via

ls /proc/$pid/fd/ | wc -l

[withinboredom]

Ah, yes! I was using lsof and notice that my monitoring does as well. I will open an issue there because as you pointed out, this is not a good way of counting them :)