max file descriptors is too low on AWS EKS

[ksemaev]

So debugged it from very begining on my node:

systcl -a | grep vm.max_map_count
>vm.max_map_count = 262144
cat /proc/sys/fs/file-max
>1636119
ulimit -Hn
>4096

docker pull quay.io/pires/docker-elasticsearch-kubernetes:6.2.3
systcl -a | grep vm.max_map_count
>vm.max_map_count = 262144
cat /proc/sys/fs/file-max
>1636119
ulimit -Hn
>4096

/run.sh
>...
>ERROR: [1] bootstrap checks failed
>[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

I believe that you should specify in prerequsites ulimit setup

[ksemaev]

In my case it was defaut docker setup on EKS, had to do on node (or add to AMI):
sed -i -e 's/1024:4096/65536:65536/g' /etc/sysconfig/docker

[pires]

Thanks for the suggestion. I'll leave this here for future reference.

[gageorsburn]

Out of all of the cat memes on the internet, there exists only one issue for this and it was extremely difficult to find.

[pires]

Hopefully, this tweet will help.

[slavaaaaaaaaaa]

@ksemaev just wondering, what were the symptoms of this?

[ksemaev]

@smaslennikov elastic was starting log with that warning message

[maauso]

Hi,

we found the same problem using the last amazon ami (ami-0440e4f6b9713faf6), but if we use the previous one (ami-0b2ae3c6bda8b5c06), we didn't find this problem.