When creating containers, we have observed errors where a container has complained that there are too many open files

Typical error message

unable to apply config for monitoring/clone-loki: 
unable to create logs instance: failed to make file target manager: too many open files

Resolution

Increase the maximum number of inotify instances allowed for each user. You can view the current limits by running the following command

sysctl fs.inotify

rgadmin@ubuntu:/$ sysctl fs.inotify
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 501604

To increase the max user instances value, say to 512, you can use the following command.

sudo sysctl fs.inotify.max_user_instances=512

In order to persist this change across reboots you will need to edit the /etc/sysctl.conf file and add the line 

fs.inotify.max_user_instances=512

You can then run sudo sysctl -p /etc/sysctl.conf to load the change.