No longer able to pull config from live on "Too many open files" error

Workaround

Do not use config pull (cpull), just sync down the database and config export instead:

ddev drush sql-sync @live @self
ddev drush cr
ddev drush -y cex

Fix

The fix will probably have to be done by Siteground.

On the live server instance (ssh -p 18765 u7-5w7gvgfcdysg@c54895.sgvps.net), ulimit -n -S reports 1024 as the maximum number of open file descriptors.

The number of configuration files is 1867 currently— and while 1867 is more than 1024, the number of config file we were trying to pull was only 10 more than last time they were committed, so the reason for getting this error now remains unclear, unless something else changed on the live instance side. Presumably not every exported config file counts as an open file, and perhaps something else on the live site is holding more files ‘open’ and the about 120 new or changed files in the current export put it over the top. We do not have access to the tools on the siteground instance (such as lsof) to identify the source of the open files.

The most expedient fix is to ask Siteground to increase this limit.

On most systems we have soft open file limit (ulimit -Sn) of 1024, but non-root processes can change the soft limit and the hard limit (ulimit -Hn) is 1048576— ulimit -S simply reports unlimited.

Here is a general introduction to open file limits that may be useful to Siteground support staff to get clear on what we are asking: https://bobcares.com/blog/too-many-open-files/

Symptom

The error, when trying ddev drush cpull @live @self:

   [warning] include(/home/customer/www/vhfa.org/public_html/core/lib/Drupal/
  Component/FileSecurity/FileSecurity.php): failed to open stream: Too many o
  pen files ClassLoader.php:571
   [warning] include(): Failed opening '/home/customer/www/vhfa.org/vendor/co
  mposer/../../public_html/core/lib/Drupal/Component/FileSecurity/FileSecurit
  y.php' for inclusion (include_path='/home/customer/www/vhfa.org/vendor/pear
  /archive_tar:/home/customer/www/vhfa.org/vendor/pear/console_getopt:/home/c
  ustomer/www/vhfa.org/vendor/pear/pear-core-minimal/src:/home/customer/www/v
  hfa.org/vendor/pear/pear_exception:.:/usr/local/php74/pear') ClassLoader.ph
  p:571
   [warning] include(/home/customer/www/vhfa.org/vendor/symfony/console/Event
  /ConsoleErrorEvent.php): failed to open stream: Too many open files ClassLo
  ader.php:571
   [warning] include(): Failed opening '/home/customer/www/vhfa.org/vendor/co
  mposer/../symfony/console/Event/ConsoleErrorEvent.php' for inclusion (inclu
  de_path='/home/customer/www/vhfa.org/vendor/pear/archive_tar:/home/customer
  /www/vhfa.org/vendor/pear/console_getopt:/home/customer/www/vhfa.org/vendor
  /pear/pear-core-minimal/src:/home/customer/www/vhfa.org/vendor/pear/pear_ex
  ception:.:/usr/local/php74/pear') ClassLoader.php:571
   [warning] include(/home/customer/www/vhfa.org/public_html/core/lib/Drupal/
  Core/Utility/Error.php): failed to open stream: Too many open files ClassLo
  ader.php:571
   [warning] include(): Failed opening '/home/customer/www/vhfa.org/vendor/co
  mposer/../../public_html/core/lib/Drupal/Core/Utility/Error.php' for inclus
  ion (include_path='/home/customer/www/vhfa.org/vendor/pear/archive_tar:/hom
  e/customer/www/vhfa.org/vendor/pear/console_getopt:/home/customer/www/vhfa.
  org/vendor/pear/pear-core-minimal/src:/home/customer/www/vhfa.org/vendor/pe
  ar/pear_exception:.:/usr/local/php74/pear') ClassLoader.php:571
   [warning] Unknown: failed to open stream: Too many open files Unknown:0
  PHP Fatal error:  Unknown: Failed opening required '/var/lib/sec/primary.ph
  p' (include_path='/home/customer/www/vhfa.org/vendor/pear/archive_tar:/home
  /customer/www/vhfa.org/vendor/pear/console_getopt:/home/customer/www/vhfa.o
  rg/vendor/pear/pear-core-minimal/src:/home/customer/www/vhfa.org/vendor/pea
  r/pear_exception:.:/usr/local/php74/pear') in Unknown on line 0

See also