eDirectory core dump files are consuming all disk space

  • 7023055
  • 06-Jun-2018
  • 06-Jun-2018

Environment

eDirectory 8.8 SP8
eDirectory 9
Open Enterprise Server 2 (OES 2) Linux
Open Enterprise Server 11 (OES 11) Linux
Open Enterprise Server 2015 (OES 2015) Linux
SUSE Linux Enterprise Server 12 Service Pack 2 (SLES 12 SP2)
SUSE Linux Enterprise Server 11 Service Pack 4 (SLES 11 SP4)
SUSE Linux Enterprise Server 10 Service Pack 4 (SLES 10 SP4)

Situation

SLES 10 and SLES 11, out of the box, are set to not allow application core dumps to be created.  If the creation of core dumps is enabled, the default is then to create the core file within the process' own data directory and append the PID number to the name.  For the NDSD process the directory would be /var/opt/novell/eDirectory/data/dib.

Over time, if disk space is limited and multiple large core files are created, an out of disk space event can occur.  If it does NDSD will core and its database, unable to process the RBL, can possibly get corrupted.

Resolution

In order to prevent this it is recommended to direct core dumps to an alternate disk storage location, preferably one on its own disk or partition.

1. Configure the system to create application core dumps on another volume, possibly dedicated to core dump files.

and\or

2. Configure the system to not append the PID number.  This will result in all cores getting the same name, thereby, overwriting the previous core file.  Keep in mind that the solution below will affect all application core dumps.  Consult your SLES documentation on how to make these settings specific to one or more applications.

Additional Information


Detailed Steps

Note: the following information is for SLES 10 and 11.

In a production environment this server would be getting a new disk drive or dedicated partitioned space in which to save its larger temporary files such as application or kernel cores.  For this exercise we will simply place the core files in a different directory and specify that only one will be saved. 

1. Confirm core dump creation is enabled.

Type ulimit -c on the command line.  If unlimited is returned the system is ready to create the core files.  If 0 is returned none are allowed.  To enable this the /etc/profile can be edited and the following line added:
ulimit -c unlimited
Then reboot the server.

2. Configure Application Core Dump Location and force the creation of only one core file at a time

As mentioned the default location for eDirectory is in /var/opt/novell/eDirectory/data/dib.  Edit the /etc/sysctl.conf file.  Place the following line in the file:
kernel.core_pattern=/var/local/dumps/core

This specifies that all application core files will be placed in /var/local/dumps/ and named "core" with no extension.  Reboot the server. 



Verification:

cat /proc/sys/kernel/core_uses_pid should return 0.
cat /proc/sys/kernel/core_pattern should return the directory and file name that cores will use.  In this case the return is: /var/local/dumps/core


Additionally a simple C program can be used to generate a SIGSEGV and a resulting core file.

- Create a sigsegv.c file with only the following line:
int main=0;

- Compile the program:
gcc -o sigsegv sigsegv.c

- Make sure that sigsegv is flagged executable then run the program
./sigsegv

There are two things to note:
1. It should say Segmentation fault (core dumped).  The message "core dumped" indicates a core file was actually created.
2. There should be one core file named core in /var/local/dumps and there should only be one file no matter how often the program is run.