Setting logfile location, swap file, SNMP and vmkcore partition in ESXi

With the release of VMware vSphere 4.1 it has become even more apparent that the next vSphere release will only have an ESXi version. This isn’t world shocking news since VMware had announce this already at VMworld 2009, but somehow many have been triggered to finally make the switch and install ESXi on their ESX hosts. In this post I will show you how to change the location for the logfiles, set a different swap space location, activate SNMP and change the vmkcore partition.

Setting the logfile location

To me, switching to ESXi was also the moment to switch to a diskless install, run ESXi from USB stick and that does bring some changes on how you handle swap files and loggings. I first noticed how much I missed having log files when my ESXi host had a PSOD and I couldn’t trace any loggings after reboot. I really needed to be able to access the logs even after a reboot.

When you run ESXi on local storage, you’ll notice that there are no more vmkernel, vmkwarning logs etc. They have all been merged into one log. In a diskless install there is no disk to write these logs to so you should switch to a syslog server to collect all logs or place them on a datastore. This datastore can be iSCSI, FC or NFS, just like any other datastore but it doesn’t have to be shared between multiple hosts. You could assign a small datastore per host uniquely but I doubt this is a smart thing because you also have to manage all these datastores. In my lab I choose to create a NFS volume on which I collect all logs from all ESXi hosts.

In my home lab I decided to have all logs sent to a syslog server and also have them written to shared storage. Let’s first set the syslog server, in my case a SolarWinds host running in a VM. In the VMware VI Client select your ESXi host, configuration tab and advanced settings. There you see the tree with all options and near the bottom you see: Syslog. Click on it and on the right hand screen you see three options. One of them is called: Syslog.Remote.Hostname. Here you enter the FQDN of the syslog server. After saving these settings by clicking OK, you should receive all log entries on your syslog server.

Syslog.Remote.Hostname = FQDN of syslog server

A second method is writing all the logfiles to a datastore. Make sure that your ESXi host already sees the datastore. In my case it is a NFS datastore called “nfs-syslog” and on it I made directories per each ESXi host. Now again open the advanced settings and the syslog tree. On the right hand side in the Syslog.Local.DatastorePath enter the datastore like this: [nfs-syslog]/esx01/esx01.log. You should almost immediately see the esx01.log file created in your datastore.

yslog.Local.DatastorePath = [datastore]/path/logfile.log

Setting swap space

To set the swap space you again need to go to the advanced settings. Now go to ScratchConfig and on the right hand find the ScratchConfig.ConfiguredScratchLocation. Here you now have to enter the path to your datastore. To refer to the same path as with syslog, now a different notation is used. It should be like: /vmfs/volumes/<datastore>. In my case that is /vmfs/volumes/nfs-syslog/esx01. Now set the ScratchConfig.ConfiguredSwapState option to enabled, by selecting the tickbox. Changes will take effect on next reboot. After reboot you see a few new directories created on the datastore and the uwswap file of 1GB.

ScratchConfig.ConfiguredScratchLocation = /vmfs/volumes/<datastore>
ScratchConfig.ConfiguredSwapState = enabled

Activating SNMP

In ESX you could find an SNMP option in the security profile settings, but that is no longer available in the ESXi version. To still be able to use SNMP, you can active SNMP through the vMA console like this:

Configure: vicfg-snmp –server <ESXi_ip> -c <communityname> -p 5567 -t destination_host>@162/<community name>
Enable: vicfg-snmp –server <ESXi_ip> -E
Show config: vicfg-snmp –server <ESXi_ip> -s
Test config: vicfg-snmp –server <ESXi_ip> -T

Setting VMKcore partition

When running ESXi from local storage a VMKcore partition is created during install and if a PSOD should occur you can retrieve the dump information using the esxcfg-dumppart command: esxcfg-dumppart –log <ESX dump file> or esxcfg-dumppart –L <ESX dump file>.

However when running ESXi from USB stick without local disks it is better to move the core dumps to a datastore. This has to be a VMFS volume, which rules out NFS. Since the vmkcore dump partition has to be available at boot time, software iSCSI is ruled out too. Only hardware iSCSI or FC LUNs are possible.

The following steps are needed to configure the vmkcore partition. In my example I’m using a 10GB LUN provisioned by iSCSI.

Create the LUN

On my shared storage I created a 10GB iSCSI target and assigned it to my ESXi host. Then on the ESXi host you add the iSCSI target. Do a rescan and then add the iSCSI target like you would normally add a new datastore by pressing the “Add Storage” option in the Storage menu on the configuration tab. Choose to add a Disk/LUN and name it something like: vmkcore-esx01. After a rescan the LUN should be available in your storage view.

Change the partition type

Now the datastore needs to have the disk type changed. To do this you will have to logon to the ESXi host using tech support mode. After you are logged in, list all partitions using the fdisk -l command. You will now see a list of partitions in which you should search for your 10GB disk. In my case it looked like:

Disk /dev/disks/naa.5000144f33903730: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Now to change the partition type, run fdisk /dev/disks/naa.5000144f33903730 (copy it from your fdisk list). In fdisk hit “t” to change the partition’s ID, then hit “fc” to change the partition type to VMKcore. Now hit “w” to write the partition table and exit fdisk.

Set and activate the partition

The last step is now to tell ESXi to use a new vmkcore partition using the following command. First we double check for suitable vmkcore partitions:
esxcfg-dumppart –f
If the fdisk action went well, you should now see the /dev/disks/naa.5000144f33903730 partition again in the list. To set the partition use the following command:
esxcfg-dumppart -s naa.5000144f33903730:1

Now, after playing with this in my lab for over 2 hours I received the message: “Unable to set dump partition naa.5000144f33903730:1. Error Message was: Unsupported disk type: Software iSCSI LUNs are not supported”. So this last part is flying blind on the docs.

Last step is now to activate the partition using the following command:
esxcfg-dumppart -a naa.5000144f33903730:1

Reading the dump file

After a PSOD has occurred log in to the ESXi host using Tech Support mode. First step is to list the dump partition that is active and then copy the dump to a different volume and extract the logs.

–          esxcfg-dumppart –l

–          esxcfg-dumppart –copy –devname /vmfs/devices/disks/naa.xxxxx:x –newonly –zdumpname /vmfs/volumes/nfs-StorCent03/esxdump/esxdump

–          cd /vmfs/volumes/nfs-StorCent03/esxdump/esxdump

–          esxcfg-dumppart –L /vmfs/volumes/nfs-StorCent03/esxdump/esxdump

You will now find a vmkernel-log.1 file that you can use to examine why the PSOD happened.