If you have a VMware environment and you haven’t yet deployed the vRealize Log Insight appliance you really should. While the ‘Log Insight’ platform is a licensed product VMware do provide it with a number of free licenses included so depending on your environment size you may not even need to pay. In this post I’ll go through the steps involved deploying the appliance and configuring ESXi hosts and vCenter to communicate with it.
First off some useful links –
- VMware vRealize Log Insight Information
- VMware vRealize Log Insight Overview, Features, Reources and Purchasing
- VMware vRealize Log Insight Datasheet PDF
I don’t want to pull all the licensing information into this post so go to the following link for the complete details –
Suffice to say you will require a valid vCenter license, a quick summary –
Q: What is the difference between vRealize Log Insight for vCenter Server and vRealize Log Insight?
A: Each vCenter Server customer is entitled to receive a 25-OSI pack of vRealize Log Insight 3.3 for vCenter Server at no charge. Log Insight 3.3 for vCenter Server supports a single Log Insight 3.3 for vCenter Server license and limits Log Insight to a single Log Insight node, a single vSphere integration, Content Packs from the in-product marketplace for only VMware products, and disables enterprise features including event forwarding, archiving, and SSL.
Virtual Appliance Deployment
Now it’s time to start deployment and configuration. I’m going to assume you have downloaded the appliance and selected the OVF for deployment in vCenter. In this case I’ll show the steps via the traditional C# client though of course the web client is going to be the only option moving forward (later this year the C# client will be discontinued).
It’s important to remember that this is a logging and analysis system so disk space requirements will be highly dependent on your setup – i.e the number of systems you want to monitor.
As with all appliances you will be asked to accept a license agreement.
Select an appropriate name for the virtual machine – you can either leave it at the default or change it to the hostname you will configure later. In my case I’m leaving it as the default which is fine for my home lab environment. You can also select an appropriate folder in your vCenter hierarchy.
Now for an important choice, the system requires a decision on how ‘big’ an appliance to configure. Obviously you need to consider the current size of your environment as well as future growth. Along with those variables you need to be mindful of how much compute and storage resource a larger configuration will require.
As this deployment is for my home lab I’m going with the ‘Extra Small’ deployment configuration. The system provides a warning that this config is intended for testing or POC usage – not production. Contrast the ‘Extra Small’ resource requirements to the next screenshot showing a ‘Large’ deployment.
As you can see the ‘Large’ deployment has a fairly considerable resource requirement. It should also be noted that with a fully licensed version you can cluster these appliances for greater availability etc. so in a truly large environment you could invest quite a lot of resource in this sort of monitoring.
The next step is to select a vCenter Cluster to deploy the appliance to – in my lab I only currently have a single cluster with two hosts.
My current setup requires that I manually select an ESXi host to deploy to.
I’d love to run all my virtual machines from solid-state drives however I don’t have the finances for that and with this sort of appliance (lots of writes to the VM) deploying to an SSD would just create wear on the flash cells. For this reason I’m deploying to a standard magnetic spinning disk. As a point of comparison my deployment at work runs on a SAN attached datastore with much higher performance characteristics – required for the size of deployment in those datacenters.
The next configuration decision is whether to deploy the virtual disks as thin or thick provisioned. It is strongly recommended to deploy the disks as ‘Thick Provisioned Eager Zeroed’ (TPEZ) due to the large write workload this appliance will generate. As this is a deployment in a home lab I’m thin provisioning the disk however, if I again contrast with my work datacenter deployment – there the virtual machine disk is TPEZ.
Select the appropriate network setting for your environment – in my case I’ve selected a VLAN I use for infrastructure related systems. It includes my ESXi hosts and vCenter so it made sense for me to add this appliance to the same network.
Now for the final configuration settings. Again these should be configured according to your environment.
The wizard presents a final page confirming the choices made earlier – assuming all is as desired it’s time to hit the ‘Finish’ button and let the system deploy the appliance.
Now that the virtual machine is deployed we can start it up and move on to configuring our environment to use it.
First off I’d suggest checking you have a DNS record for the new virtual machine – how you do this depends on your setup. In my case I just added it to my Microsoft DNS server forward and reverse lookup zones. Once that’s done power up the virtual machine and once booted you should see something similar to the screenshot below on the VM console.
When connecting to the web console a certificate error will be presented as the appliance generates a self-signed certificate. Unfortunately you have to pay for the full license to be able to replace the SSL/TLS certificate – this is something I strongly disagree with. Security shouldn’t be behind a pay wall. That being said, let’s continue.
The web interface now requires us to walk through a wizard configuration process, let’s begin.
First off we are asked whether this is a new deployment or if we’ll be adding to an existing setup. Obviously in my situation this is a new deployment.
Time to setup some credentials.
Once you’ve added your vCenter key you should get a confirmation that it’s active.
Next we need to decide whether to send VMware data for their CEIP along with defining the e-mail notification address and any HTTP notification URLs.
Accurate time keeping is vital in any digital infrastructure and especially when one is dealing with logging. If devices/systems do not share a common time source it can make incident correlation much harder. I’m presenting a few screenshots for this – the first shows the default NTP servers as provided by VMware. Next I’ll show you my configuration which uses the same NTP time source as my ESXi hosts (and other systems) use.
Once we’ve configured time settings it’s time to setup SMTP e-mail notification.
Huzzah the wizard is complete!
Now for the final step…
Configure Devices For Log Insight Communication
Logon to the web console.
The splash screen presents us with a few options – depending on your environment you may wish to configure some or all of these options. In my case I’m just going to use the ‘vSphere Integration’ as an example.
As part of the vCenter integration it’s necessary to provide a set of credentials. Depending on the level of integration you’re after the requirements differ, this link should provide all the info you need – Connect vRealize Log Insight to a vSphere Environment.
In my case I simply created a new vCenter permissions role called ‘vRealize Log Insight’ with no assigned permissions. When you add a custom role and do not assign any privileges to it, the role is created as a Read Only role with three system-defined privileges: System.Anonymous, System.View, and System.Read. If you want vRealize to be able to configure ESXi hosts automatically to talk to it then you need to add ‘Host.Configuration.Change settings’ and ‘Host.Configuration.Network configuration’. In my case I’m happy to configure the ESXi hosts myself via other means so I’m leaving those permissions off.
You must make sure you assign this permission to the root vCenter level. As you can see I’ve created a service account in my Active Directory domain and assigned it to the custom role, then applied at the vCenter root.
Here you can see my credentials have been entered and tested. Note that I have chosen to un-tick the option to configure my ESXi hosts automatically. If you’re wondering why I’ve disabled this option when it could handle configuration for me the answer is simple. As this is a home lab setup I don’t necessarily want things automatically being configured to log data out to this VM. I spin up ESXi hosts frequently to test things out and I don’t want their data being dragged into this. I’d much rather configure it myself when required. That being said, if you’re deploying to a production datacenter you may wish to enable this option (and configure the relevant permissions on the vCenter role) to make things easier.
OK so at this point everything we need is setup – there are certainly other options to play with but right now the system is ready. Let’s add some hosts manually as this is the way I am demonstrating for this home lab setup.
Configure ESXi Hosts
First off let’s configure an ESXi host via the GUI. Select a host, navigate to the ‘Configuration’ tab and then click on ‘Advanced Settings’.
Syslog data can be sent either via UDP, TCP or SSL. In my case I’m going to use SSL – if I’m sending all this logging data over the network I want it to be secure and not sent over the wire in plain text. The syntax is very simple, basically the protocol followed by the remote hostname and port number. For example –
The next step is to make sure the ESXi host firewall is configured to allow the traffic.
The properties window will open and you’ll need to scroll down to find the syslog entry.
You can also configure the firewall rule to only allow traffic from specific networks if desired – in this case I’m going to leave it on the default of any IP address.
Command Line Configuration
OK now that we’ve configured a host via the GUI let’s do it over an SSH connection.
First let’s use the esxcli command to define the remote syslog host.
[root@BSA-ESXI01:~] esxcli system syslog config set --loghost=ssl://bsa-vrli01.ad.bytesizedalex.com:1514
Now we need to reload the syslog service.
[root@BSA-ESXI01:~] esxcli system syslog reload
If we get the current configuration we can see the new remote host entry at the bottom.
[root@BSA-ESXI01:~] esxcli system syslog config get
Default Network Retry Timeout: 180
Dropped Log File Rotation Size: 100
Dropped Log File Rotations: 10
Enforce SSLCertificates: false
Local Log Output: /scratch/log
Local Log Output Is Configured: false
Local Log Output Is Persistent: true
Local Logging Default Rotation Size: 1024
Local Logging Default Rotations: 8
Log To Unique Subdirectory: false
Message Queue Drop Mark: 90
Remote Host: ssl://bsa-vrli01.ad.bytesizedalex.com:1514
Now time to configure the firewall rule.
[root@BSA-ESXI01:~] esxcli network firewall ruleset set --ruleset-id=syslog --enabled=true
[root@BSA-ESXI01:~] esxcli network firewall refresh
Finally we can use the netcat command built-in to ESXi to test the remote host connection. In this case I’m using the appliance IP address and port.
[root@BSA-ESXI01:~] nc -z 172.16.1.37 514
Connection to 172.16.1.37 514 port [tcp/shell] succeeded!
If we check the hosts section we should see our added devices.
Now we can enjoy the in depth metrics and analysis of our logging data.
I know this is a pretty long post (as is usually my way) but hopefully it’s been informative. As ever any questions/suggestions please drop them in the comments section below.