VMware vRealize Log Insight Deployment

If you have a VMware environment and you haven’t yet deployed the vRealize Log Insight appliance you really should. While the ‘Log Insight’ platform is a licensed product VMware do provide it with a number of free licenses included so depending on your environment size you may not even need to pay. In this post I’ll go through the steps involved deploying the appliance and configuring ESXi hosts and vCenter to communicate with it.

First off some useful links –

 

Licensing

I don’t want to pull all the licensing information into this post so go to the following link for the complete details –

VMware FAQ: Log Insight for vCenter Server (2144909)

Suffice to say you will require a valid vCenter license, a quick summary –

Q: What is the difference between vRealize Log Insight for vCenter Server and vRealize Log Insight?


A: Each vCenter Server customer is entitled to receive a 25-OSI pack of vRealize Log Insight 3.3 for vCenter Server at no charge. Log Insight 3.3 for vCenter Server supports a single Log Insight 3.3 for vCenter Server license and limits Log Insight to a single Log Insight node, a single vSphere integration, Content Packs from the in-product marketplace for only VMware products, and disables enterprise features including event forwarding, archiving, and SSL.


Virtual Appliance Deployment

Now it’s time to start deployment and configuration. I’m going to assume you have downloaded the appliance and selected the OVF for deployment in vCenter. In this case I’ll show the steps via the traditional C# client though of course the web client is going to be the only option moving forward (later this year the C# client will be discontinued).

It’s important to remember that this is a logging and analysis system so disk space requirements will be highly dependent on your setup – i.e the number of systems you want to monitor.

Deploy OVF Template

 

As with all appliances you will be asked to accept a license agreement.

Deploy OVF Template

 

Select an appropriate name for the virtual machine – you can either leave it at the default or change it to the hostname you will configure later. In my case I’m leaving it as the default which is fine for my home lab environment. You can also select an appropriate folder in your vCenter hierarchy.

Deploy OVF Template

 

Now for an important choice, the system requires a decision on how ‘big’ an appliance to configure. Obviously you need to consider the current size of your environment as well as future growth. Along with those variables you need to be mindful of how much compute and storage resource a larger configuration will require.

Deploy OVF Template

 

As this deployment is for my home lab I’m going with the ‘Extra Small’ deployment configuration. The system provides a warning that this config is intended for testing or POC usage – not production. Contrast the ‘Extra Small’ resource requirements to the next screenshot showing a ‘Large’ deployment.

Deploy OVF Template

Deploy OVF Template

 

As you can see  the ‘Large’ deployment has a fairly considerable resource requirement. It should also be noted that with a fully licensed version you can cluster these appliances for greater availability etc. so in a truly large environment you could invest quite a lot of resource in this sort of monitoring.

The next step is to select a vCenter Cluster to deploy the appliance to – in my lab I only currently have a single cluster with two hosts.

Deploy OVF Template

 

My current setup requires that I manually select an ESXi host to deploy to.

Deploy OVF Template

 

I’d love to run all my virtual machines from solid-state drives however I don’t have the finances for that and with this sort of appliance (lots of writes to the VM) deploying to an SSD would just create wear on the flash cells. For this reason I’m deploying to a standard magnetic spinning disk. As a point of comparison my deployment at work runs on a SAN attached datastore with much higher performance characteristics – required for the size of deployment in those datacenters.

Deploy OVF Template

 

The next configuration decision is whether to deploy the virtual disks as thin or thick provisioned. It is strongly recommended to deploy the disks as ‘Thick Provisioned Eager Zeroed’ (TPEZ) due to the large write workload this appliance will generate. As this is a deployment in a home lab I’m thin provisioning the disk however, if I again contrast with my work datacenter deployment – there the virtual machine disk is TPEZ.

Deploy OVF Template

 

Select the appropriate network setting for your environment – in my case I’ve selected a VLAN I use for infrastructure related systems. It includes my ESXi hosts and vCenter so it made sense for me to add this appliance to the same network.

Deploy OVF Template

 

Now for the final configuration settings. Again these should be configured according to your environment.

Deploy OVF Template

 

Deploy OVF Template

 

The wizard presents a final page confirming the choices made earlier – assuming all is as desired it’s time to hit the ‘Finish’ button and let the system deploy the appliance.

Deploy OVF Template

Deploy OVF Template

Deploy OVF Template

Deploy OVF Template

 

Now that the virtual machine is deployed we can start it up and move on to configuring our environment to use it.

First off I’d suggest checking you have a DNS record for the new virtual machine – how you do this depends on your setup. In my case I just added it to my Microsoft DNS server forward and reverse lookup zones. Once that’s done power up the virtual machine and once booted you should see something similar to the screenshot below on the VM console.

Virtual Machine Console

 

When connecting to the web console a certificate error will be presented as the appliance generates a self-signed certificate. Unfortunately you have to pay for the full license to be able to replace the SSL/TLS certificate – this is something I strongly disagree with. Security shouldn’t be behind a pay wall. That being said, let’s continue.

Webpage Certificate Alert

 

The web interface now requires us to walk through a wizard configuration process, let’s begin.

vRealize Log Insight Configuration Wizard

 

First off we are asked whether this is a new deployment or if we’ll be adding to an existing setup. Obviously in my situation this is a new deployment.

vRealize Log Insight Configuration Wizard

vRealize Log Insight Configuration Wizard

 

Time to setup some credentials.

vRealize Log Insight Configuration Wizard

vRealize Log Insight Configuration Wizard

 

Once you’ve added your vCenter key you should get a confirmation that it’s active.

vRealize Log Insight Configuration Wizard

 

Next we need to decide whether to send VMware data for their CEIP along with defining the e-mail notification address and any HTTP notification URLs.

vRealize Log Insight Configuration Wizard

 

Accurate time keeping is vital in any digital infrastructure and especially when one is dealing with logging. If devices/systems do not share a common time source it can make incident correlation much harder. I’m presenting a few screenshots for this – the first shows the default NTP servers as provided by VMware. Next I’ll show you my configuration which uses the same NTP time source as my ESXi hosts (and other systems) use.

vRealize Log Insight Configuration Wizard

vRealize Log Insight Configuration Wizard

 

Once we’ve configured time settings it’s time to setup SMTP e-mail notification.

vRealize Log Insight Configuration Wizard

 

Huzzah the wizard is complete!

vRealize Log Insight Configuration Wizard

 

Now for the final step…

 

 

Configure Devices For Log Insight Communication

Logon to the web console.

vRealize Log Insight Device Configuration

 

The splash screen presents us with a few options – depending on your environment you may wish to configure some or all of these options. In my case I’m just going to use the ‘vSphere Integration’ as an example.

vRealize Log Insight Device Configuration

 

As part of the vCenter integration it’s necessary to provide a set of credentials. Depending on the level of integration you’re after the requirements differ, this link should provide all the info you need – Connect  vRealize Log Insight to a vSphere Environment.

In my case I simply created a new vCenter permissions role called ‘vRealize Log Insight’ with no assigned permissions. When you add a custom role and do not assign any privileges to it, the role is created as a Read Only role with three system-defined privileges: System.Anonymous, System.View, and System.Read. If you want vRealize to be able to configure ESXi hosts automatically to talk to it then you need to add ‘Host.Configuration.Change settings’ and ‘Host.Configuration.Network configuration’. In my case I’m happy to configure the ESXi hosts myself via other means so I’m leaving those permissions off.

vCenter Permissions

 

You must make sure you assign this permission to the root vCenter level. As you can see I’ve created a service account in my Active Directory domain and assigned it to the custom role, then applied at the vCenter root.

vCenter Permissions

 

Here you can see my credentials have been entered and tested. Note that I have chosen to un-tick the option to configure my ESXi hosts automatically. If you’re wondering why I’ve disabled this option when it could handle configuration for me the answer is simple. As this is a home lab setup I don’t necessarily want things automatically being configured to log data out to this VM. I spin up ESXi hosts frequently to test things out and I don’t want their data being dragged into this. I’d much rather configure it myself when required. That being said, if you’re deploying to a production datacenter you may wish to enable this option (and configure the relevant permissions on the vCenter role) to make things easier.

vRealize Log Insight Device Configuration

 

OK so at this point everything we need is setup – there are certainly other options to play with but right now the system is ready. Let’s add some hosts manually as this is the way I am demonstrating for this home lab setup.

 

Configure ESXi Hosts

 

GUI Configuration

First off let’s configure an ESXi host via the GUI. Select a host, navigate to the ‘Configuration’ tab and then click on ‘Advanced Settings’.

ESXi Syslog Configuration

 

Syslog data can be sent either via UDP, TCP or SSL. In my case I’m going to use SSL – if I’m sending all this logging data over the network I want it to be secure and not sent over the wire in plain text. The syntax is very simple, basically the protocol followed by the remote hostname and port number. For example –

  • udp://bsa-vrli01.ad.bytesizedalex.com:514
  • tcp://bsa-vrli01.ad.bytesizedalex.com:514
  • ssl://bsa-vrli01.ad.bytesizedalex.com:1514

ESXi Syslog Configuration

 

The next step is to make sure the ESXi host firewall is configured to allow the traffic.

ESXi Syslog Configuration

 

The properties window will open and you’ll need to scroll down to find the syslog entry.

ESXi Syslog Configuration

 

You can also configure the firewall rule to only allow traffic from specific networks if desired – in this case I’m going to leave it on the default of any IP address.

ESXi Syslog Configuration

 

 

Command Line Configuration

OK now that we’ve configured a host via the GUI let’s do it over an SSH connection.

First let’s use the esxcli command to define the remote syslog host.

[root@BSA-ESXI01:~] esxcli system syslog config set --loghost=ssl://bsa-vrli01.ad.bytesizedalex.com:1514

Now we need to reload the syslog service.

[root@BSA-ESXI01:~] esxcli system syslog reload

If we get the current configuration we can see the new remote host entry at the bottom.

[root@BSA-ESXI01:~] esxcli system syslog config get
    Default Network Retry Timeout: 180
    Dropped Log File Rotation Size: 100
    Dropped Log File Rotations: 10
    Enforce SSLCertificates: false
    Local Log Output: /scratch/log
    Local Log Output Is Configured: false
    Local Log Output Is Persistent: true
    Local Logging Default Rotation Size: 1024
    Local Logging Default Rotations: 8
    Log To Unique Subdirectory: false
    Message Queue Drop Mark: 90
    Remote Host: ssl://bsa-vrli01.ad.bytesizedalex.com:1514

Now time to configure the firewall rule.

[root@BSA-ESXI01:~] esxcli network firewall ruleset set --ruleset-id=syslog --enabled=true 
[root@BSA-ESXI01:~] esxcli network firewall refresh

Finally we can use the netcat command built-in to ESXi to test the remote host connection. In this case I’m using the appliance IP address and port.

[root@BSA-ESXI01:~] nc -z 172.16.1.37 514 
Connection to 172.16.1.37 514 port [tcp/shell] succeeded!
[root@BSA-ESXI01:~]

 

If we check the hosts section we should see our added devices.

Hosts

 

 

Now we can enjoy the in depth metrics and analysis of our logging data.

Dashboard

 


 

I know this is a pretty long post (as is usually my way) but hopefully it’s been informative. As ever any questions/suggestions please drop them in the comments section below.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.