VMware vROps - Custom Dashboard Design Part 1

I think that Custom Dashboards are one of the most powerful features in vRealize Operations Manager (vROps). They enable you to commission and build compelling frames of view, almost like paintings, into your virtual or cloud environment. These dynamic paintings can tell the user a story and empower them to make educated decisions about many facets of a Software Defined Data Center.

So, you want to start building your own Custom Dashboard in vROps but don't know where to start. Do custom dashboards seem a little overwhelming and a bit like black magic? Well, maybe a little, but with understanding of some basic concepts, I will try to demystify this artform and make you a dashboard ninja in no time. To get you started, we will begin with a very simple but somewhat powerful dashboard, build on it to add more functionality, and finally move on to some more complex use cases. Hopefully by the end of this series, you will be able to dream up your own dashboards and impress everyone with your creativity.

As I mentioned, we will be easing into dashboard design with the most plain vanilla of all custom dashboard designs. I like to call it the "Do it Yourself" (DIY) dashboard, meaning that instead of pre-defining a lot of options, we will let the user do most of the exploring. Keeping it simple will also allow us to cover all of the custom dashboard elements in depth to arm you with the knowledge you need to tackle more complex designs.

Dashboard Overview: Let's start with the dashboard overview before we dig into the configuration. In the top left corner is an Object List widget (1) configured to only show us Virtual Machines (VMs). The object list allows us to focus on and select an object, such as a VM, and drives the interaction with other widgets. On the right, next to the object list, is a Metric Chart widget (2). The purpose of the metric chart is to visualize our metrics as time series data in charts. Sort of similar to a stock ticker chart. Back to the left, below the object list, is a Metric Picker widget (3) which allows us to select specific metrics and add them to the metric chart on the right.

Dashboard Layout:

Now that you know the lay of the land, let's build this thing in vROps. After you login to the Product UI and have an appropriate Content Creator role assigned to your account, you basically have two options to create a new dashboard.

Content:

One way to create a new custom dashboard is to click the Content button on the Quick Links menu or in the Navigation Pane.







This should bring you to the Dashboards under Content area by default. Here just click the Create Dashboard button (green plus) to launch the New Dashboard wizard.

Create Dashboard Option 1:

The second way to create a new custom dashboard is to just use the Actions menu on the Home screen and select Create Dashboard, which will launch the New Dashboard wizard. This is also my preferred way to create and edit dashboards, as it allows you to preview your work as soon as you hit the save button and enables you to easily make other dashboard modifications as needed.

Create Dashboard Option 2:

Regardless of which way you launch the New Dashboard wizard, the first thing you have to do is enter a dashboard Name in the Dashboard Configuration section of the wizard. I like to keep mine short and to the point, as this ends up being the dashboard tab name. The longer you make it, the more space it will take up.

Dashboard Name:

Click on the Widget List to move on to the next wizard section and start selecting the widgets you need. For our example dashboard, we need Object List, Metric Picker, and Metric Chart. Just drag and drop them onto the dashboard space on the right. Don't worry about the layout here, we'll fix it later to fit your screen resolution. You will never be able to get it exactly right in the dashboard wizard, so don't bother. Once the dashboard has been created, you will have much more layout flexibility and can adjust the widgets any way you want them.

Add Object List widget

Add Metric Picker widget

Add Metric Chart widget

Now that we have our basic layout, let's look at the configuration of the interactions. Interactions are the glue that bind the widgets and enable you to pass parameters between them. For example: selecting a VM object in the object list passes that parameter name (VM name in our case) to both the Metric Picker and Metric Chart widgets. This in turn enables those two widgets to display appropriate information about the selected object. If you think about it, this is very similar to passing a parameter to a module when programming. Click on the Widget Interaction section of the wizard to get started.

Widget Interactions:

In our example, we need to set the Object list to feed data into the Metric Picker and Metric Chart. Also, Metric Chart has a second input, set this one to Metric Picker. This will allow you to add additional metrics to the chart later. Remember that you always go from left to right. Take data from the source, object list in our case, and feed it into the destination widgets as the green arrows indicate. Make sure you understand this concept, as it is the foundation on which powerful dashboards hinge.

Widget Interactions Settings:

Apply Interactions:

Click the Apply Interactions button! If you skip this step the Save button does not do it.



















We will cover Dashboard interactions in another post, so just click the Save button to create your first custom dashboard.

Save Dashboard:

With your basic dashboard build completed, let's rearrange the widgets the way we need them to appear in order to be most useful. You can move the individual widgets to almost any location on the dashboard. All you have to do is point your mouse at the Title Bar and once you get the cross-hair pointer...

Widget Layout 1:

...drag the widget into the desired position and let go of it. You will see a dotted outline where the widget will land.

Widget Layout 2:

You can also change the column width and widget height by positioning your mouse cursor between the columns or over the bottom of the widget frame. These operations will change the pointer shape to indicate your have control over the column width or widget height.

Widget Sizing:

Click and drag between the columns to adjust the column width.

Column Width:

Click and drag the widget bottom border to adjust the height. When adjusting widget height, the best practice is to make everything fit on a single screen without scrolling, if possible.

Widget Height:

When laying out the widgets, some aesthetic principles come to mind: keep things clean and simple, make sure things flow naturally and that navigation is intuitive.

With widget interactions and layout out of the way, let's look at the configuration of each widget. Click the Edit Widget button on the Widget Title Bar.

Edit Widget:

This will open the Edit Object List window that allows for adjustment of various widget settings. All widgets have this option with various settings depending on visualization provided by the widget.

In the Edit Object List window, I typically change the widget Title to something more descriptive. This provides a hint to the user on how to interact with the dashboard.

Widget Title:

The Refresh Content and Refresh Interval settings are pretty self-explanatory. Turn Refresh Content On and keep it at 300 seconds,as default vROps collection interval is 5 minutes.

Widget Refresh:

Mode controls two things. When set to Self, it allow the widget to act as a Self Provider. Meaning the widget is its own source of data, such as the list of VMs, Hosts or Clusters, to display. Second, if the Mode is set to Children or Parent, it allows the widget to take input from another widget and leverage the hierarchical relationship among objects to display related objects. For example, let's say you have two object lists and the first one serves as the input for the second one leveraging interactions. The first object list has a list of vSphere Hosts (parent containers), and the second one has the list of VMs (children) running on a selected host. For this to work, you would set the input object list to Self, and the receiving one to Children. We will go more in-depth on this topic in future posts, but for now this is enough to keep you going.

Widget Mode:

The next thing we need to configure for our object list is to define the scope. Using the "Select which tags to filter" option, we select the Virtual Machines (###) under Object Types node; this will narrow down the list of visible objects within the widget to VMs only. This is important, as it allows us to focus on a subset of the environment and get very specific as to what objects we want to interact with. The filter allows you to select different object types, allowing for display of other objects, such as hosts or clusters. We will be leveraging this, relationships and interactions extensively in more complex future dashboards. Multiple selections are also supported. If you selected the wrong node, you can click it again to deselect it, or clear all selections by clicking the Deselect All button. Lastly, the list of Object Types will get longer as you extend the database schema by installing additional management packs or defining tags. 

Widget Filter:

Optionally if you don't want to see VMs with question marks for metris, you can hide the powered off VMs. This can be simply done by selecting the PoweredOn option in the tag filter.

Powered On Filter:

With the scope filter in place, let's move on to making our object list display some more relevant and useful information in the columns. While in Edit Object List window, click on Additional Columns sidebar, hidden on the right side, to expand it.

Additional Columns:

Once the Additional Columns sidebar is exposed, it will reveal a Pick Metrics button. Click it.

Pick Metrics:

We have some work to do on the Pick Metrics with Object Type window. First, we need to set the Object Type we want to get metrics for. In our case, since we limited the widget scope to Virtual Machines only, it makes sense to select Virtual Machines under Object Type section. There are two methods for finding the object type you want. First, you can pick the adapter type, then scroll and browse the list, but the more management packs you have, the more cumbersome that becomes. Therefore, I prefer to use the search and just type in the "virtual m" phrase (1). This will return two object types that match this criteria. Click on Virtual Machine (2). Selecting the Virtual Machine will load metrics applicable to the VM object in the Metric Picker. Here again I would recommend searching (3) to find the metrics you care about. Once you find the metric you want, double click it (4) to add to your column list below. Repeat this step until you have all the metrics you need, keeping in mind that only 10 additional columns are allowed. Once you have all the metrics, it's time to change the column labels to something shorter than the metric names, which tend to be very long. Double click in the space below Box Label (5) for each metric and type in the label you want to see in the column heading. I typically abbreviate the metric name, for example for CPU Contention % I would type something like "CPUcont%". After you provide each label name, click the Update button (6) and repeat the process for the remaining metrics. When you're finished, click the OK button (7).

Column Metric Picker:

When back at the Edit Object List window, review your column list and click the Save button to save your new columns.

Save Widget Settings:

With new columns, it's time to remove the less useful default ones. Hover your mouse over one of the column headings and click on the down arrow/triangle icon that appears (1). Hover the mouse over Columns until a list of columns is revealed (2). Deselect all the checkboxes (3 and 4) except Name. Click somewhere away from the list.

Column Cleanup:

With all those columns out of the way, you should have a nice looking object list. Time to make it even better by adjusting the column widths to get as many of them visible without scrolling left and right.

Column Resize:

You should now have a beautiful looking Object List with useful matrix of VM data to sort and explore. Sorting by any column allows you to quickly stop VMs with potential bottlenecks, like high CPU Contention %, etc.

Column Sorting:

Moving on, there is not much to configure in the Metric Picker widget other than Title and refresh rate, which is mostly not necessary for this example dashboard. Make the title something useful for the end user like "Select VM Metric Picker" and Save.

Edit Metric Picker:

The last thing to configure in order to make this custom dashboard whole is the Metric Chart. Click the Edit Widget button, then the Metric Chart to go into edit mode.

Edit Metric Chart:

As always, make the widget Title (1) something useful to the end user like "Selected VM Metric Chart". Set the Refresh Content to On and leave Refresh Interval (2) at 300 seconds or 5 minutes. Step 3 and 4 are optional. Click Save (5) to apply the settings. I don't mean to glance over steps 3 and 4, but this post is getting long enough so I'll have to revisit in a future post about creating XML Interaction Files. For now, you have two options. First, selecting nothing will display a random set of metrics in the metric chart when you pick a VM in the object list. Two, selecting "sampleScoreboard.xml" as your list is probably much shorter than what you see in my screenshot, since we have not covered creating any XML Interaction Files yet. Trying sampleScoreboard.xml will give you a sneak peek of what's possible.

Metric Chart Configuration:

With all widgets configured, let's take a look at the final product and talk about using it. As I mentioned at the beginning of this monster post in the Dashboard Overview section, you should be able to select a VM in the object list. This action should load some metrics into the Metric Chart based on the specific Metric Configuration XML Interaction File. You should also be able to select additional metrics from the Metric Picker and add them to the Metric Chart at will.

Using Dashboard:

Let's quickly cover some useful Metric Chart options. The Metric Chart widget can be a very powerful time-series data visualization tool so make sure you get intimately familiar with the full range of settings it provides. See the crazy busy screenshot below for most important options on the metric chart toolbar and make sure you can use and read all of the visual aids provided.

Metric Chart Options:

One of the most important features of the Metric Chart widget is the ability to specify the time range for the metric charts displayed. The Date Controls button allow you to specify the timeframe for the chart. You can start with last hour and go back as far as your data retention policy allows. To set the chart timeframe click on the Date Controls button (1) to reveal the date/time options. Click the Range dropdown (2) and pick (3) a predefined time period or specify a custom range. Click the Go button to apply the setting to the charts. The charts will be rerendered with new time range. Be sure to get comfortable using various zoom options, as the longer the timeframe, the harder it is to see and read individual metric values.

Set Metric Chart Date/Time Range:

Deselecting the Split Charts button allows you to overly all the individual metric charts on a single chart to enable easy comparison. 

Split Charts:

Clicking the Stacked Charts button allows you to stack chart on top of one another. 

Stacked Charts:

In the next post, we will add further complications to our dashboard in order to get more functionality.

In future posts, I will cover how to get useful KPIs for your dashboards, create super metrics when needed, and configure XML interaction files, so stay tuned.

Four, probably most frequently used, Object Types in vROps are listed below. These names are slightly different than typical vSphere/vCenter nomenclature, so get familiar with those terms.

1. Cluster Compute Resource = vSphere Cluster
2. Host System = vSphere ESXi Host
3. Virtual Machine = VM
4. Datastore = vSphere Datastore

Some useful vROps metrics for Virtual Machines used in this example dashboard are listed below, with explanations. Get familiar with these, as some are slightly different than what you see in vSphere and vCenter. Metric Definitions:

1. CPU Contention %: Physical CPU demand that was denied, 5min average per-vCPU. Same as CPU Latency % in vCenter and %LAT_C in esxtop. Includes Co-stop, Ready, Hyper Threading sharing, and CPU power-saving Dynamic Frequency Scaling Voltage effects. Higher values are worse, ex: > 5-10%. Overly high values may be due to CPU power-saving settings in BIOS. If still high and BIOS Settings are OK, then look at Co-Stop and Ready.
2. CPU Demand %: Percent of the CPU Capacity which was demanded by the VM (not necessarily Delivered.) Tells whether the VM is highly utilized or not. A high value means the VM may need more compute resources. A low value with associated High VM and Host CPU Contention may indicated a host configuration problem. CPU Workload % metric is based on CPU Demand %.
3. CPU Usage %:  Percent of CPU Capacity that was actually delivered by the vSphere Host to the VM.
4. CPU Ready % per Core, this is a Super Metric: Percent of time a vCPU was denied a pCPU (excluding Co-Stop). Higher values indicate starvation, hurting VM performance. Values < 5% per vCPU are usually not a problem (and Ready(%) is rarely zero). Most common causes: host is overloaded or slow storage is wasting host pCPUs in WAIT (for IO) state. 
5. CPU Co-Stop %: Time that a busy vCPU was denied a pCPU because the less-busy vCPUs needed to catch-up to it. Higher values may indicate unnecessary vCPUs are hurting this VM's performance. Note: Sum (not average) of co-stop per vCPU, so if high, look at per-vCPU stats.
6. CPU Swap Wait %: Percent of time VM is waiting for the VMkernel to swap VM memory in back from disk to physical RAM in the Host. Check if VM has compressed memory and/or is swapping, look at Memory Decompression Rate (KBps) and Memory Swap In Rate (KBps).
7. CPU IO Wait %: Percent of time VM is waiting for data to be read from physical disk (local or shared eg: SAN). Check if VM has high disk latency by looking at Disk Command Latency (ms).
8. Memory Contention %: Percent time spent waiting for data which was expected to be in Physical RAM already. Mostly Swap-in from VM's .vswp file and time to decompress data that the host compressed. Non-zero values indicate likely performance problem. Indicates host had too little available free Physical RAM.
9. Disk Command Latency (ms): The average amount of time in milliseconds taken for a command from the perspective of a Guest OS. This is the sum of Kernel Command Latency and Physical Device Command Latency.
10. Network Packets Dropped: Number of network packets dropped.

Congratulations, you just built your first custom dashboard! Doesn't it feel good?!?

Please checkout part 2 of this series.

For more information about vROps, see the following resources:

Books:

VMware vRealize Operations Managers Essentials by Matthew Steiner

Mastering vRealize Operations Manager by Scott Norris
VMware vRealize Operations Manager Capacity and Performance Management by Iwan 'e1' Rahabok

Official VMware:
VMware Professional Services
Official vROps Documentation
VMware Operations Management White Papers
Extensibility and Management Packs
vROps product page

Blogs:
vXpress by @Sunny_Dua 
virtual red dot by @e1_ang
Virtualise Me by @auScottNorris
Elastic Sky Labs by @JAGaudreau
i'm all vIRTUAL by @LiorKamrat