ViPR and ViPR SRM Resources and Getting Started

…Updated 9/30/2014 with new links…

The below list will continue to grow as I find specific resources that are valuable to those looking to learn about, deploy, and leverage EMC ViPR and ViPR SRM.

An overall excellent public community devoted to ViPR Software-Defined-Storage, ViPR SRM, and ECS:
https://community.emc.com/community/products/vipr

ViPR Product Documentation Index. This contains concepts, install and configure guides, download links, and important information like the support matrix.

ViPR quick install: I created this youtube video to show installing and configuring ViPR to running self service provisioning in under 15 minutes. It is still rough with no voice over but a great way to see the basic steps and speed of deployment.

ViPR SRM Product Documentation Index.  Well organized index of concepts, install and configure guides, and a listing of all SolutionPacks that are available and their install and their usage.  SolutionPacks plug-in to ViPR SRM to provide reporting for storage (EMC and 3rd party), SAN fabrics, virtualization, SQL, Oracle, and more.

ViPR SRM Quick Start steps:

  1. Deploy OVF template.  A small eval environment can use the single VM OVF.
  2. Login to http://<IP/hostname>:58080/APG
  3. Go to Administration -> Centralized Management
  4. Click Ok to save the default server configuration
  5. Use the SolutionPack Center to configure reporting for each component (ie: VNX, VMAX, vCenter)
  6. Enjoy the visibility into your environment
  7. Check out the topology views by clicking explore in the left pane and selecting a host or vm.
  8. See the ViPR SRM Product Documentation Index for more details.

Download ViPR Free!

The links below require an EMC Support login:

ViPR support and Download full ViPR paid version (requires a license key)

Download ViPR SRM (includes a 30-day trial).  Follow the link and download “ViPR SRM 3.5SP1 vApp Deployment”  See above for quick start steps.

A New Year, a brand new SRM 3 at EMC

This post was featured on EMC Advanced Software Division’s rethinkstorage.com.  Head over there too for SDDC perspectives from our management team and product gurus.

If one of your New Year’s resolutions is to improve storage resource management and gain better visibility into your environment then you are in luck. EMC released Storage Resource Management Suite (SRM Suite) 3.0 on January 30, 2014. This is a major new product release with awesome enhancements and a complete consolidation of the software suite known affectionately as the SRM Suite.  Previous versions of SRM Suite consisted of three discrete products:

  • Watch4Net for single pane of glass monitoring and reporting to enterprise infrastructure
  • Storage Configuration Advisor for SAN and array compliance validation against EMC best practice and support matrices
  • ProSphere for end-to-end topology mapping with capacity trend analysis with chargeback

SRM Suite 3.0 now includes all of the above in a single interface, single agentless polling architecture, and a single back-end!  It even installs simply via a vApp OVF template.  Early access customers who previously used ECC now feel that they have a management solution to supersede ECC. SRM exceeds their needs by eliminating ongoing agent maintenance, providing end-to-end visibility with thorough alerting, and presenting a more intuitive interface.

With the shipment of SRM Suite 3.0, EMC has delivered on its promise to consolidate multiple products into one with the best features of each previous software component preserved in the new offering. Look for an installation and configuration post shortly but in the meantime, “These are a few of my favorite things…”

New Look and Feel
New dashboards guide you through typical operations, reporting, and troubleshooting workflows. You can now see combined information across different components in your infrastructure for complete end-to-end analysis because storage is not the only thing that matters in a private cloud.

simple dashboard workflow

Capacity Trend Analysis and Chargeback
You can now perform enterprise wide capacity trend analysis to determine storage growth rates and when new capacity will be needed.  Context sensitive tables allow you to drill-down into the details about LUN Consumers and Disk Contributors.  Now you can predict when virtual machines or physical hosts will run out of capacity plus you will avoid the frantic request to expand or create new datastores.

Gain rapid insight into VMs that are nearly reached their file system capacities

Gain rapid insight into VMs that are nearly reached their file system capacities

We all appreciate heat maps for quickly determining trouble areas.  This one shows what VM fileysystems are running out of space.

Chargeback maps resource utilization to VMs, custom departments, or projects and can even include costs for each class of storage service.  This example shows the out-of-the-box chargeback details for projects provisioned via the self-service catalog in ViPR.

How much is a department or project costing you per month, last week, over the last quarter? This is at your fingertips and works out of the box with ViPR!

How much is a department or project costing you per month, last week, over the last quarter? This is at your fingertips and works out of the box with ViPR!

 

End-to-end Topology
Viewers quickly understand the relationship of a VM through the entire stack – VM, vSphere Host, SAN, and storage.

End-to-end topology lets you see the relationships of the VM, vSphere host, SAN fabric, and storage.

End-to-end topology lets you see the relationships of the VM, vSphere host, SAN fabric, and storage.

Drill into each component to determine what ports are being used.  Also, selecting a VM, host, or storage changes the context to reveal performance or capacity data about that particular component.

expanded topology view

Drill down into the details to determine what array or host ports are connected to the fabric ports.

Want to know where a particular virtual hard disk lives on your storage array?  Say goodbye to the time consuming processes of cross-referencing spreadsheets or viewing multiple management interfaces to map a VM back to the storage array.

Get a detailed mapping of VMDK to Datastore to Array LUN

Get a detailed mapping of VMDK to Datastore to Array LUN

Performance Trend Analysis
Oh the dreaded performance troubleshooting.  The DBA walks up to you and tells you his database is performing poorly.  Do you A) Tell him to stop complaining, B) Open all of the various points of management for your infrastructure, C) Summon SRM, or D) Blame the network guys?

Our suggestion is that you invoke SRM Suite 3.0 since it grants you the ability to execute rapid root cause analysis of performance issues at all levels.  Take a look at the database VM to see if disk IO is experiencing some kind of bottleneck.  Looks good?  Export any report to a PDF, image, or email it directly to the DBA, manager, and other curious people.  You can even create a custom view that contains tables, charts, and graphs from other reports by simply pinning them to build a dashboard.

Overview of VM performance and capacity

Overview of VM performance and capacity

Performance Concern vm expanded

This VM appears to have disk performance problem. Quick and easy root cause analysis for the entire stack.

Drill into the details of the virtual disk performance to get a better picture of what is happening. That’s odd, the latency is high but IOPS are low, then suddenly latency drops off and IOPS spike to over 30k.  It looks like VNX FAST Cache kicked in and began servicing the workload from flash in real-time.
VNX FAST Cache is a latency killer and IOPS master

VNX FAST Cache is a latency killer and IOPS master

There are a ton of canned reports that customers over the last few weeks have loved.  Here are a few others:

  • VM over/under sized – to help you understand if the CPU, Memory, or Disk are over/under sized
  • LUN not masked/mapped – find those pesky LUNs that are not being used and wasting space!
  • VMs running with snapshots
  • Datastore trend analysis with precise time that a Datastore will reach 90% full based on the growth rate – a life saver!
  • Pin view – grab any number of reports into one single view for a dashboard or to share via PDF with a co-worker.

Finally, here is a summary of other notable new features:

  • Quick installation of the product into Windows, Linux, and VMware environments with a vApp for vSphere
  • Windows host agent that collects data without requiring any credentials on the host or a completely agentless option for easy deployment and maintenance-free operations
  • Host capacity utilization reporting that allows the storage admin to see how existing capacity is being utilized at the host level
  • Management of SLAs and Chargeback reports in terms of FAST Policy or disk/array characteristics
  • A new use-case driven user interface
  • Performance monitoring and troubleshooting
  • Topology maps, end-to-end tabular views of the data
  • Consolidated monitoring of health, configuration, compliance breaches, and threshold-based alerts
  • Tabular summaries of information about zonesets, zones, and zone members
  • Global search across all discovered configuration items
  • A consolidated Alert console that includes:
    • Alerts for breaches of EMC’s eLab Support Matrix and configuration best practices
    • EMC PowerPath alerts
    • Threshold alerts based on performance data
    • Health alerts collected from Brocade switches, Cisco switches, VMware, and VNX and VMAX arrays

With the shipment of SRM Suite 3.0, I hope you fulfill at least one of your New Year’s resolutions!

ViPR: The Harmony Remote for Your Data Center

I find myself speaking more and more about ViPR with customers and partners.  The common thing amongst everyone is a general misunderstanding of what ViPR does for them.  In this post I will walk through my talk track which uses a simple analogy that I think we can all at some level relate back to our lives.

Data growth is explosive with the budgets for maintaining that growth staying the same or shrinking. Not only is data growing but we are seeing all sorts of new data types like Hadoop and Object along side of traditional File and Block.  IDC labels the next generation of applications as Platform 3.  This next platform demands infrastructure agility that most enterprise environments cannot provide.  At a storage only perspective you start to see the picture below with many storage platforms, vendors, and each with its own element managers.  This complexity drives the cost up nearly as fast as the data growth itself.

complexity

 

This leads to an example you can hopefully relate to.  I have a home theater that consists of many components – a TV, VCR (kidding!), Surround Receiver, DVD, Roku, Home Theater PC, and a cable box.  home theaterWhat happens when I want to watch cable TV or a DVD or even better when my wife wants to watch any of the above?  There are a series of steps that must be performed in a certain order otherwise I might have audio but missing the video or vice versa.  There are many remotes that we juggle to control each device and you must jump between each one to hit the buttons in the proper order.

remotes

My wife has never enjoyed this feature of our TV room.  Believe it or not, she does not like tinkering with the electronics like I do.  She just wants to watch House Hunters and does not care that the audio comes out in hi-def surround sound with 1080p quality through HDMI source 2.  A common scene at our house after wrestling with remote controls is below… This may be a slight exaggeration as she is pretty accommodating of my technology snafus.  For the purpose of this post, you may imagine IT is the guy and the biz is the wife.  🙂

D@mN home theater There is a solution.  First, you can get a universal remote to consolidate all remotes down to one.

universal remote

This is great for giving one point of control but you still need to understand which buttons to press in which order to get everything to work properly.  If you go away for a week you will likely forget the order and maybe make a tragic mistake that gets in between you and Breaking Bad.  This error at home is easy to overlook, but what about in IT?

You were only thinking Harmony Remote because it is in the title.  harmonyYes, the answer is a Logitech Harmony Remote which not only consolidates all remotes down to one by abstracting the control away from the individual components, it also has powerful built-in macros that trigger events to happen automatically in the same specific order every single time.

Happy wife, happy life.  Need I say more?

happy wife, happy life

 

ViPR Controller software (get it for FREE, no hardware required) is a universal remote control that abstracts the control plane of the underlying heterogenous storage arrays providing a single ridiculously simple management interface.  ViPR is also an automation engine that handles all tasks automagically for storage management – provisioning, replication, expansion, zoning, mounting a volume, snaps/clones, creating distributed VPLEX volumes across metro distances, RecoverPoint journals/source/target, and more.  All of this out of the box without any need to write custom scripts.  All accessible through a self-service web interface or REST APIs.  Powerful.  Did I mention we just launched it as FREE for non-production?

ViPR slide

 

Consequently, IT transforms from labor intensive and slow processes to immediate access to low cost, elastic compute, storage & network infrastructure.  This is just scratching the surface of ViPR’s capabilities.  This is really exciting stuff and further makes my decision to become part of the EMC Advanced Software Division a clear win.  Those of you that are along for this ride and leveraging these automation and orchestration tools are building a killer resume.  Enjoy!  Come back for more soon on these other transformational functionalities:

  • Chargeback/Showback
  • Self-Service
  • Data Services – Object and Hadoop on your existing storage infrastructure.
  • Commodity storage
  • Project Nile
  • A fast installation guide for ViPR

Update 1/28/14: I just learned that @keithnorbie presented a similar concept at VMworld 2013 titled “How SDDC is like a harmony remote” for a vBrownbag session.  I guess it is true, great minds think alike. Check it out.

New title and role and more blogging

Many are aware at this point that I transitioned to a new role at EMC in the last few months. This month I round out the end of my third year at EMC. It has been a blast as I have had the awesome opportunity to work beside the best sales and engineering force on the planet. Everyone is always willing to lend a hand, share their experiences, and teach new things not to mention the education that EMC provides is way beyond what I had at any other job.

So, the new gig. It was time for a change. I made the decision to explore options within EMC to grow my career, peer network, and expertise. Among the options was the chance to start a new role as an Advanced Software Division Specialist for our MidMarket Division. The new buzzword these days is Software-Defined ‘Everything’ (data center, storage, networking). The Advanced Software Division is defining Software Defined Storage at EMC. My thinking was, hey, the industry is changing rapidly and moving to this whole Software Defined strategy so what better place for me to be than right at the front of it helping to pave the way at EMC…?? Bam, done.

Software consulting is not new to me but I have spent the last few years advising customers on primarily hardware storage platforms.  But wait, nowadays storage is just a bunch of software intellectual property executing on commodity x86 hardware platforms, right?  “What is ASD at EMC?” everyone asks.  The Advanced Software Division is comprised of everything SDDC at EMC including the new and exciting ViPR Software Defined Storage and the Storage Resource Management Suite for cloud monitoring and reporting, plus UIM, SAS, and more.  Uh, you didn’t think I was going to miss this opportunity to shamelessly plug my new tech, did you?

Finally, as a technical expert in this area, you should expect to see way more blogging and sharing of these technologies and related SDDC tech like Puppet Labs (go Portland!), OpenStack, Amazon AWS, and maybe a post or two about my somewhat-new 11 month old son or geek personal tech!  Looking forward to hearing from you for things you want to read or learn about now that I am “Keen on SDDC.”  First question – corny title?  Now go follow me on twitter dangit!  The first three new followers who mention this post get a starbucks gift card and you better be a human and not some annoying spammer!

Happy Holidays!  See you next year.

@jeremykeen

 

Chrome browser search coolness

I was just browsing around with the chrome browser and discovered a cool little gem. BTW, I stopped using Firefox about 5 months ago and have no plans of going back. This post is not about that, though I could write a bunch about why (faster browsing, sleek interface, nice crash handling).

Back to the discovery… I wanted to search for “own it” on Amazon’s site to rate a book and when I pulled up the search box and entered “own” in the bar, I was surprised to see some lines in the scroll bar. What’s this, a bug? No, it’s showing where the search term exists in the document relative to the scroll bar. Cool.

Nice search feature

Transparent Page Sharing and Address Space Layout Randomization

I recently had a customer ask, “How is VMware’s Transparent Page Sharing impacted by Windows’ “new” Address Space Layout Randomization?”  Here is my response and some other interesting tidbits about what the other hypervisors are doing (or not doing) to increase consolidation ratios.

Microsoft’s Address Space Layout Randomization (ASLR) has little to no impact on VMware’s Transparent Page Sharing (TPS).  The intention of ASLR is to protect physical memory from things such as viruses by placing dll’s and executables in a virtual address space.  This virtual address space is then translated into the physical address by the operating system kernel.  Therefore, only the kernel has knowledge of the location of the physical mappings.

When a hypervisor like ESX/ESXi sits between the physical memory and the operating system, it takes the “physical mappings” from the OS and places then in true physical RAM.  Because the hypervisor controls the RAM, it can still perform TPS operations on this memory by identifying redundant memory pages and sharing them rather than duplicating them in memory.

Note that to-date, Hyper-V and XenServer do not offer any form of transparent page sharing.  Both XenServer and Hyper-V have implemented a memory ballooning technology (Dynamic Memory Control  and Dynamic Memory respectively) which VMware pioneered many years ago (~10 years).  These technologies utilize a driver in the guest operating system that intelligently “borrows” memory from the guest by essentially reducing the memory available to applications and the operating system without actually changing what is visibly shown in Windows.  Microsoft’s Dynamic Memory is only available with Hyper-V Server 2008 R2 Service Pack 1 which is currently only a release candidate.  It is supported in most server guest OS’s from 2003 through 2008 with particular Service Pack levels but it is only supported on client OS’s from Vista through Windows 7.  This means, VDI implementations that require XP will not benefit from the ballooning.

To summarize, no, TPS is not negatively impacted by ASLR.  Hyper-V and XenServer do not do any level of page sharing.  Actually, a year ago, Citrix and Microsoft strongly believed customers did not have a need to overcommit memory but it seems they are changing their stance on this because customers achieve far greater consolidation ratios on VMware vSphere.

My View – Design documents, photos, image editing

Today was a documentation day. Yippee! Oh Visio, how you manage to manipulate the simplest lines into complex, tangled webs, one will never know.

What I do know is that PCoIP provided an excellent user experience when working with graphics in Visio. One of the great features of PCoIP is that it builds the image to full quality over time unlike some other protocols that will generate a fuzzy image when bandwidth is unavailable. Indeed, a nice feature when working with graphics and attempting to read the small print on drawings. Dragging objects, working with text, connecting devices, and editing images were all accomplished from within the virtual desktop with no frustrating waiting while the image is redrawn as in RDP. Keep in mind I am accessing a virtual desktop in Steelhead Data’s cloud in Sacramento, CA from my home office in Portland, OR.

Scott Davis, VMware View CTO, does a great job of describing the PCoIP technology in his blog

More importantly, by transmitting compressed bitmaps or frames, we can adjust the protocol in real time to account for the available bandwidth and latency of the communications channel. On a WAN connection with typically less bandwidth and higher latency, a less crisp image is produced quickly, typically with 0.2-0.5 bits/pixel producing a grainy, but still recognizable image. Kind of like an analog TV… This rapidly sharpens with increasing clarity and detail visibility with each succeeding frame until the image is perceptually lossless. This is a high quality image at a total of approximately 1-3 bits/pixel. Think of it as now Digital HD to stick with our TV analogy. On a higher performance LAN, the images become sharp instantly and will build to complete lossless at 5-15 bits per pixel. Think of it as Blu-Ray!

After completing some documentation, I shifted over to responding to customer requests in our managed services platform where we host VMware View desktops for businesses. Pros to using a View desktop: My desktop is located in the datacenter with the managed environment so connectivity to the infrastructure is fast and easy. Running tools such as the vSphere client to access the console of VMs from within the View desktop provides a better experience than if I run the same tool over a VPN connection on my local client. In the past I would have made a similar connection into a terminal server or Citrix environment where I could then access these tools. The difference here is that I have my own dedicated desktop where I get to install the tools that are useful to me like the great automation tool from thevesi.org for performing tasks in a vSphere environment. Or maybe I want to use the Webex one-click application. This is not something that I would want to install on a shared terminal server but it’s my desktop and I’ll do as I please! If an application decides to misbehave, I have the option of rolling back to a snapshot or refreshing my desktop to a point where it is running like a well oiled machine. Try doing that on a terminal server or traditional desktop.

This post is part of a project I am undertaking where I will be using a VMware View desktop for - hopefully - all of my work computing.  See more by clicking the "myview" tag.

My View- Day 2

I was only at the keys for part of the day today so there wasn’t too much interaction with my VDI, View, virtual desktop, whatever you want to call it. It’s tempting to call it VD for virtual desktop but that just seems wrong. One “fun” thing happened while connected from a café over 4G. I was working on a document and writing some emails when suddenly I got a message that my laptop battery was nearly dead and I should connect to a power supply. Oooops. No power outlet in sight. I decided to let it run out because there was no risk of losing the data I was working on. Add that to the list of things I did not consider as a benefit of desktop virtualization.

This blog is about the good and the bad so let me tell you about something that is really frustrating. In order to keep my work and personal computing separated, I have a personal laptop and thin client connected to a KVM (keyboard, video, mouse) switch to allow for sharing a single monitor, keyboard, and mouse. For some dang reason, the thin client recognizes the keyboard and mouse properly but often times – not always – when I connect to my desktop in the cloud, the keyboard and mouse aren’t recognized or I receive a prompt saying that they couldn’t be installed. GRRRRRRRrrrrrrrrr. This typically is resolved by connecting to the desktop using the View client on my laptop and rebooting it but sometimes just requires switching through the KVM connections or unplugging and reconnecting the USB plug. Any thoughts? Let me make it clear that I am running Windows 7 as my virtual desktop which is “experimental” according to VMware.

My View – Using VMware View as my primary desktop

This post is coming to you from my VMware View virtual desktop “in the cloud.”

Let me start by explaining that I’ve promised some coworkers and colleagues that I will be attempting to use a VMware View desktop as my primary work computing device. I have a Wyse P20 in my home office and a laptop that I will use to connect to View. I live in Portland, OR, use Comcast as my ISP, have a 4G Clear wireless connection, and 3G through my Blackberry all of which I will use to connect to a desktop in Sacramento, CA. What’s the likely outcome? There will probably be some frustrating moments when I can’t get to the View environment or maybe I will be unable to resist viewing rich media on a local client. On that note, I can’t stop raving about my ability to stream Hulu through the virtual desktop with smooth playback and synchronized audio over my Comcast internet connection. PCoIP performance is fantastic! I’m sure my employer loves to hear that I’m watching TV on a desktop hosted in our cloud…

Today, I’m onsite with a customer working on a VMware Health Check report. A virtual desktop limitation that I encountered almost immediately is that I need to connect to their environment using remote desktop from my fat client that is plugged into their network. My desktop in the cloud can’t help me here. I guess technically I could VPN into their network from the virtual desktop but depending on their VPN policies I would almost certainly drop the connection upon initiating the tunnel.

However, I did manage to take my notes and customize the report within my virtual desktop. I’m using Microsoft’s Live Mesh to sync screenshots, performance captures, etc over from the fat client where I’ve copied these items from the customer’s systems over the LAN.

There are definitely some hoops that I have to jump through to make this all work out but two clear benefits are 1. The report and data is kept in the datacenter; 2. I simply disconnected the session last night with the document draft still open and it was waiting for me this morning when I launched the virtual desktop. One word: AGILITY! I can access my apps, data, tools… everything from anywhere that I have a connection. Which, with the prevalence of connectivity be it through 3G, 4G, or wifi, I rarely find a place where I cannot get connected. We’ll see what happens the next time I hop on a plane.

Blank Screen on Windows 7 VMware View Desktop Using PCoIP

Updated 11/5/2010:

You may want to check out this KB from VMware in reference to this issue.

———————————————- ———————————————————

Original Post:

I was recently setting up a Windows 7 image for our VMware View environment and the overall process was very quick and simple compared to other OSes due to the simple, fast installation of Windows 7. However, when I first attempted to connect to a View desktop using PCoIP, I was presented with a blank or black screen that in a minute just closed seemingly refusing the connection. The console of the virtual desktop appeared to logon with the specified user but the remote session was not working. Quickly, I remembered setting up Windows 7 in another View environment and recalled the need to change the video driver. Here is a quick “how to” that will get you up and running with Windows 7 in VMware View using PC-over-IP. NOTE: Windows 7 is still “experimental” in VMware View.

Right-click Computer then click Manage.

In Device Manager, expand Display adapters, right-click VMware SVGA 3D, click Update Driver Software…

Click Browse my computer for driver software

Click Let me pick from a list…


Click Have Disk…

Browse to C:\Program Files\Common Files\VMware\Drivers\video and click OK.

Choose VMware SVGA II and click Next.

The driver is installing…

Done. Click Close and restart Windows.

Finally, it is a good idea to increase the amount of video memory by editing the virtual machine settings. 40 MB seems to be a recommended amount floating around the blogosphere.