• A Look Inside Webster’s Lab Phase 2

    I am in the process of rebuilding my lab with Microsoft Windows Server 2012, Hyper-V 3 and System Center Virtual Machine Manager (SCVMM).  This rebuilding project has become quite a learning experience.  This article highlights some of the “learning opportunities” I was blessed with (sarcasm intended).

    Webster’s Lab Phase 1 is shown in this article: https://carlwebster.com/a-look-inside-websters-lab-2/

    Goals

    The goals for this lab rebuilding project are to get experience with the Microsoft System Center products, Hyper-V and automation.  Far too many enterprises that I work with still do the majority of their tasks with manual processes.  I need to learn how to use the System Center products and learn what they are capable of doing for customers.  Also, since I am one of the growing number of people who no longer believe that XenServer is Enterprise capable, I am moving to Hyper-V.  Several of the Citrix partners I work with and through also have customers moving away from XenServer and mainly to Hyper-V.  As far as automation, that is the present and future.  You can’t have a Private or Public Cloud without automation.  Even a well-documented manual process is rife with errors and shortcomings along with the possibility (or is that probability) that whoever does the build process will make errors whether they are intended or not, intentional or not.  Automation is the only way to achieve high quality, reliable and duplicable processes and products.

    Lab Setup

    I have two older lab servers that will not install Windows Server 2012 but they work perfectly with XenServer 6.2.  I installed XenServer 6.2 on both servers, created a pool and then created two Server 2012 Virtual Machines (VMs).  Each VM became a 2012 Domain Controller (DC) for the WebstersLab.com Forest and Domain.  When creating the Forest/Domain, I set the Forest Functional Level (FFL) and Domain Function Level (DFL) to Windows Server 2012.

    The first thing I did after creating the first DC was to configure DNS Aging and Scavenging in DNS Manager (Figures 1 through 4).

    Figure 1
    Figure 1
    Figure 2
    Figure 2
    Figure 3
    Figure 3
    Figure 4
    Figure 4

    The next thing I did was to use the Group Policy Management console to create a Group Policy using Preferences to configure the DC that holds the PDCe FSMO Role as shown in Figure 5.  This Group Policy configures this specific DC to be the authoritative time source for the WebstersLab.com domain.

    Note: PDCe – Primary Domain Controller emulator, see http://technet.microsoft.com/en-us/library/cc780487(v=ws.10).aspx

    Note: FSMO – Flexible Single-Master Operations, see http://technet.microsoft.com/en-us/library/cc961936.aspx

    Figure 5
    Figure 5

    Then I created a WMI Filter to select the DC that holds the PDCe FSMO Role (Figure 6).

    Figure 6
    Figure 6

    Next, I added the WMI Filter to the GPO (Figure 7).

    Figure 7
    Figure 7

    And finally, link the GPO to the Domain Controllers OU (Figures 8, 9  and 10).

    Figure 8
    Figure 8
    Figure 9
    Figure 9
    Figure 10
    Figure 10

    Because this is a Computer policy with a WMI Filter, either reboot the DC or run GPUPDATE /FORCE and verify the new GPO and GPP applied (Figure 11).

    Figure 11
    Figure 11

    Verify the registry entries (Figure 12 and 13).

    Figure 12
    Figure 12
    Figure 13
    Figure 13

    Learning Opportunity #1

    After the domain setup and configuration stuff was completed, next up was installing the first of five Hyper-V hosts.  I read the first two chapters of Aidan Finn’s Windows Server 2012 Hyper-V Installation and Configuration Guide along with the TechNet article http://technet.microsoft.com/en-us/library/jj735302.aspx#bkmk_2.  When I installed the Hyper-V Role, I checked the box seen in Figure 14.

    Figure 14
    Figure 14

    That was a mistake. When I went to run the following commands from Aidan’s book and the TechNet article, nothing worked:

    Add-VMNetworkAdapter –ManagementOS –Name Migration  –SwitchName "Converged Switch"
    Add-VMNetworkAdapter –ManagementOS –Name Cluster    –SwitchName "Converged Switch"
    Add-VMNetworkAdapter –ManagementOS –Name Management –SwitchName "Converged Switch"
    
    Set-VMNetworkAdapter –ManagementOS –Name "Migration"  -MinimumBandwidthWeight "30"
    Set-VMNetworkAdapter –ManagementOS –Name "Cluster"    -MinimumBandwidthWeight "20"
    Set-VMNetworkAdapter –ManagementOS –Name "Management" -MinimumBandwidthWeight "5"
    

    I found the following errors in the server’s System event log:

    Event ID: 82, Source Hyper-V-VmSwitch – Failed to complete bandwidth policy operation ‘Policy Set’ on port 44BB72E9-3394-4BE7-97AC-A6143A681DD7 (Friendly Name: Migration) on switch 65BB7332-04D3-4302-A3F5-95B4B0303849 (Friendly Name: Converged Switch) Reservation – 0, Weight – 20, Limit – 0, BurstLimit – 0, BurstSize – 0, Reason – The switch’s bandwidth reservation mode does not support this bandwidth configuration, Status = The request is not supported..

    Event ID: 78, Source Hyper-V-VmSwitch -Failed to complete operation ‘Policy Add’ on port 44BB72E9-3394-4BE7-97AC-A6143A681DD7 (Friendly Name: Migration) on switch 65BB7332-04D3-4302-A3F5-95B4B0303849 (Friendly Name: Converged Switch) Property Id {24ad3ce1-69bd-4978-b2ac-daad389d699c} Instance Id {a26eb467-9414-437b-9a3e-325aca3ec730}. Status = The request is not supported..

    Event ID: 82, Source Hyper-V-VmSwitch -Failed to complete bandwidth policy operation ‘Policy Set’ on port 630000B1-9369-4F96-9856-12176F8441D7 (Friendly Name: Cluster) on switch 65BB7332-04D3-4302-A3F5-95B4B0303849 (Friendly Name: Converged Switch) Reservation – 0, Weight – 5, Limit – 0, BurstLimit – 0, BurstSize – 0, Reason – The switch’s bandwidth reservation mode does not support this bandwidth configuration, Status = The request is not supported..

    Event ID: 78, Source Hyper-V-VmSwitch -Failed to complete operation ‘Policy Add’ on port 630000B1-9369-4F96-9856-12176F8441D7 (Friendly Name: Cluster) on switch 65BB7332-04D3-4302-A3F5-95B4B0303849 (Friendly Name: Converged Switch) Property Id {24ad3ce1-69bd-4978-b2ac-daad389d699c} Instance Id {ac849e68-5d24-473d-8d56-0a095df2816a}. Status = The request is not supported..

    Event ID: 82, Source Hyper-V-VmSwitch -Failed to complete bandwidth policy operation ‘Policy Set’ on port 9FE32C66-1930-4237-BF81-D2CC8278240E (Friendly Name: Management) on switch 65BB7332-04D3-4302-A3F5-95B4B0303849 (Friendly Name: Converged Switch) Reservation – 0, Weight – 5, Limit – 0, BurstLimit – 0, BurstSize – 0, Reason – The switch’s bandwidth reservation mode does not support this bandwidth configuration, Status = The request is not supported..

    Event ID: 78, Source Hyper-V-VmSwitch -Failed to complete operation ‘Policy Add’ on port 9FE32C66-1930-4237-BF81-D2CC8278240E (Friendly Name: Management) on switch 65BB7332-04D3-4302-A3F5-95B4B0303849 (Friendly Name: Converged Switch) Property Id {24ad3ce1-69bd-4978-b2ac-daad389d699c} Instance Id {d9885672-3a23-4321-93e9-4ccd68d8bb15}. Status = The request is not supported..

    Someone told me by selecting that checkbox, Hyper-V created an “old style” virtual switch.  I am assuming that means a 2008 R2 Hyper-V compatible virtual switch.  That “old style” virtual switch does not support the new Server 2012 Hyper-V virtual networking.  What I had to do was remove the Hyper-V role, reboot the server, add the Hyper-V role, not select the checkbox and then all the commands worked perfectly.  I probably could have used PowerShell to remove the virtual switch and start over but I didn’t think about it at the time.

    new-vmswitch -name "Converged Switch" -MinimumBandWidthMode Weight -NetAdapterName "Network Team"
    
    rename-vmnetworkadapter -managementos -name "Converged Switch" -NewName "Management"
    
    Add-VMNetworkAdapter –ManagementOS –Name Migration  –SwitchName "Converged Switch"
    Add-VMNetworkAdapter –ManagementOS –Name Cluster    –SwitchName "Converged Switch"
    
    Set-VMNetworkAdapter –ManagementOS –Name "Migration"  -MinimumBandwidthWeight "30"
    Set-VMNetworkAdapter –ManagementOS –Name "Cluster"    -MinimumBandwidthWeight "20"
    Set-VMNetworkAdapter –ManagementOS –Name "Management" -MinimumBandwidthWeight "5"
    
    set-vmswitch "Converged Switch" -defaultflowminimumbandwidthweight "10"
    

    Once that was done, the other four Hyper-V hosts were a piece of cake to install and configure.

    Learning Opportunity #2

    The next learning opportunity came when trying to add my five Hyper-V hosts into SCVMM.  The process kept giving me the following error:

    Error (2910)
    VMM does not have appropriate permissions to access the resource  on the server.
    Access is denied (0x80070005)

    I created another “Run As” account  using the Administrator Domain Admin account.  I thought that since that account by default is in the local Administrators group on every server, I should have no problem.  When I tried to join the hosts to SCVMM again, I received the same error message.

    I thought maybe it was a firewall issue so I stopped and disabled the Windows Firewall service on the SCVMM server and the five Hyper-V hosts.  When I tried adding the Hyper-V hosts to SCVMM, the process still did not work.

    Someone on Twitter asked if I could access the IPC$ share on each Hyper-V host and could they all be pinged.  Yes to both as shown in Figure 15.

    Figure 15
    Figure 15

    Someone else on Twitter asked if I could do any remote management commands against the Hyper-V hosts.  Yes I could, as shown in Figure 16.

    Figure 16
    Figure 16

    Then @virtualfat on Twitter sent me to http://social.technet.microsoft.com/Forums/systemcenter/en-US/4bd9be4b-0ff9-46f3-bf32-1b7c1245c494/scvmm-sp1-beta-and-server-2012-error-2910 which described the exact error message I was getting AND had a solution.  I needed to run the following commands on each Hyper-V host:

    reg add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /v LocalAccountTokenFilterPolicy /t REG_DWORD /d 1 /f
    
    winrm set winrm/config/service/auth @{CredSSP="True"}
    winrm set winrm/config/winrs @{AllowRemoteShellAccess="True"}
    winrm set winrm/config/winrs @{MaxMemoryPerShellMB="2048"}
    winrm set winrm/config/client @{TrustedHosts="*"}
    winrm set winrm/config/client/auth @{CredSSP="True"}
    

    Once I ran those commands, I was able to successfully add all five Hyper-V hosts to SCVMM.

    UPDATE: My friend Michael B. Smith says the line winrm set winrm/config/client @{TrustedHosts=”*”} is a security risk.  “TrustedHosts=* means you are open to significant potential issues.”  Michael says instead of “*” to use the name(s)  of the SCVMM host(s).  For me, that would mean the line should be winrm set winrm/config/client @{TrustedHosts=”SCVMM”}.

    Learning Opportunity #3

    I was never able to figure out how to get my Synology 412+ NAS configured in SCVMM 2012 SP1.  So I added a 2TB drive to one of my XenServer hosts, configured the drive and installed a Server 2012 VM.  I then installed some well-known vendor’s iSCSI NAS software and their SMI-S provider since that is what is supported by SCVMM (Figure 17).

    Figure 17
    Figure 17

    After many, many hours of playing with the iSCSI software and SMI-S provider and dealing with very sparse documentation, I could not get SCVMM 2012 SP1 to even acknowledge the existence of the SMI-S Provider.  i.e. the Next button never became active as shown in Figure 18.

    Figure 18
    Figure 18

    I checked this vendor’s support forum and found I was not the only one having this issue.  I emailed support and the response I received was basically “Microsoft has a bug in SCVMM 2012 SP1 that prevents any SMI-S Provider from working.  Microsoft needs to fix their bug and besides, our Provider is experimental and we don’t provide support for it”.  Well OK then.  I uninstalled all of that vendor’s software and will look at using an SMB3 file share.

    Learning Opportunity #4

    The next thing I needed to do in SCVMM was to get my 879GB of ISO files in my Synology NAS into the Library.  This was also not easy to do but I finally figured it out.

    Expand Library Servers, right-click the SCVMM server name and select Add Library Shares (Figure 19).

    Figure 19
    Figure 19

    Select Add Unmanaged Share…(Figure 20).

    Figure 20
    Figure 20

    Enter the path to the ISO share on the Synology and click OK (Figure 21).

    Figure 21
    Figure 21

    Click Next (Figure 22).

    Figure 22
    Figure 22

    Click Add Library Share (Figure 23).

    Figure 23
    Figure 23

    Depending on the number and size of the files in the ISO share, it could take many minutes to well over an hour and the ISO files from the ISO share on the Synology start appearing in the Library and the Add Library Shares wizard closes.

    It took six minutes before the first files appeared in my Library as shown in Figure 24.

    Figure 24
    Figure 24

    My next step is to get some storage for VMs into the Fabric part of SCVMM so I can start creating VMs, Templates, Profiles and Services so I can finally use my new Private Cloud.

    About Carl Webster

    Webster is a Sr. Solutions Architect for Choice Solutions, LLC and specializes in Citrix, Active Directory and Technical Documentation. Webster has been working with Citrix products for many years starting with Multi-User OS/2 in 1990.

    View all posts by Carl Webster

    to “A Look Inside Webster’s Lab Phase 2”

    1. Brian L Says:

      I noticed you are running full windows server 2012 with hyper-v role. Is there an overhead associated with that vs running just hyper-v server (core install basically?)

      Reply

      • Carl Webster Says:

        Yes, but I found the GUI to be easier to work with for configuration. Once I had everything configured, I removed the GUI.

        Webster

        Reply

    2. James Smith Says:

      Carl, did you ever get your Synology 412+ NAS working with SC VMM as a iSCSI target? Or did you end up staying with using SMB 3 file shares? I’m looking at adding a Synology or Drobo iSCSI NAS to my lab and have been looking at the pros and cons of each model. The Synology looks good at not breaking the bank.

      Thanks for sharing.

      Reply

      • Carl Webster Says:

        I am sure I did but the lab is very fluid. It went from Hyper-V to ESX to XenServer and more than likely going back to Hyper-V. It all depends upon the projects I am doing and what the customer needs. I like my Synology much better than the original QNap.

        Webster

        Reply

    3. Emma Citrix Says:

      Hey Carl,
      Thanks for sharing such an excellent post.This will help me a lot like a tutorial.Thanks again.

      Reply

    4. Leo Says:

      Awesome Carl! I got to learn this automation stuff.

      Reply

    5. Eric Says:

      Great article Carl, I’m in the same process. Great checklist as well for new environments.

      Reply

    6. Jamie T Says:

      Wow..great article.

      Reply

    Leave a Reply