Apr
15

Once you have ESXi 5.1 up and running download the CSR1000v OVA (Open Virtualization Archive) file here. Note that you will need to use your Cisco CCO login to download. Once the download is complete you need to open your vSphere Client and connect to your ESXi server. Once connected select the Deploy OVF Template option from the file menu.

CSR1000v Installation

Select the source of the OVA file you downloaded from your local machine.

The next screen should be simular to the one below after the source has been selected.

The next screen will display the name for the virtual machine. You can change this if you would like.

Now you need to select the datastore:

Next you will be offered the select the disk format. The default option of Thick Provision Lazy Zeroed is fine but for better performance you can select Thick Provision Eager Zeroed. To read more on the options go here.

The next screen will ask you for the mappings for the three Gigabit Ethernet interfaces that are defined in the OVF file. Just select the default here for now and we’ll come back to them after our machine is installed as we need to make a couple other changes anyways that can’t be done here.

The next screen will be a summary screen so just click finish and then VMWare will start importing the OVA file.

It may take a few minutes to import depending on the connection speed between your vSphere client and the ESXi server.

Once your machine uploaded you can then edit the settings.

Two things we want to do here. First is that we want to alter the Network Adapter settings if needed and add a serial port so that we can use the virtual serial on the CSR1000v. This functionality (network based serial port) requires the Enterprise version of ESXi 5.1. I would recommend that you use the demo version which gives you 59 days unless you have to reinstall the demo ;-)

From here click “Add..” and then select Serial Port and click “Next”

Select “Connect via Network”

Now here is where you want to select “Server” and then enter the IP address of the ESXi server along with the TCP port you want to assign to this machine. Also check “Connect at power on”.

After this click finish and we’re ready to start up the CSR1000v. Once booted you should see the GRUB menu. Select CSR 1000V Serial Console and hit enter.

You will need to alter the default security settings for the ESXi server to allow TCP port 2005 or whatever port you selected to allow you to telnet to the CSR1000v’s serial port. To alter the security settings go to the ESXi’s configuration and then select “Security Profile”

From here click on the Firewall Properties link on the right. It will open a window like below:

Here you will need to allow TCP port 2005 or if you are in a lab environment just select “VM serial port connected over network” which will open up all TCP high ports. Now telnet to the IP address of the ESXi machine and port number you entered for the serial port and you should see the router booting.

ccie2210$ telnet 10.4.101.168 2005
Trying 10.4.101.168...
Connected to 10.4.101.168.
Escape character is '^]'.
%IOSXEBOOT-4-BOOT_CDROM: (rp/0): Expanding super package on /bootflash
%IOSXEBOOT-4-BOOT_CDROM: (rp/0): Creating /boot/grub/menu.lst
%IOSXEBOOT-4-BOOT_CDROM: (rp/0): CD-ROM Installation finished
%IOSXEBOOT-4-BOOT_CDROM: (rp/0): Ejecting CD-ROM tray
%IOSXEBOOT-4-BOOT_CDROM: (rp/0): Rebooting from HD

    GNU GRUB  version 0.97  (638K lower / 3143552K upper memory)

-------------------------------------------------------------------
 0: CSR1000v - packages.conf
 1: CSR1000v - GOLDEN IMAGE
-------------------------------------------------------------------

      Use the ^ and v keys to select which entry is highlighted.
      Press enter to boot the selected OS, or 'c' for a command-line.

    Entry 0 will be booted automatically in 1 seconds.
  Booting 'CSR1000v - packages.conf'

root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
kernel /packages.conf rw quiet root=/dev/ram console= max_loop=64 HARDWARE=virt
ual SR_BOOT=bootflash:packages.conf
Calculating SHA-1 hash...done
SHA-1 hash:
        calculated   6f85a7c5:ebd0151a:b5ada94a:f7a7be4b:d7de713f
        expected     6f85a7c5:ebd0151a:b5ada94a:f7a7be4b:d7de713f
package header rev 1 structure detected
Calculating SHA-1 hash...done
SHA-1 hash:
        calculated   d4ab3a48:ae55f382:4237a12a:7be2c99b:d8f1f594
        expected     d4ab3a48:ae55f382:4237a12a:7be2c99b:d8f1f594
Package type:0x7531, flags:0x0
   [Linux-bzImage, setup=0x2e00, size=0x32e4620]
   [isord @ 0x7e0f6000, 0x1ef9800 bytes]

%IOSXEBOOT-4-WATCHDOG_DISABLED: (rp/0): Hardware watchdog timer disabled: watchdog device not found
%IOSXEBOOT-4-EUSB_PROVISIONING: (rp/0): Unsupported low capacity eUSB detected in VXE board

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706

Cisco IOS Software, IOS-XE Software (X86_64_LINUX_IOSD-ADVENTERPRISEK9-M), Version 15.3(2)S0a, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Sat 30-Mar-13 19:08 by mcpre

Cisco IOS-XE software, Copyright (c) 2005-2013 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.  For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

% failed to initialize nvram

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco CSR1000V (VXE) processor with 1141424K/6147K bytes of memory.
Processor board ID 940WXLZ2LL4
2 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
7774207K bytes of virtual hard disk at bootflash:.

SETUP: new interface GigabitEthernet2 placed in "shutdown" state
SETUP: new interface GigabitEthernet0 placed in "shutdown" state

Press RETURN to get started!

*Apr 16 03:17:58.679: %IOSXE_RP_NV-3-NV_ACCESS_FAIL: Initial read of NVRAM contents failed
*Apr 16 03:17:59.305: %VXE_THROUGHPUT-6-LEVEL: Throughput level has been set to 2500 kbps
*Apr 16 03:18:02.785: %IFMGR-7-NO_IFINDEX_FILE: Unable to open nvram:/ifIndex-table No such file or directory
*Apr 16 03:18:04.449: DSENSOR: protocol cdp is registered with sensor
*Apr 16 03:18:04.470: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
*Apr 16 03:18:04.911: %LINK-3-UPDOWN: Interface Lsmpi0, changed state to up
*Apr 16 03:18:04.912: %LINK-3-UPDOWN: Interface EOBC0, changed state to up
*Apr 16 03:18:04.912: %LINEPROTO-5-UPDOWN: Line protocol on Interface VoIP-Null0, changed state to up
*Apr 16 03:18:04.912: %LINEPROTO-5-UPDOWN: Line protocol on Interface LI-Null0, changed state to up
*Apr 16 03:18:05.356: %IOSXE_MGMTVRF-6-CREATE_SUCCESS_INFO: Management vrf Mgmt-intf created with ID 1, ipv4 table-id 0x1, ipv6 table-id 0x1E000001
*Apr 16 03:18:05.406: %LINK-3-UPDOWN: Interface GigabitEthernet0, changed state to down
*Apr 16 03:18:05.410: %LINK-3-UPDOWN: Interface LIIN0, changed state to up
*Apr 16 03:18:05.455: %DYNCMD-7-CMDSET_LOADED: The Dynamic Command set has been loaded from the Shell Manager
*Apr 16 03:18:05.871: %LINK-3-UPDOWN: Interface GigabitEthernet1, changed state to down
*Apr 16 03:18:05.880: %LINK-3-UPDOWN: Interface GigabitEthernet2, changed state to down
*Apr 16 03:17:58.400: %IOSXE-5-PLATFORM: R0/0: xinetd[19698]: xinetd Version 2.3.14 started with no options compiled in.
*Apr 16 03:17:58.400: %IOSXE-5-PLATFORM: R0/0: xinetd[19698]: Started working: 1 available service
*Apr 16 03:17:58.434: %IOSXE-5-PLATFORM: R0/0: xinetd[19709]: xinetd Version 2.3.14 started with no options compiled in.
*Apr 16 03:17:58.434: %IOSXE-5-PLATFORM: R0/0: xinetd[19709]: Started working: 3 available services
*Apr 16 03:17:59.634: %CMRP-5-PRERELEASE_HARDWARE: R0/0: cmand:  F0 is pre-release hardware
*Apr 16 03:18:00.823: %CMLIB-6-THROUGHPUT_VALUE: R0/0: cmand:  Throughput license found, throughput set to 2500 kbps
*Apr 16 03:18:03.063: %CPPHA-7-START: F0: cpp_ha:  CPP 0 preparing image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-csr
*Apr 16 03:18:03.179: %CPPHA-7-START: F0: cpp_ha:  CPP 0 startup init image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-csr
*Apr 16 03:18:06.407: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0, changed state to down
*Apr 16 03:18:06.409: %LINEPROTO-5-UPDOWN: Line protocol on Interface LIIN0, changed state to up
*Apr 16 03:18:06.872: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1, changed state to down
*Apr 16 03:18:06.881: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2, changed state to down
*Apr 16 03:18:07.325: %DYNCMD-7-PKGINT_INSTALLED: The command package 'platform_trace' has been succesfully installed
*Apr 16 03:18:25.342: AUTOINSTALL: GigabitEthernet1 is assigned 10.4.101.228
*Apr 16 03:18:41.567: %LINK-5-CHANGED: Interface GigabitEthernet0, changed state to administratively down
*Apr 16 03:18:41.782: %LINK-5-CHANGED: Interface GigabitEthernet2, changed state to administratively down
*Apr 16 03:18:43.361: %IOSXE_OIR-6-INSCARD: Card (rp) inserted in slot R1
*Apr 16 03:18:43.361: %IOSXE_OIR-6-INSCARD: Card (fp) inserted in slot F0
*Apr 16 03:18:43.361: %IOSXE_OIR-6-ONLINECARD: Card (fp) online in slot F0
*Apr 16 03:18:43.423: %SYS-5-RESTART: System restarted --
Cisco IOS Software, IOS-XE Software (X86_64_LINUX_IOSD-ADVENTERPRISEK9-M), Version 15.3(2)S0a, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Sat 30-Mar-13 19:08 by mcpre
*Apr 16 03:18:43.501: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF
*Apr 16 03:18:43.501: %CRYPTO-6-GDOI_ON_OFF: GDOI is OFF
*Apr 16 03:18:43.540: %LINK-3-UPDOWN: Interface GigabitEthernet1, changed state to up
*Apr 16 03:18:44.541: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1, changed state to up
%Error opening tftp://255.255.255.255/network-confg (Timed out)
%Error opening tftp://255.255.255.255/cisconet.cfg (Timed out)
%Error opening tftp://255.255.255.255/router-confg (Timed out)
%Error opening tftp://255.255.255.255/ciscortr.cfg (Timed out)
%Error opening tftp://255.255.255.255/network-confg (Timed out)
%Error opening tftp://255.255.255.255/cisconet.cfg (Timed out)
%Error opening tftp://255.255.255.255/router-confg (Timed out)
%Error opening tftp://255.255.255.255/ciscortr.cfg (Timed out)
Router>

At this point I haven’t activated the license so the router is limited to 2.5Mbps throughput. To activate the demo license use the platform hardware throughput level command in global configuration mode.

Router>en
Router#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
Router(config)#platform hardware throughput level ?
  10000  throughput in kbps
  25000  throughput in kbps
  50000  throughput in kbps

Router(config)#platform hardware throughput level 50000
	 Feature Name:prem_eval

PLEASE  READ THE  FOLLOWING TERMS  CAREFULLY. INSTALLING THE LICENSE OR
LICENSE  KEY  PROVIDED FOR  ANY CISCO  PRODUCT  FEATURE  OR  USING SUCH
PRODUCT  FEATURE  CONSTITUTES  YOUR  FULL ACCEPTANCE  OF  THE FOLLOWING
TERMS. YOU MUST NOT PROCEED FURTHER IF YOU ARE NOT WILLING TO  BE BOUND
BY ALL THE TERMS SET FORTH HEREIN.

Use of this product feature requires  an additional license from Cisco,
together with an additional  payment.  You may use this product feature
on an evaluation basis, without payment to Cisco, for 60 days. Your use
of the  product,  including  during the 60 day  evaluation  period,  is
subject to the Cisco end user license agreement

http://www.cisco.com/en/US/docs/general/warranty/English/EU1KEN_.html

If you use the product feature beyond the 60 day evaluation period, you
must submit the appropriate payment to Cisco for the license. After the
60 day  evaluation  period,  your  use of the  product  feature will be
governed  solely by the Cisco  end user license agreement (link above),
together  with any supplements  relating to such product  feature.  The
above  applies  even if the evaluation  license  is  not  automatically
terminated  and you do  not receive any notice of the expiration of the
evaluation  period.  It is your  responsibility  to  determine when the
evaluation  period is complete and you are required to make  payment to
Cisco for your use of the product feature beyond the evaluation period.

Your  acceptance  of  this agreement  for the software  features on one
product  shall be deemed  your  acceptance  with  respect  to all  such
software  on all Cisco  products  you purchase  which includes the same
software.  (The foregoing  notwithstanding, you must purchase a license
for each software  feature you use past the 60 days evaluation  period,
so  that  if you enable a software  feature on  1000  devices, you must
purchase 1000 licenses for use past  the 60 day evaluation period.)    

Activation  of the  software command line interface will be evidence of
your acceptance of this agreement.

ACCEPT? (yes/[no]): yes

*Apr 16 04:30:21.271: %LICENSE-6-EULA_ACCEPTED: EULA for feature prem_eval 1.0 has been accepted. UDI=CSR1000V:940WXLZ2LL4; StoreIndex=0:Built-In License Storage
Router(config)#
*Apr 16 04:30:23.939: %VXE_THROUGHPUT-6-LEVEL: Throughput level has been set to 50000 kbps
Router(config)#

This technically isn’t needed if you are going to use it in a lab environment. At this point your router is ready to go.

Router(config)#do sho run
Building configuration...

Current configuration : 1047 bytes
!
version 15.3
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console serial
platform hardware throughput level 50000
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-intf
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
!
no aaa new-model
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
spanning-tree extend system-id
!
!
redundancy
 mode none
!
!
!
!
!
!
ip tftp source-interface GigabitEthernet0
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet1
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet2
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet0
 vrf forwarding Mgmt-intf
 no ip address
 shutdown
 negotiation auto
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
!
!
!
control-plane
!
 !
 !
 !
!
!
!
!
line con 0
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 login
!
!
end

Router(config)#

Now that my router is ready to go I’ll clone it because a single router isn’t much to learn on. Ideally if you are going to reuse this machine in the future you wait to activate the license after the device is cloned. To clone the machine go to the server configuration tab and then select the datastore where you installed the CSR1000v onto and then right click on it. From there select “Browse Datastore…”

From here go under the directory for the CSR1000v and copy the contents of the directory. From there click on the root of the datastore and then select the folder icon to add a new directory.

Enter the directory name and then paste the contents into the new directory. After it has been pasted in, right click on the “Cisco CSR1000v Cloud Services Router.vmx” file and select “Add to Inventory”. Change the default name if you would like and select the “Resource Pool” and finally finish.

You should now see the second CSR1000v in your ESXi server’s inventory. From there we’ll edit the settings to change the TCP port number for the virtual serial port and apply any VLANs from the ESXi to the CSR1000v’s Gigabit Ethernet interfaces.

You are now ready to start up the second router. Note that after you start it VMware will ask you if you copied the virtual machine or moved it. Select “I copied it” and then click ok. Your router will now boot up.

ccie2210$ telnet 10.4.101.168 2006
Trying 10.4.101.168...
Connected to 10.4.101.168.
Escape character is '^]'.
    Entry 0 will be booted automatically in 1 seconds.
  Booting 'CSR1000v - packages.conf'

root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
kernel /packages.conf rw quiet root=/dev/ram console= max_loop=64 HARDWARE=virt
ual SR_BOOT=bootflash:packages.conf
Calculating SHA-1 hash...done
SHA-1 hash:
        calculated   6f85a7c5:ebd0151a:b5ada94a:f7a7be4b:d7de713f
        expected     6f85a7c5:ebd0151a:b5ada94a:f7a7be4b:d7de713f
package header rev 1 structure detected
Calculating SHA-1 hash...done

The router will now be booting up and ready to go.

Use the same procedure outlined above to create as many routers as you like. You will of course want to create VLANs within the ESXi server to allow communication between the routers. For an overview of how to manage the ESXi’s networking configuration go here.

Tomorrow I’ll post another installation guide on how to install the CSR1000v in VMware Workstation and use it as a production router providing internet access.

Lastly if anyone would like to try out a few of these send me an email and I’ll get you access.

About Brian Dennis, CCIE #2210:

Brian Dennis has been in the networking industry for more than 22 years, with a focus on Cisco networking for the past 16 years. Brian achieved his first CCIE in Routing & Switching in 1996, and is currently the only ten year CCIE that holds five CCIE certifications. Prior to working with INE, Brian taught and developed CCIE preparation courses for various well known training organizations. Brian not only brings his years of teaching experience to the classroom, but also years of real world enterprise and service provider experience.

Find all posts by Brian Dennis, CCIE #2210 | Visit Website


You can leave a response, or trackback from your own site.

26 Responses to “Installing the Cloud Services Router 1000v in VMware ESXi 5.1”

 
  1. Joshua Walton, CCIE #19763 - Security says:

    Thanks, Brian!

  2. Daniel Dib says:

    Very cool stuff Brian. I am going to play around with these later. Just need to get the virtual networking stuff done so that I can connect multiple routers.

    Is that the only way of cloning device? Shouldn’t we be able to clone it directly without going to the datastore?

  3. Alex Berdan says:

    It’d be nice to see a “how to” with Nexsus 1000v as well.

  4. Ismail Elshalh says:

    Lovely!

    So eventually I am going to play with LISP!

  5. Nick Bonifacio says:

    Thanks Brian, your instructions worked flawlessly. I am going to create a lab/dev VDC in our 7000s and connect a few of these up for testing. I am wondering if there is an easy way to integrate these to the Nexus 1000V or maybe just use a breakout switch as well.

  6. Shuai says:

    Not sure if this only happens to me, when I try to download from Cisco, I got File is unavailable error…

    Does this mean cisco cut the file?

  7. Salman Naqvi says:

    Awesome post. Great starter guide. Keep these coming :)

  8. [...] To install the CSR you can go to Cisco CSR config guide or read the guide by Brian Dennis at INE [...]

  9. [...] the software here. You can also check out some tutorials such as this blog post from INE on Installing the Cloud Services Router in VMWare ESXi, or this work by Daniel Dib on integrating the CSR1000V with Dynamips. As the CSR is distributed in [...]

  10. [...] Miroslaw Burnejko´s Build IOS-XE Lab for Free INE´s Installing the cloud services router 1000v in ESXi 5.1 [...]

  11. Ronald Lopez says:

    I had issues when selecting Serial Console got a error about dumb serial port, anyways after checking Cisco site I found that you can select Virtual Console and once boot up just configure platform console serial. It works nice!!

    http://www.cisco.com/en/US/docs/routers/csr1000/software/configuration/swinstallcsr.html#wp1197812

  12. Vivek says:

    Hi brian,

    Excellent how to article. Can you advise when you would be coming out with the “how to install the CSR1000v in VMware Workstation”.

    Thanks

  13. Viktor says:

    Hi,

    Does it really need 4096MB Ram each instance? Anytime I lower it i got crash with not enough memory…. Maybe its just my ESXi 5.5 … 4GB Memory each router eats up quickly my home server…I have to think of smashing that 96GB of DDR3 10600 RDIMM Modules into it

  14. Danny says:

    Im new to ESXi and csr1000v for that matter.

    Is this this the OVA file that we should use?
    csr1000v-universalk9.03.12.00.S.154-2.S-std.ova

    How do you connect the server to your physical switch?

    What kind of HW (memory,CPU ) would be applicable for setting up 20 routers?

    Thanks,

  15. Danny says:

    How do you connect the server to the physical switch?

    And how much memory would you need to allocated for each router , thats 20 virtual ones right?

  16. Ryan says:

    I am unable to get the serial port working on ESXi v5.5U1. Has anyone been able to get this working on this version?

  17. Rafay says:

    Hi
    I have added Network Serial Port , just like Brian Dennis explained earlier, telnet://my esxi ip:2001
    I have Allowed TCP port 2001 in ESXI 5.1 FW.

    When I telnet to
    telnet myesxiip 2001 I can connect but I don’t see any output from router nor I see any thing I type.

 

Leave a Reply

Categories

CCIE Bloggers