OVM 3.3.4 Released

OVM 3.3.4 has finally been released after what seems like months since the last update.  Even so- it appears that there are only minor enhancements and mostly bug fixes in this release.  You can find them under patch 20492240 and 20492250.

 

I was hoping for some new features or major updates in this release since it’s been so long.  Will have to hold my breath a little longer I guess :).

Advertisements

OVM Disaster Recovery In A Box (Part 4 of 5)

Now that you’ve touched a file inside the VM- we have a way to prove that the VM which will be replicated to the other side via replication is actually the one we created.  Apparently in my case, faith is overrated.

 

Now that I’ve fire-hosed a TON of information at you on how to set up your virtual prod and dr sites, this would be a good breaking point to talk a little about how the network looks from a 10,000 foot view.  Here’s a really simple diagram that should explain how things work.  And when I say simple, we’re talking crayon art here folks.  Really- does anyone have a link to any resources on the web or in a book that could help a guy draw better network diagrams?  Ok- I digress.. here’s the diagram:

OVM DR Network Diagram

 

One of the biggest take aways from this diagram highlights something that a LOT of people get confused about.  In OVM DR- you do NOT replicate OVM Manager, the POOL filesystem or the OVM servers on the DR side.  In other words, you don’t replicate the operating environment, only the contents therein (i.e. the VM’s via their storage repositories).  You basically have a complete implementation of OVM at each location just as if it were a standalone site.  The only difference is that some of the repositories are replicated.  The only other potential difference (and I don’t show it or deal with it in my simulation) is RAW lun’s presented to the VM.  Those would have to be replicated at the storage layer as well.

 

I’ve not bothered to mess up the diagram with the VM or Storage networks- you know they’re there and that they’re serving their purpose.  You can see that replication is configured between the PROD Repo LUN and a LUN in DR.  This would be considered an Active/Passive DR Solution.  In this scenario, I don’t show it but you could potentially have some DR workloads running at the DR site.  It isn’t replicated back to PROD but note the next sentence. Now, some companies might have a problem with shelling out all that money for the infrastructure at the DR site and have it sitting unused until a DR event occurred.  Those companies might just decide to run some of their workload in the DR site and have PROD be its DR.  In this Active/Active scenario, your workflow would be pretty much the same, there are just more VM’s and repositories at each site so you need to be careful and plan well.  Here is what an Active/Active configuration would look like:

OVM DR Network Diagram active active

 

Again- my article doesn’t touch on Active/Active but you could easily apply the stuff you learn in these 5 articles to accommodate an Active/Active configuraiton fairly easily.  We’ll be focusing on Active/Passive just as a reminder.  We now have a Virtual Machine running in PROD to facilitate our replication testing.  Make sure the VM runs and can ping the outside network so we know we have a viable machine.  Don’t be expecting lightning performance either, we’re running a VM inside a VM which is inside of a VM.  Not exactly recommended for production use.  Ok- DO NOT use this as your production environment.  There- all the folks who ignore the warnings on hair dryers about using them in the shower should be covered now.

 

Below are the high level steps used to fail over to your DR site.  Once you’ve accomplished this, make sure to remember failback.  Most people are usually so excited about getting the failover to work that they forget they’ll have to fail back at some point once things have been fixed in PROD.

 

FAILOVER (this works if you’re doing a controlled fail over or if a real failure at prod occurs):

  • Ensure all PROD resources are nominal and functioning properly
  • Ensure all DR resources are nominal and functioning properly
  • Ensure replication between PROD and DR ZFS appliances is in place and replicating
  • on ZFSDR1, Stop replication of PROD_REPO
  • on ZFSDR1, Clone PROD_REPO project to new project DRFAIL
  • Rescan physical disk on ovmdr1 (may have to reboot to see new LUN)
  • Verify new physical disk appears
  • Rename physical disk to PROD_REPO_FAILOVER
  • Take ownership of replicated repository in DR OVM Manager
  • Scan for VM’s in the unassigned VM’s folder
  • Migrate the VM to the DR pool
  • Start the VM
  • Check /var/tmp/ and make sure you see the ovmprd1 file that you touched when it was running in PROD.  This proves that it’s the same VM
  • Ping something on your network to establish network access
  • Ping or connect to something on the internet to establish external network access

 

FAILBACK:

  • Ensure all PROD resources are nominal and functioning properly
  • Ensure all DR resources are nominal and functioning properly
  • Restart replication in the opposite direction from ZFSDR1 to ZFSPRD1
  • Ensure replication finishes successfully
  • Rescan physical disks on ovmprd1
  • Verify your PROD Repo LUN is still visible and in good health
  • Browse the PROD Repo and ensure your VM(s) are there
  • Power on your VM’s in PROD and ensure that whatever data was modified while in DR has been replicated back to PROD successfully.
  • Ping something on your network to establish network access
  • Ping or connect to something on the internet to establish external network access

 

Now that we’ve shown you how all this works, I’ll summarize in part 5.

OVM Disaster Recovery In A Box (Part 5 of 5)

Finishing lineCongratulations in your successful failover!  There’s a lot of manual labor involved in failing OVM resources from one site to another- hence the reason for writing this article.  It’s not by any stretch of the imagination an easy or clear cut task.  I’ve attempted to show you in great detail how to set up your environment so it’s ready to accommodate Disaster Recovery.  Out of the box, it won’t work without some tweaks which I’ve shown you in the second and third postings.

 

Since the inception of these posts, Oracle has released a new product offering that is outlined in InfoDoc 1959182.1.  Head on over to MOS and give it a look see.  Basically it’s OVM with OEM integration for the DR failover piece.  That’s an oversimplification but essentially it makes what I laid out in the last 4 posts much easier.

 

I hope this has been an informative series of posts and that someone got some good information out of it!  Take care and happy replicating!

OVM Disaster Recovery In A Box (Part 3 of 5)

Continued from Part 2 of 5:

OVMMDR1

  • create a VM based on Oracle Linux 64bit OS
  • rename the VM to ovmmdr1
  • Give the VM 4gb memory and 2 cpu’s
  • Give the VM a 30gb hard drive
  • configure ovmmdr1 with the following network adapters:
    adapter 1: (Host Only) DR Management
    adapter 2: (NAT Network) DRPublic

  • boot the VM and install Oracle Linux 6.5 and select the Desktop server type.
    We do this so you have a GUI to log into- if that’s not a priority for you personally, then just pick Basic server

  • configure the VM with the following information:


Host Name: ovmmdr1
IP Address (eth0): 10.1.12.110
IP Netmask: 255.255.255.0
IP Address (eth1): 192.168.12.110
IP Netmask: 255.255.255.0
Default Router: 10.1.12.1
DNS Server: 127.0.0.1
Root Password: Way2secure

  • turn off iptables and selinux:


[root@ovmmdr1 ~]# service iptables stop ; chkconfig iptables off
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
[root@ovmmdr1 ~]#

[root@ovmmdr1 ~]# setenforce 0
setenforce: SELinux is disabled
[root@ovmmdr1 ~]#

[root@ovmmdr1 ~]# vi /etc/selinux/config

NOTE: Set SELINUX=disabled in file below:

This file controls the state of SELinux on the system.

SELINUX= can take one of these three values:

enforcing – SELinux security policy is enforced.

permissive – SELinux prints warnings instead of enforcing.

disabled – No SELinux policy is loaded.

SELINUX=disabled

SELINUXTYPE= can take one of these two values:

targeted – Targeted processes are protected,

mls – Multi Level Security protection.

SELINUXTYPE=targeted

  • add following line to /etc/hosts

    10.1.12.110 ovmmdr1

  • set parameters in /etc/sysconfig/network

    NETWORKING=yes
    HOSTNAME=ovmmdr1
    GATEWAY=192.168.12.1

  • reboot to make selinux disabled permanently

  • attach the OVM Manager 3.3.2 install ISO to the VM
  • run the createOracle.sh script to prep the VM for the installation of OVM Manager


[root@ovmmdr1 mnt]# ./createOracle.sh
Adding group 'oinstall' with gid '54323' ...
groupadd: group 'oinstall' already exists
Adding group 'dba'
groupadd: group 'dba' already exists
Adding user 'oracle' with user id '54322', initial login group 'dba',
supplementary group 'oinstall' and home directory '/home/oracle' ...
User 'oracle' already exists ...
uid=54321(oracle) gid=54322(dba) groups=54322(dba),54321(oinstall)
Creating user 'oracle' succeeded ...
For security reasons, no default password was set for user 'oracle'.
If you wish to login as the 'oracle' user, you will need to set a password for this account.

Verifying user ‘oracle’ OS prerequisites for Oracle VM Manager …
oracle soft nofile 8192
oracle hard nofile 65536
oracle soft nproc 2048
oracle hard nproc 16384
oracle soft stack 10240
oracle hard stack 32768
oracle soft core unlimited
oracle hard core unlimited
Setting user ‘oracle’ OS limits for Oracle VM Manager …
Altered file /etc/security/limits.conf
Original file backed up at /etc/security/limits.conf.orabackup
Verifying & setting of user limits succeeded …
Changing ‘/u01’ permission to 755 …
Changing ‘/u01/app’ permission to 755 …
Changing ‘/u01/app/oracle’ permission to 755 …
Modifying iptables for OVM
Adding rules to enable access to:
7002 : Oracle VM Manager https
54322 : Oracle VM Manager core via SSL
123 : NTP
10000 : Oracle VM Manager CLI Tool
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
iptables: Applying firewall rules: [ OK ]
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
iptables: Applying firewall rules: [ OK ]
Rules added.
[root@ovmmdr1 mnt]#

NOTE: you will need to gather the UUID of OVM Manager in production and install this instance
with that UUID. Run the following command on OVMMPRD1:

grep UUID /u01/app/oracle/ovm-manager-3/.config

  • run the runInstaller.sh script to install OVM Manager


[root@ovmmdr1 mnt]# ./runInstaller.sh -u {UUID from previous step}

Oracle VM Manager Release 3.3.2 Installer

Oracle VM Manager Installer log file:
/var/log/ovmm/ovm-manager-3-install-2015-03-04-170449.log

Please select an installation type:
1: Install
2: Upgrade
3: Uninstall
4: Help

Select Number (1-4): 1

Starting production with local database installation …

Verifying installation prerequisites …
*** WARNING: Recommended memory for the Oracle VM Manager server installation using Local MySql DB is 7680 MB RAM

One password is used for all users created and used during the installation.
Enter a password for all logins used during the installation: Way2secure
Enter a password for all logins used during the installation (confirm): Way2secure

Please enter your fully qualified domain name, e.g. ovs123.us.oracle.com, (or IP address) of your management server for SSL certification generation, more than one IP address are detected: 10.1.12.110 192.168.12.110 [ovmmdr1]: ovmmdr1

Verifying configuration …

Start installing Oracle VM Manager:
1: Continue
2: Abort

Select Number (1-2): 1

Step 1 of 9 : Database Software…
Installing Database Software…
Retrieving MySQL Database 5.6 …
Unzipping MySQL RPM File …
Installing MySQL 5.6 RPM package …
Configuring MySQL Database 5.6 …
Installing MySQL backup RPM package …

Step 2 of 9 : Java …
Installing Java …

Step 3 of 9 : Database schema …
Creating database ‘ovs’ …
Creating database ‘appfw’
Creating user ‘ovs’ for database ‘ovs’…
Creating user ‘appfw’ for database ‘appfw’

Step 4 of 9 : WebLogic and ADF…
Retrieving Oracle WebLogic Server 12c and ADF …
Installing Oracle WebLogic Server 12c and ADF …
Applying patches to Weblogic …

Step 5 of 9 : Oracle VM …
Installing Oracle VM Manager Core …
Retrieving Oracle VM Manager Application …
Extracting Oracle VM Manager Application …

Retrieving Oracle VM Manager Upgrade tool …
Extracting Oracle VM Manager Upgrade tool …
Installing Oracle VM Manager Upgrade tool …

Step 6 of 9 : Domain creation …
Creating Oracle WebLogic Server domain …
Starting Oracle WebLogic Server 12c …
Creating Oracle VM Manager user ‘admin’ …

Retrieving Oracle VM Manager CLI tool …
Extracting Oracle VM Manager CLI tool…
Installing Oracle VM Manager CLI tool …

Step 7 of 9 : Deploy …
Configuring Https Identity and Trust…
Deploying Oracle VM Manager Core container …
Configuring Client Cert Login…
Deploying Oracle VM Manager UI Console …
Deploying Oracle VM Manager Help …
Disabling HTTP access …

Step 8 of 9 : Oracle VM Tools …

Retrieving Oracle VM Manager Shell & API …
Extracting Oracle VM Manager Shell & API …
Installing Oracle VM Manager Shell & API …

Retrieving Oracle VM Manager Wsh tool …
Extracting Oracle VM Manager Wsh tool …
Installing Oracle VM Manager Wsh tool …

Retrieving Oracle VM Manager Tools …
Extracting Oracle VM Manager Tools …
Installing Oracle VM Manager Tools …
Copying Oracle VM Manager shell to ‘/usr/bin/ovm_shell.sh’ …
Installing ovm_admin.sh in ‘/u01/app/oracle/ovm-manager-3/bin’ …
Installing ovm_upgrade.sh in ‘/u01/app/oracle/ovm-manager-3/bin’ …

Step 9 of 9 : Start OVM Manager …
Enabling Oracle VM Manager service …
Shutting down Oracle VM Manager instance …
Starting Oracle VM Manager instance …
Waiting for the application to initialize …
Oracle VM Manager is running …

Please wait while WebLogic configures the applications…
Oracle VM Manager installed.

Installation Summary

Database configuration:
Database type : MySQL
Database host name : localhost
Database name : ovs
Database listener port : 49500
Database user : ovs

Weblogic Server configuration:
Administration username : weblogic

Oracle VM Manager configuration:
Username : admin
Core management port : 54321
UUID : 0004fb00000100006231d80f2ca9856b

Passwords:
There are no default passwords for any users. The passwords to use for Oracle VM Manager, Database, and Oracle WebLogic Server have been set by you during this installation. In the case of a default install, all passwords are the same.

Oracle VM Manager UI:
https://ovmmdr1:7002/ovm/console
Log in with the user ‘admin’, and the password you set during the installation.

Note that you must install the latest ovmcore-console package for your Oracle Linux distribution to gain VNC and serial console access to your Virtual Machines (VMs).
Please refer to the documentation for more information about this package.

For more information about Oracle Virtualization, please visit:
http://www.oracle.com/virtualization/

Oracle VM Manager installation complete.

Please remove configuration file /tmp/ovm_configcKjMF_.
[root@ovmmdr1 mnt]#

Capture

  • Install ovmcore-console-1.0-41.el6.noarch.rpm on ovmmdr1

yum install -y /var/tmp/ovmcore-console-1.0-41.el6.noarch.rpm

OVMPRD1
==============================================

  • create a VM based on Oracle Linux 64bit OS
  • rename the VM to ovmprd1
  • Give the VM 2gb memory and 2 cpu’s
  • Give the VM a 6gb hard drive
  • configure ovmprd1 with the following network adapters:
    adapter 1: (Host Only) Prod Management
    adapter 2: (NAT Network) ProdPublic
    adapter 3: (Host Only) Prod Storage
    adapter 4: (Host Only) Prod Storage

  • boot the VM and install OVM Server 3.3.2

  • configure the VM with the following settings:


Host Name: ovmprd1
IP Address (eth0): 10.1.11.101
IP Netmask: 255.255.255.0
Default Router: 10.1.11.1
DNS Server: 192.168.11.110
OVS Agent Password: Way2secure
Root Password: Way2secure

  • Once the VM has booted fully and is at the splash screen you can continue to the next step


* log into PROD OVM Manager
* discover the PROD OVM server
* choose Servers and VM's tab
* click on PROD OVM server
* select the "Bond Ports" perspective
* create a new bond with the following parameters:

Interface Name: bond1
Addressing: static
IP Address: 172.16.11.101
Mask: 255.255.255.0
MTU: 1500
Description: (optional)
Bonding: Load Balanced
Selected Ports: eth2 and eth3

  • choose Networking tab
  • select the Network labeled 10.1.11.0 and configure with the following parameters:

** Configuration Tab **

Name: Management
Description: (optional)
Network Uses: Check Management, Live Migrate and Cluster Heartbeat

** Ports Tab **

Port Name: bond0

** VLAN Interfaces **

None

== NETWORK CONFIGURATION ==

* Create a new network
* select "Create a Network with Ports/Bond Ports/VLAN Interfaces" radio button and click next
* Give it a name of "Storage", select the "Storage" checkbox then click next
* add bond1 from ovmprd1 and click ok
* click next - there will not be any VLAN interfaces so click Finish
* Create a new network
* select "Create a Network with Ports/Bond Ports/VLAN Interfaces" radio button and click next
* Give it a name of "Public", select the "Virtual Machine" checkbox then click next
* add eth1 from ovmprd1 and click ok
* click next - there will not be any VLAN interfaces so click Finish

== STORAGE CONFIGURATION ==

* Click on the "Storage" tab
* Discover SAN Server
* Assign name of "PROD-ZFS"
* Make sure Storage Type says "iSCSI Storage Server"
* Make sure Storage Plug-in says "Oracle Generic SCSI Plugin"
* Click next
* Add an Access Host with IP address of 172.16.11.100
* Click next
* Add ovmprd1 to Selected Servers then click next
* Edit the default access group
* On the storage initiators tab, add ovmprd1's iqn to Selected Storage Initiators then click ok
* Click Finish
* Highlight the PROD-ZFS SAN server and click Refresh SAN Server
* Verify that two physical disks are visible
* Rename the 12gb LUN to PROD-PoolFS
* Rename the 30gb LUN to PROD-REPO

== SERVER POOL CREATION ==

* Click on the Servers and VM's tab
* Create a new Server Pool called PROD
* Give it a VIP of 10.1.11.102
* Select Physical Disk radio button
* Select Storage Location and choose the PROD-PoolFS LUN
* Click next
* move ovmprd1 to Selected servers then click finish

== STORAGE REPOSITORY CREATION ==

* Click on the Repositories tab
* Create a new Repository called PROD-REPO
* Select the Physical Disk radio button under Repository Location
* Click on the magnifying glass and choose PROD-REPO then click next
* move ovmprd1 to Present to Servers then click finish

== STORAGE REPOSITORY REPLICATION ==

* Log into zfsprd1
* Click on Configuration, then services
* Edit the replication service
* Add a target

Name: ovmdr1
Hostname: 172.16.10.101
Root password: Way2secure

  • Click on shares
  • Edit the PROD-REPO Project
  • Click on the Replication sub group
  • Add an action

Target: ovmdr1
Pool: DR
Update Frequency: Scheduled
Add a Schedule for every half hour at 00 minutes after
Leave the rest of the settings at the default

  • Click Add
  • Hover over the Target near the STATUS column and click on the picture of the two circular
    arrows pointing to eachother. This will kick off a manual replication.
  • Monitor replication status until completely replicated (should take about 5 minutes)

==============================================

OVMDR1

  • create a VM based on Oracle Linux 64bit OS
  • rename the VM to ovmdr1
  • Give the VM 2gb memory and 2 cpu’s
  • Give the VM a 6gb hard drive
  • configure ovmdr1 with the following network adapters:
    adapter 1: (Host Only) DR Management
    adapter 2: (NAT Network) DRPublic
    adapter 3: (Host Only) DR Storage
    adapter 4: (Host Only) DR Storage
  • boot the VM and install OVM Server 3.3.2
  • configure the VM with the following settings:

Host Name: ovmdr1
IP Address (eth0): 10.1.12.101
IP Netmask: 255.255.255.0
Default Router: 10.1.12.1
DNS Server: 192.168.12.110
OVS Agent Password: Way2secure
Root Password: Way2secure

  • Once the VM has booted fully and is at the splash screen you can continue to the next step
  • Copy following configuration files from ovmprd1 to ovmdr1. Note that in your installation,
    the bridge name referenced below will be different

/etc/sysconfig/network-scripts/meta-eth1
/etc/sysconfig/network-scripts/ifcfg-{bridge} (example /etc/sysconfig/network-scripts/ifcfg-1080940192)

  • Edit /etc/sysconfig/network-scripts/ifcfg-{bridge} on ovmdr1 to make the
    MAC address match that of eth1 on ovmdr1 but leave the bridge number intact

  • log into DR OVM Manager

  • discover the DR OVM server
  • choose Servers and VM’s tab
  • click on DR OVM server
  • select the “Bond Ports” perspective
  • create a new bond with the following parameters:

Interface Name: bond1
Addressing: static
IP Address: 172.16.12.101
Mask: 255.255.255.0
MTU: 1500
Description: (optional)
Bonding: Load Balanced
Selected Ports: eth2 and eth3

  • choose Networking tab
  • select the Network labeled 10.1.12.0 and configure with the following parameters:

** Configuration Tab **
Name: Management
Description: (optional)
Network Uses: Check Management, Live Migrate and Cluster Heartbeat

** Ports Tab **
Port Name: bond0

** VLAN Interfaces **
None

== NETWORK CONFIGURATION ==
* Create a new network
* select “Create a Network with Ports/Bond Ports/VLAN Interfaces” radio button and click next
* Give it a name of “Storage”, select the “Storage” checkbox then click next
* add bond1 from ovmdr1 and click ok
* click next – there will not be any VLAN interfaces so click Finish
* Create a new network
* Observe that the Public network is already there. This is done by copying the meta file and the
bridge file from OVMPRD1

== STORAGE CONFIGURATION ==
* Click on the “Storage” tab
* Discover SAN Server
* Assign name of “DR-ZFS”
* Make sure Storage Type says “iSCSI Storage Server”
* Make sure Storage Plug-in says “Oracle Generic SCSI Plugin”
* Click next
* Add an Access Host with IP address of 172.16.12.100
* Click next
* Add ovmdr1 to Selected Servers then click next
* Edit the default access group
* On the storage initiators tab, add ovmdr1’s iqn to Selected Storage Initiators then click ok
* Click Finish
* Highlight the DR-ZFS SAN server and click Refresh SAN Server
* Verify that one physical disk is visible
* Rename the 12gb LUN to DR-PoolFS

== SERVER POOL CREATION ==
* Click on the Servers and VM’s tab
* Create a new Server Pool called DR
* Give it a VIP of 10.1.12.102
* Select Physical Disk radio button
* Select Storage Location and choose the DR-PoolFS LUN
* Click next
* move ovmdr1 to Selected servers then click finish

== TEMPLATE IMPORT ON PROD ==
* Download template to /var/tmp on ovmprd1 (should be .ova format to proceed- unzip if needed)
* Start Python web server on ovmprd1

python -m SimpleHTTPServer 8080

  • Navigate to the Repositories tab in ovmmprd1
  • Expand the PROD-REPO repository and highlight the Assemblies folder
  • Click on the import VM Assembly button
    VM Template URLs: http://10.1.11.101:8080/OVM_OL6U6_x86_64_PVM.ova

  • Click on assembly that was just imported

  • Create template from assembly
    Assembly Virtual Machines: {select the assembly you just imported}
    VM Template Name: t_OL6.6

  • Click ok

  • Edit the t_OL6.6 template
    Add public network
    change sizing to 1gb memory and 1vcpu

  • Clone the t_OL6.6 template to a virtual machine
    Clone Name: ol6.6
    Target Server Pool: PROD

  • Edit ol6.6.0 VM
    Change VM name to ol6.6
    If there is an extra VNIC, remove it

  • Start VM and connect to console

  • Configure VM with hostname and root password
  • Touch a file

touch /var/tmp/ovmprd1

 

Continued in part 4 of 5

Oracle VM Disaster Recovery In A Box (Part 1 of 5)

VirtualizationVirtualization in todays IT world is practically a given.  Whether you’re a fledgling developer writing applications to get that first break or a seasoned IT Architect designing a solution for a fortune 500 company, there’s a good chance you use virtualization in some aspect of your day to day life at work.  You may not realize it but you’re probably using virtualization right now reading this article.  The Java Virtual Machine that is used on the server to present some of the components that you see on this page is a form of virtualization, but not the kind we’re going to talk about in this series of posts.

With modern x86 processors approaching 18 cores and more, combined with Intel’s Hyperthreading technology we find ourselves with a TON of computing power in todays servers.  With even a modest low end 2 socket server, you’re looking at up to 72 virtual CPU’s to work with.  Applications and especially operating systems have been designed for years to take advantage of SMP (Symmetric Multi Processing) to spread the work among multiple processors and get more work done quicker.  It didn’t used to be that way however.  Back in the mid 80’s, Rockwell International designed one of the first 8 bit processors with two cores on the same chip.  Later in the 2000’s, Intel and AMD went on to develop multi-core processors as well and for the most part, Intel has stayed on top of the game continuing to push the envelope in the x86 processor market.  And lest we forget that Oracle (formerly Sun Microsystems) released their UltraSPARC T1 in March of 2006 with either 4, 6 or 8 cores.  Additionally, each core had up to 4 threads for a total maximum number of 32 virtual CPU’s on a single socket.  2006 folks- this was a big deal!

With so many processors comes the fact that most of the time, servers sit idle and only utilize the bulk of their processing capability during peaks that typically come once a month or at year end processing times.  Multiple studies conducted by Gartner and other organizations have observed that the average utilization of servers ranges from 6-12%.  This means that 88-94% of the time they are doing little or no work.  Not a very good return on investment is it?  With Virtualization, we can maximize the utilization of the resources that have already been purchased and gain a lot of flexibility at the same time.

Another aspect of datacenter processing is the nature of their criticality.  In the IT world, you have to be up all the time.  There’s no room for unplanned outages due to unforseen circumstances like power outage or natural disaster.  Planning ahead is the key and disasters have to be accounted for.  This brings us to the reason for this article.  Most companies will have a plan for how they will provide services in the event of a complete failure of their primary datacenter.  Typically it will involve placing a set of servers and some storage at an alternate location to take over business functions in the event of a catastrophe.  It isn’t enough to simply have servers and storage, there’s a layer of complexity in today’s business software that needs to be accounted for.  Virtualization definitely falls into this category.  I will be talking about how a disaster recovery solution based on Oracle VM for x86 can be designed, implemented and tested with little or no investment in capital hardware.  What’s more- the very same recipie or “run book” you wind up using to design your simulated data centers can be used to implement the real thing!

Untitled211In the next couple posts, I’ll lay out step by step how to put this together in a very basic form to see how the pieces work together.  If will be a very slimmed down environment based on the work I’ve done as well as some really useful training from Greg King (principal technical consultant with Oracle VM product management) to put this together.  The main reason for that is that my laptop only has 16gb of memory so I was somewhat restricted by that.  You may not be as limited on horsepower so feel free to scale this out to look closer to what you’d implement in reality.  My solution is based on all Oracle technology but yours doesn’t have to be.  I chose Oracle VirtualBox as my virtual sandbox, Oracle Linux for the guest OS and Oracle VM for x86 for the simulated hypervisor.

In Part 2, I’ll start laying out the VM configurations.

Oracle VM Disaster Recovery In A Box (Part 2 of 5)

gold-dollar-signTypically, when setting up a new solution it’s always advantageous to do a POC (Proof Of Concept) in your environment to make sure it works the way you expect it to.  With disaster recovery this could wind up being rather expensive.  I’ll show you how you can use tools that are available today to simulate an entire production and dr site including storage replication for FREE.

In the next few posts, I’ll lay out step by step how to get your OVM DR POC up and running on a machine with relatively low resources.  I run the simulation on my laptop.  Whatever hardware you choose to run this on, it MUST be 64bit capable and support either Intel VT-x or AMD-V instruction sets.  If you don’t have this, you can’t run a 64 bit VM. I found that memory was the number one factor that affected performance for my tests.  Surprisingly (or maybe not) the SSD was a close second.  I started this on a standard spinning disk (7200 rpm SATA) and after the second VM started to boot, the system just tanked.  Once I upgraded to the significantly faster iop packed SSD, things got much better.  Here are my specs:

Lenovo G780

Windows 8 64bit
Core i5-3210m
16gb memory
500gb Samsung 840 EVO SSD

Even though this series of articles is focusing on Oracle VM, you could do something very similar with other hypervisors such as VMware.  Keep in mind- we’re focusing on the concept and the step by step process of switching over to a DR site.  Don’t think you’re gonna get screaming performance out of the VM’s you stand up in your virtualized OVM servers, but it will be functional and that’s all we care about.

Without any more delay- on to the demo!

The first thing you need to do is make sure you have enough space on your SSD.  I would conservatively set aside at least 100gb (and that’s thin provisioned).  You’ll also need to download the software we’ll be using.  You’ll need an MOS (My Online Support) account to download these- it’s free to sign up if you don’t already have one:

Oracle Linux (either version 6 or 7 doesn’t matter)

Oracle VM VirtualBox (grab the latest version)

Oracle VM for x86 (grab the latest version)

Oracle ZS3-2 Simulator (grab latest version)

Oracle Linux 6 Update 6 Template for OVM*

  • log into edelivery.oracle.com/linux
  • de-select “Programs”
  • select “Linux/OVM/VMs”
  • type linux in the search box
  • select “Oracle VM Templates for Oracle Linux
  • select platform – x86 64bit
  • click continue
  • Select “Oracle VM Templates for Oracle Linux 6.0.0.0.0 for x86 64 bit 5 files”
  • click continue and accept T&C’s then continue
  • download the template “V73659-01.zip Oracle Linux 6 Update 6 template (OVF) – Paravirtualized x86_64 (64 bit)”

Installation and configuration

* download and install virtualbox
* download zfs appliance simulator
* create host only networks in virtualbox and give your machine the .1 address on each network.  You will not be able to give these networks a name in VirtualBox:

  • PROD Storage Network (172.16.11.1/24)
  • PROD Management Network (10.1.11.1/24)
  • PROD Public Network (192.168.11.1/24)
  • DR Storage Network (172.16.12.1/24)
  • DR Management Network (10.1.12.1/24)
  • DR Public Network (192.168.12.1/24)
  • Replication Network (172.16.10.1/24)
* create NAT networks in VirtualBox and check the “supports DHCP” box for both of them.  You will be able to give these networks a name in VirtualBox:
  • ProdPublic (192.168.11.0/24)
  • DRPublic (192.168.12.0/24)
NOTE: If you’re running virtualbox in Windows, I suggest going into the network control panel applet and renaming the virtualbox Host-Only Ethernet Adapter to reflect it’s function.  You can’t give host only network adapters a specific name in VirtualBox like you can NAT networks.  If you’re running VirtualBox in Linux, then just rename each virtual interface in the ifcfg-xxxx script:
 Capture2 (1)
* create the following VM’s:
  • zfsprd1 – zfs appliance simulator for production
  • zfsdr1 – zfs appliance simulator for dr
  • ovmmprd1 – OVM Manager for production
  • ovmmdr1 – OVM Manager for dr
  • ovmprd1 – OVM Server for production
  • ovmdr1 – OVM Server for dr
NOTE: Don’t get lazy and clone the production ZFS simulator to make the dr one.  When you go to set up replication between the two you’ll get an EAK_PEER_AUTH error.  This is because the UUID is the same on both appliances and it uses that to generate the key for replication.  Most likely you’ll be scratching your head until you read this and realize what you did (like me).  Do a fresh import/install of the appliance from the zip file you downloaded instead for all the VM’s to be safe.
Now we can move on to configuring each VM in VirtualBox and install the software.
==============================================
ZFSPRD1
==============================================
* import the zfs appliance OVA into VirtualBox
* configure the production zfs appliance VM with the following network adapters:
  • adapter 1: (Host Only) Prod Management
  • adapter 2: (Host Only) Replication Network
  • adapter 3: (Host Only) Prod Storage
  • adapter 4: (Host Only) Prod Storage
* rename the VM to zfsprd1 and boot it
* start configuration and enter the following information:
Host Name: zfsprd1
DNS Domain: initech.local
IP Address: 10.1.11.100
IP Netmask: 255.255.255.0
Default Router: 10.1.11.1
DNS Server: 0.0.0.0
Password: Way2secure
Re-enter Password: Way2secure
 
NOTE: you will only need to configure the management interface at this point.  The rest of the configuration will be done in the BUI.  Tab out of the password re-enter field and hit ESC-1 to continue the boot process.  Once you see the login prompt, move on to the next step.
* log into the zfsprd1 web interface
* click on Start to begin configuration
* configure networking as follows:
 
Datalinks:
(DL) Management via e1000g0
(DL) Replication via e1000g1
(DL) Storage via e1000g2
Interfaces:
(INT) Management (10.1.11.100/24) via e1000g0
(INT) Replication (172.16.10.100/24)  via e1000g1
(INT) Storage (172.16.11.100/24) via e1000g2
* Click apply
* Click commit
* DNS Domain set to initech.local
* DNS Server(s) set to 0.0.0.0
* Click commit
* Leave all default settings at configure time, click sync then click commit
* Leave all default settings at configure name services then click commit
* Create new storage pool called PROD, click ok
Leave all default settings at Verify and allocate devices then click commit
* Choose striped data profile then click commit
* Click commit at configure storage
* Click on Later for Registration and Support
* Click on Configuration then SAN then iSCSI
* Add a new Target called PROD-ZFS bound to e1000g2
* Create a new target group called ZFS
* Add PROD-ZFS Target to ZFS target group then click apply
* Navigate to Shares -> Projects
* Create 2 new projects called “PROD-POOL” and “PROD-REPO”
* Click on LUNs
* Click the + sign to Create 2 new LUNs as follows:
Project: PROD-POOL
Name: PROD-PoolFS
Volume Size: 12G
Thin Provisioned: (checked)
Target Group: ZFS
All other options keep defaults
Project: PROD-REPO
Name: PROD-Repo1
Volume Size: 30G
Thin Provisioned: (checked)
Target Group: ZFS
All other options keep defaults
==============================================
ZFSDR1
==============================================
* import the zfs appliance OVA into VirtualBox
* configure the dr zfs appliance VM with the following network adapters:
  • adapter 1: (Host Only) DR Management
  • adapter 2: (Host Only) Replication Network
  • adapter 3: (Host Only) DR Storage
  • adapter 4: (Host Only) DR Storage
* rename the VM to zfsdr1 and boot it
* start configuration and enter the following information:
Host Name: zfsdr1
DNS Domain: initech.local
IP Address: 10.1.12.100
IP Netmask: 255.255.255.0
Default Router: 10.1.12.1
DNS Server: 0.0.0.0
Password: Way2secure
Re-enter Password: Way2secure
 
NOTE: you will only need to configure the management interface at this point.  The rest of the configuration will be done in the BUI.  Tab out of the password re-enter field and hit ESC-1 to continue the boot process.  Once you see the login prompt, move on to the next step.
* log into the zfsdr1 web interface
* click on Start to begin configuration
* configure networking as follows:
 
Datalinks:
(DL) Management via e1000g0
(DL) Replication via e1000g1
(DL) Storage via e1000g2
Interfaces:
(INT) Management (10.1.12.100/24) via e1000g0
(INT) Replication (172.16.10.101/24)  via e1000g1
(INT) Storage (172.16.12.100/24) via e1000g2
* Click apply
* Click commit
* DNS Domain set to initech.local
* DNS Server(s) set to 0.0.0.0
* Click commit
* Leave all default settings at configure time, click sync then click commit
* Leave all default settings at configure name services then click commit
* Create new storage pool called DR, click ok
Leave all default settings at Verify and allocate devices then click commit
* Choose striped data profile then click commit
* Click commit at configure storage
* Click on Later for Registration and Support
* Click on Configuration then SAN then iSCSI
* Add a new Target called DR-ZFS bound to e1000g2
* Create a new target group called ZFS
* Add DR-ZFS Target to ZFS target group then click apply
* Navigate to Shares -> Projects
* Click the + sign to create a new project called “DR-POOL”
* Create a new LUN as follows:
Project: DR-POOL
Name: DR-PoolFS
Volume Size: 12G
Thin Provisioned: (checked)
Target Group: ZFS
All other options keep defaults
==============================================
OVMMPRD1
==============================================
* create a VM based on Oracle Linux 64bit OS
* rename the VM to ovmmprd1
* Give the VM 4gb memory and 2 cpu’s
* Give the VM a 30gb hard drive
* configure ovmmprd1 with the following network adapters:
  • adapter 1: (Host Only) Prod Management
  • adapter 2: (NAT Network) ProdPublic
* boot the VM and install Oracle Linux 6.5 and select the Desktop server type.  We do this so you have a GUI to log into- if that’s not a priority for you personally, then just pick Basic server
* configure the VM with the following information:
Host Name: ovmmprd1
IP Address (eth0): 10.1.11.110
IP Netmask: 255.255.255.0
IP Address (eth1): 192.168.11.110
IP Netmask: 255.255.255.0
Default Router: 192.168.11.1
DNS Server: 8.8.8.8
Root Password: Way2secure
* turn off iptables and selinux:
[root@ovmmprd1 ~]# service iptables stop ; chkconfig iptables off
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
[root@ovmmprd1 ~]#

[root@ovmmprd1 ~]# setenforce 0
setenforce: SELinux is disabled
[root@ovmmprd1 ~]#

[root@ovmmprd1 ~]# vi /etc/selinux/config
NOTE: Set SELINUX=disabled in file below:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

* add following line to /etc/hosts

10.1.11.110     ovmmprd1

* set parameters in /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=ovmmprd1
GATEWAY=192.168.11.1
  
* reboot to complete the selinux disabling
* attach the OVM Manager 3.3.2 install ISO to the VM
* run the createOracle.sh script to prep the VM for the installation of OVM Manager
[root@ovmmprd1 mnt]# ./createOracle.sh
Adding group 'oinstall' with gid '54323' ...
groupadd: group 'oinstall' already exists
Adding group 'dba'
groupadd: group 'dba' already exists
Adding user 'oracle' with user id '54322', initial login group 'dba', supplementary group 'oinstall' and home directory '/home/oracle' ...
User 'oracle' already exists ...
uid=54321(oracle) gid=54322(dba) groups=54322(dba),54321(oinstall)
Creating user 'oracle' succeeded ...
For security reasons, no default password was set for user 'oracle'. If you wish to login as the 'oracle' user, you will need to set a password for this account.
Verifying user 'oracle' OS prerequisites for Oracle VM Manager ...
oracle soft nofile 8192
oracle hard nofile 65536
oracle soft nproc 2048
oracle hard nproc 16384
oracle soft stack 10240
oracle hard stack 32768
oracle soft core unlimited
oracle hard core unlimited
Setting user 'oracle' OS limits for Oracle VM Manager ...
Altered file /etc/security/limits.conf
Original file backed up at /etc/security/limits.conf.orabackup
Verifying & setting of user limits succeeded ...
Changing '/u01' permission to 755 ...
Changing '/u01/app' permission to 755 ...
Changing '/u01/app/oracle' permission to 755 ...
Modifying iptables for OVM
Adding rules to enable access to:
 7002 : Oracle VM Manager https
 54322 : Oracle VM Manager core via SSL
 123 : NTP
 10000 : Oracle VM Manager CLI Tool
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
iptables: Applying firewall rules: [ OK ]
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
iptables: Applying firewall rules: [ OK ]
Rules added.
[root@ovmmprd1 mnt]#

* run the runInstaller.sh script to install OVM Manager

[root@ovmmprd1 mnt]# ./runInstaller.sh
 
Oracle VM Manager Release 3.3.2 Installer
 
Oracle VM Manager Installer log file:
/var/log/ovmm/ovm-manager-3-install-2015-03-04-170449.log
 
Please select an installation type:
   1: Install
   2: Upgrade
   3: Uninstall
   4: Help
 
   Select Number (1-4): 1
 
Starting production with local database installation ...
 
Verifying installation prerequisites ...
*** WARNING: Recommended memory for the Oracle VM Manager server installation using Local MySql DB is 7680 MB RAM
 
One password is used for all users created and used during the installation.
Enter a password for all logins used during the installation: Way2secure
Enter a password for all logins used during the installation (confirm): Way2secure
 
Please enter your fully qualified domain name, e.g. ovs123.us.oracle.com, (or IP address) of your management server for SSL certification generation, more than one IP address are detected: 10.1.11.110 192.168.11.110 [ovmmprd1]: ovmmprd1
 
Verifying configuration ...
 
Start installing Oracle VM Manager:
   1: Continue
   2: Abort
 
   Select Number (1-2): 1
 
Step 1 of 9 : Database Software...
Installing Database Software...
Retrieving MySQL Database 5.6 ...
Unzipping MySQL RPM File ...
Installing MySQL 5.6 RPM package ...
Configuring MySQL Database 5.6 ...
Installing MySQL backup RPM package ...
 
Step 2 of 9 : Java ...
Installing Java ...
 
Step 3 of 9 : Database schema ...
Creating database 'ovs' ...
Creating database 'appfw'
Creating user 'ovs' for database 'ovs'...
Creating user 'appfw' for database 'appfw'
 
Step 4 of 9 : WebLogic and ADF...
Retrieving Oracle WebLogic Server 12c and ADF ...
Installing Oracle WebLogic Server 12c and ADF ...
Applying patches to Weblogic ...
 
Step 5 of 9 : Oracle VM  ...
Installing Oracle VM Manager Core ...
Retrieving Oracle VM Manager Application ...
Extracting Oracle VM Manager Application ...
 
Retrieving Oracle VM Manager Upgrade tool ...
Extracting Oracle VM Manager Upgrade tool ...
Installing Oracle VM Manager Upgrade tool ...
 
Step 6 of 9 : Domain creation ...
Creating Oracle WebLogic Server domain ...
Starting Oracle WebLogic Server 12c ...
Creating Oracle VM Manager user 'admin' ...
 
Retrieving Oracle VM Manager CLI tool ...
Extracting Oracle VM Manager CLI tool...
Installing Oracle VM Manager CLI tool ...
 
Step 7 of 9 : Deploy ...
Configuring Https Identity and Trust...
Deploying Oracle VM Manager Core container ...
Configuring Client Cert Login...
Deploying Oracle VM Manager UI Console ...
Deploying Oracle VM Manager Help ...
Disabling HTTP access ...
 
Step 8 of 9 : Oracle VM Tools ...
 
Retrieving Oracle VM Manager Shell & API ...
Extracting Oracle VM Manager Shell & API ...
Installing Oracle VM Manager Shell & API ...
 
Retrieving Oracle VM Manager Wsh tool ...
Extracting Oracle VM Manager Wsh tool ...
Installing Oracle VM Manager Wsh tool ...
 
Retrieving Oracle VM Manager Tools ...
Extracting Oracle VM Manager Tools ...
Installing Oracle VM Manager Tools ...
Copying Oracle VM Manager shell to '/usr/bin/ovm_shell.sh' ...
Installing ovm_admin.sh in '/u01/app/oracle/ovm-manager-3/bin' ...
Installing ovm_upgrade.sh in '/u01/app/oracle/ovm-manager-3/bin' ...
 
Step 9 of 9 : Start OVM Manager ...
Enabling Oracle VM Manager service ...
Shutting down Oracle VM Manager instance ...
Starting Oracle VM Manager instance ...
Waiting for the application to initialize ...
Oracle VM Manager is running ...
 
Please wait while WebLogic configures the applications...
Oracle VM Manager installed.
 
Installation Summary
--------------------
Database configuration:
  Database type               : MySQL
  Database host name          : localhost
  Database name               : ovs
  Database listener port      : 49500
  Database user               : ovs
 
Weblogic Server configuration:
  Administration username     : weblogic
 
Oracle VM Manager configuration:
  Username                    : admin
  Core management port        : 54321
  UUID                        : 0004fb00000100006231d80f2ca9856b
 
 
Passwords:
There are no default passwords for any users. The passwords to use for Oracle VM Manager, Database, and Oracle WebLogic Server have been set by you during this installation. In the case of a default install, all passwords are the same.
 
Oracle VM Manager UI:
  https://ovmmprd1:7002/ovm/console
Log in with the user 'admin', and the password you set during the installation.
 
Note that you must install the latest ovmcore-console package for your Oracle Linux distribution to gain VNC and serial console access to your Virtual Machines (VMs).
Please refer to the documentation for more information about this package.
 
For more information about Oracle Virtualization, please visit:
  http://www.oracle.com/virtualization/
 
Oracle VM Manager installation complete.
 
Please remove configuration file /tmp/ovm_configcKjMF_.
[root@ovmmprd1 mnt]#
 
 
* Log into the OVM Manager web interface at https://10.1.11.110:7002/ovm/console and make sure you see a screen similar to the one below:
Capture


* Install ovmcore-console-1.0-41.el6.noarch.rpm on ovmmprd1
# yum install -y /var/tmp/ovmcore-console-1.0-41.el6.noarch.rpm
 
The rest of the configurations are in Part 3.  Stay Tuned!

Official Oracle VM for x86 DR solution coming soon!

Greg King (@poorfrodo) with Oracle has posted a blog entry outlining Oracle’s upcoming DR solution for making Oracle VM for x86 capable of DR at multiple sites.  Check out his blog post at https://goo.gl/EcHZnW and also give a look at the MOS infodoc referenced inside that he wrote which has lots of good information in it as well!