This tutorial will show how to migrate a VM that’s running on Proxmox 8.1.4 into Oracle Linux Virtualization Manager (OLVM) 4.5. Before we get into the good stuff, I want to clarify some specifics about Proxmox (sometimes referred to as PVE or Proxmox Virtual Environment) and OLVM. Let’s start with Proxmox since that’s where we’re exporting from.
Background
Proxmox Virtual Environment (PVE) is an open-source server virtualization management platform that combines two powerful virtualization technologies: Kernel-based Virtual Machine (KVM) and Linux Containers (LXC). As of version 8, Proxmox VE continues to provide a robust and flexible solution for deploying and managing virtualized infrastructures. KVM serves as the hypervisor, enabling full virtualization and allowing users to run multiple isolated operating systems simultaneously on x86 hardware. LXC, on the other hand, offers lightweight containerization, enabling efficient and high-performance management of applications within isolated environments. Proxmox VE also integrates QEMU as the hardware emulator, which facilitates the emulation of a complete hardware stack, supporting a wide range of guest operating systems and configurations.
Proxmox VE’s history dates back to its initial release in 2008, developed by Proxmox Server Solutions GmbH. Since its inception, it has evolved to include high-availability clustering, live migration, integrated backup solutions, and web-based management interfaces. The platform has gained a strong reputation within the IT community for its ease of use, scalability, and active development.
Oracle Linux Virtualization Manager (OLVM) is a virtualization management platform designed for managing virtualized environments. As of its latest version, OLVM leverages the KVM hypervisor and QEMU just like Proxmox does. OLVM 4.5 is based on version 4.5.4 of the oVirt project.
If you didn’t already know this, Oracle’s cloud platform (OCI) uses KVM/QEMU to run their VMs. OLVM supports features such as live migration, snapshotting, and disaster recovery. Since OCI uses the same technology, you can easily export VMs from OLVM to OCI. I may be writing an article on this process soon!
Migration
Now that you have a basic understanding of the two platforms, let’s talk about the process at a high level. Essentially what you’re going to do is shut down the VM running in Proxmox, convert the disk image to qcow2 format, upload it into OLVM, create a VM and attach the disk image, and boot it. That’s a very abbreviated version of what I’ll be walking through, but that’s essentially it! The tough part for me was figuring out how to upload a disk image into OLVM without having to make a copy on your workstation and uploading through the web interface. Since you really shouldn’t install a browser on a KVM server because of the implications of needing to install the entire desktop GUI as well as a browser and all the dependencies, we’ll do it via API instead.
The oVirt 4 SDK is pivotal in this process. There are some good blog posts from Simon Coter (Director, Oracle Linux and Virtualization Product Management) on how to use parts of the SDK to migrate VMs from older Oracle VM for x86, but nothing on migrating from Proxmox that I can find. I stumbled upon a python script written by Gurjeet Kaur where she shows an example of how to utilize the SDK to upload disk images directly into OLVM manager. I took creative license with the script and put some wrappers around it to make it a little more flexible and utilitarian for bulk migrations.
Scripts
This is the disk_upload.py script that I put some guardrails around the core code to make it more portable. Gurjeet Kaur deserves all credit for the hard stuff!
#!/usr/bin/env python3
##
## Based on work by Gurjeet Kaur
##
##
from __future__ import print_function
from getpass import getpass
from six.moves.http_client import HTTPSConnection
from six.moves.urllib.parse import urlparse
import getopt
import sys
import os
import json
import logging
import ovirtsdk4 as sdk
import ovirtsdk4.types as types
import ssl
import subprocess
import time
def main():
print(f"\n\n")
variables = {}
with open("/usr/local/etc/olvm_vars") as f:
for line in f:
if "=" in line:
key, value = line.strip().split("=", 1)
variables[key.strip()] = value.strip().strip('"')
olvm_fqdn = variables.get("olvm_fqdn")
olvm_user = variables.get("olvm_user")
olvm_pass = variables.get("olvm_pass")
cert_path = variables.get("cert_path")
xfer_mode = variables.get("xfer_mode")
image_path = None
disk_name = None
disk_desc = None
stor_dom = None
num_args = len(sys.argv)
num_args -= 1
if num_args != 8:
print("Error. Not enough arguments. See usage statement below:")
usage()
try:
opts, args = getopt.getopt(sys.argv[1:], "hm:u:p:c:x:f:a:d:s:", ["help", "olvm_fqdn=", "olvm_user=", "olvm_pass=", "cert_path=", "xfer_mode=", "image_path=", "disk_name=", "disk_desc=", "stor_dom="])
except getopt.GetoptError as err:
print(str(err))
usage()
for opt, arg in opts:
if opt in ("-h", "--help"):
usage()
elif opt in ("-m", "--olvm_fqdn"):
olvm_fqdn = arg
elif opt in ("-u", "--olvm_user"):
olvm_user = arg
elif opt in ("-p", "--olvm_pass"):
olvm_pass = arg
elif opt in ("-c", "--cert_path"):
cert_path = arg
elif opt in ("-x", "--xfer_mode"):
xfer_mode = arg
elif opt in ("-f", "--image_path"):
image_path = arg
elif opt in ("-a", "--disk_name"):
disk_name = arg
elif opt in ("-d", "--disk_desc"):
disk_desc = arg
elif opt in ("-s", "--stor_dom"):
stor_dom = arg
else:
print(f"\nInvalid option")
usage()
if "https://" not in olvm_fqdn.lower():
print(f"\nolvm_fqdn must contain https://")
usage()
elif ".pem" not in cert_path.lower():
print(f"\ncert_path must contain .pem")
usage()
elif xfer_mode.lower() != "direct" and xfer_mode.lower() != "proxy":
print(f"\nxfer_mode must be either direct or proxy")
usage()
elif ".img" not in image_path.lower():
print(f"\nimage_path must have .img")
usage()
else:
pass
# This seems to give the best throughput when uploading from my laptop
# SSD to a server that drop the data. You may need to tune this on your
# setup.
BUF_SIZE = 128 * 1024
logging.basicConfig(level=logging.DEBUG, filename='example.log')
direct_upload = False
if xfer_mode.lower() == "direct":
direct_upload = True
image_size = os.path.getsize(image_path)
# Get image info using qemu-img
print("Checking image...")
out = subprocess.check_output(
["qemu-img", "info", "--output", "json", image_path])
image_info = json.loads(out)
if image_info["format"] not in ("qcow2", "raw"):
raise RuntimeError("Unsupported image format %(format)s" % image_info)
print("Disk format: %s" % image_info["format"])
# Detect disk content type
#
# ISO format structure
# ---------------------------------------------------------------------------
# offset type value comment
# ---------------------------------------------------------------------------
# 0x0000 system area (e.g. DOS/MBR boot sector)
# 0x8000 int8 0x01 primary volume descriptor type code
# 0x8001 strA "CD001" primary volume descriptor indentifier
# 0x8006 int8 0x01 primary volume desctptor version
# 0x8007 0x00 unused field
#
# See https://wiki.osdev.org/ISO_9660#Overview_and_caveats for more info.
content_type = types.DiskContentType.DATA
if image_info["format"] == "raw":
with open(image_path, "rb") as f:
f.seek(0x8000)
primary_volume_descriptor = f.read(8)
if primary_volume_descriptor == b"\x01CD001\x01\x00":
content_type = types.DiskContentType.ISO
print("Disk content type: %s" % content_type)
# This example will connect to the server and create a new `floating`
# disk, one that isn't attached to any virtual machine.
# Then using transfer service it will transfer disk data from local
# qcow2 disk to the newly created disk in server.
# Create the connection to the server:
print("Connecting...")
# Create the connection to the server:
connection = sdk.Connection(
url=olvm_fqdn,
username=olvm_user,
password=olvm_pass,
ca_file=cert_path,
debug=True,
log=logging.getLogger(),
)
# Get the reference to the root service:
system_service = connection.system_service()
# Add the disk. Note the following:
#
# 1. The size of the disk is specified in bytes, so to create a disk
# of 10 GiB the value should be 10 * 2^30.
#
# 2. The disk size is indicated using the 'provisioned_size' attribute,
# but due to current limitations in the engine, the 'initial_size'
# attribute also needs to be explicitly provided for _copy on write_
# disks created on block storage domains, so that all the required
# space is allocated upfront, otherwise the upload will eventually
# fail.
#
# 3. The disk initial size must be bigger or the same as the size of the data
# you will upload.
print(f"Creating disk [{disk_name}]")
if image_info["format"] == "qcow2":
disk_format = types.DiskFormat.COW
else:
disk_format = types.DiskFormat.RAW
sd_name = stor_dom
disks_service = connection.system_service().disks_service()
disk = disks_service.add(
disk=types.Disk(
name=disk_name,
content_type=content_type,
description=disk_desc,
format=disk_format,
initial_size=image_size,
provisioned_size=image_info["virtual-size"],
sparse=disk_format == types.DiskFormat.COW,
storage_domains=[
types.StorageDomain(
name=sd_name
)
]
)
)
# Wait till the disk is up, as the transfer can't start if the
# disk is locked:
disk_service = disks_service.disk_service(disk.id)
while True:
time.sleep(5)
disk = disk_service.get()
if disk.status == types.DiskStatus.OK:
break
print("Creating transfer session...")
# Get a reference to the service that manages the image
# transfer that was added in the previous step:
transfers_service = system_service.image_transfers_service()
# Add a new image transfer:
transfer = transfers_service.add(
types.ImageTransfer(
image=types.Image(
id=disk.id
)
)
)
# Get reference to the created transfer service:
transfer_service = transfers_service.image_transfer_service(transfer.id)
# After adding a new transfer for the disk, the transfer's status will be INITIALIZING.
# Wait until the init phase is over. The actual transfer can start when its status is "Transferring".
while transfer.phase == types.ImageTransferPhase.INITIALIZING:
time.sleep(1)
transfer = transfer_service.get()
print(f"Uploading [{disk_name}]")
# At this stage, the SDK granted the permission to start transferring the disk, and the
# user should choose its preferred tool for doing it - regardless of the SDK.
# In this example, we will use Python's httplib.HTTPSConnection for transferring the data.
if direct_upload:
if transfer.transfer_url is not None:
destination_url = urlparse(transfer.transfer_url)
else:
print("Direct upload to host not supported (requires ovirt-engine 4.2 or above).")
sys.exit(1)
else:
destination_url = urlparse(transfer.proxy_url)
context = ssl.create_default_context()
# Note that ovirt-imageio-proxy by default checks the certificates, so if you don't have
# your CA certificate of the engine in the system, you need to pass it to HTTPSConnection.
context.load_verify_locations(cafile='/root/ca.pem')
proxy_connection = HTTPSConnection(
destination_url.hostname,
destination_url.port,
context=context,
)
# Send the request head. Note the following:
#
# - For ovirt-engine < 4.2, we must send the 'Authorization' header with
# the signed ticket received from the transfer service.
# I.e. proxy_connection.putheader('Authorization', transfer.signed_ticket)
#
# - For ovirt-engine < 4.2, the server requires 'Content-Range' header
# even when sending the entire file.
# I.e. proxy_connection.putheader('Content-Range',
# "bytes %d-%d/%d" % (0, image_size - 1, image_size))
#
# - the server requires also Content-Length.
#
proxy_connection.putrequest("PUT", destination_url.path)
proxy_connection.putheader('Content-Length', "%d" % (image_size,))
proxy_connection.endheaders()
# Send the request body.
# Note that we must send the number of bytes we promised in the
# Content-Range header.
start = last_progress = time.time()
with open(image_path, "rb") as disk:
pos = 0
while pos < image_size:
# Send the next chunk to the proxy.
to_read = min(image_size - pos, BUF_SIZE)
chunk = disk.read(to_read)
if not chunk:
transfer_service.pause()
raise RuntimeError("Unexpected end of file at pos=%d" % pos)
proxy_connection.send(chunk)
pos += len(chunk)
now = time.time()
# Report progress every 10 seconds.
if now - last_progress > 10:
print("Uploaded %.2f%%" % (float(pos) / image_size * 100))
last_progress = now
# Get the response
response = proxy_connection.getresponse()
if response.status != 200:
transfer_service.pause()
print("Upload failed: %s %s" % (response.status, response.reason))
sys.exit(1)
elapsed = time.time() - start
print(f"Uploaded disk [{disk_name}] of size %.2fg in %.2f seconds (%.2fm/s)" % (
image_size / float(1024**3), elapsed, image_size / 1024**2 / elapsed))
print("Finalizing transfer session...")
# Successful cleanup
transfer_service.finalize()
connection.close()
proxy_connection.close()
print("Upload completed successfully")
def usage():
print(f"\n\nUsage: {sys.argv[0]}")
print("-m <OLVM Manager FQDN> Example: https://ovm-manager.example.com/ovirt-engine/api")
print("-u <OLVM Manager user> Example: admin@ovirt@internalsso")
print("-p <OLVM Manager credentials>")
print("-c <ca.cert location> Example: /root/ca.pem")
print("-x <xferMode>Either Direct or Proxy")
print("-f <image filename> Example: /mnt/VirtualDisks/abcdef.img")
print("-a <image name> Name of the disk to be in OLVM Manager in the Alias column")
print("-d <image description>")
print("-s <storage domain> Example: STOR_PROD_1")
print("")
print("for any arguments you wish to be blank, use this format: -z \"\"")
print("for any arguments you wish to have spaces, use this format: -z \"first time\"")
sys.exit(1)
if __name__ == "__main__":
main()The wrapper script that I use to call the upload script (this can be named whatever you like, I went with xfer_disks.sh):
/usr/local/bin/upload_disk.py -f {disk image} -a "{VM Name}" -d "{Description}" -s "{Storage Domain}"Finally, the olvm_vars file that is referenced when the script runs. It keeps you from having to specify things that will pretty much always be the same. Make sure root owns this script and do a chmod 400 against the file- the OLVM Manager admin password is sitting here in plain text. I’m working on a better way to store the credentials but for now this will have to suffice:
# olvm_vars
olvm_fqdn = https://{your OLVM Manager FQDN}/ovirt-engine/api
olvm_user = {your username}
olvm_pass = {your user's password}
cert_path = {ca.pem path on local server}
xfer_mode = proxyA couple notes about olvm_vars:
- for the olvm_fqdn variable, use the FQDN of the OLVM Manager that you log into. This must match the PEM certificate you copied from /etc/pki/ovirt-engine/ca.pem or it won’t work.
- Whatever account you use, you may need to append @internalsso to the end of it. Example: harvey@ovirt is your normal login, you would put harvey@ovirt@internalsso. The default admin account would look like admin@ovirt@internalsso. If you’ve linked your credentials from AD or some external naming provider, you’ll have to figure that part out on your own.
- cert_path should contain the local path to the ca.pem certificate file that was copied from OLVM Manager.
Process
Here’s the step by step. Some assumptions need to be clarified before we start:
Assumptions
- Proxmox (PVE) 8.1.4
- OLVM 4.5
- upload_disk.py -> /usr/local/bin/ (root:root 775 perms)
- xfer_disk.sh -> /usr/local/bin (root:root 775 perms)
- olvm_vars -> /usr/local/etc (root:root 400 perms)
- Power off source VM
- Log into Proxmox server and become root
- list the local filesystem where your VM lives. look for your vm’s Volid in the output:
# pvesm list local-zfs
Volid Format Type Size VMID
local-zfs:base-100-disk-0 raw images 1048576 100
local-zfs:base-100-disk-0/vm-114-disk-0 raw images 1048576 114
local-zfs:base-100-disk-1 raw images 34359738368 100
local-zfs:base-100-disk-1/vm-114-disk-1 raw images 34359738368 114
local-zfs:base-102-disk-0 raw images 107374182400 102
local-zfs:base-102-disk-0/vm-103-disk-0 raw images 107374182400 103
local-zfs:base-102-disk-0/vm-112-disk-0 raw images 107374182400 112
local-zfs:base-102-disk-0/vm-115-disk-0 raw images 107374182400 115
local-zfs:base-102-disk-0/vm-117-disk-0 raw images 107374182400 117
local-zfs:base-102-disk-0/vm-118-disk-0 raw images 107374182400 118
local-zfs:subvol-104-disk-0 subvol rootdir 32212254720 104
local-zfs:subvol-105-disk-0 subvol rootdir 32212254720 105
- Find the path to your VMs disk. look for the zfs disk path and note it:
# pvesm path local-zfs:base-102-disk-0/vm-117-disk-0
/dev/zvol/rpool/data/vm-117-disk-0- Convert the disk image to qcow2 format. Make sure wherever you choose for the target location has enough space to hold the entire size of the disk image:
# qemu-img convert -O qcow2 /dev/zvol/rpool/data/vm-117-disk-0 /var/tmp/docker.imgNOTE: make sure the filename ends in .img. The script will reject the file otherwise.
- Copy the .img file to your KVM server
- Upload the disk image using the scripts referenced earlier. Make sure to substitute the location of where your .img file was copied to in the script. Call the file transfer from xfer_disks.sh:
# ./xfer_disks.sh
Checking image...
Disk format: qcow2
Disk content type: data
Connecting...
Creating disk [Docker]
Creating transfer session...
Uploading [Docker]
Uploaded 1.39%
Uploaded 2.81%
Uploaded 4.16%
Uploaded 5.61%
Uploaded 6.91%
Uploaded 8.19%
Uploaded 9.58%
Uploaded 11.01%
Uploaded 12.39%
Uploaded 13.87%
Uploaded 15.32%
Uploaded 16.68%
Uploaded 18.16%
Uploaded 19.60%
Uploaded 20.99%
Uploaded 22.44%
Uploaded 23.84%
Uploaded 25.38%
Uploaded 26.92%
Uploaded 28.46%
Uploaded 30.16%
Uploaded 31.85%
Uploaded 33.48%
Uploaded 35.06%
Uploaded 36.64%
Uploaded 38.18%
Uploaded 39.66%
Uploaded 41.17%
Uploaded 42.67%
Uploaded 44.15%
Uploaded 45.66%
Uploaded 47.24%
Uploaded 48.79%
Uploaded 50.28%
Uploaded 51.69%
Uploaded 53.06%
Uploaded 54.45%
Uploaded 55.85%
Uploaded 57.21%
Uploaded 58.69%
Uploaded 60.12%
Uploaded 61.50%
Uploaded 62.89%
Uploaded 64.28%
Uploaded 65.62%
Uploaded 66.98%
Uploaded 68.30%
Uploaded 69.78%
Uploaded 71.25%
Uploaded 72.77%
Uploaded 74.33%
Uploaded 75.87%
Uploaded 77.36%
Uploaded 78.85%
Uploaded 80.36%
Uploaded 81.89%
Uploaded 83.46%
Uploaded 85.01%
Uploaded 86.46%
Uploaded 88.00%
Uploaded 89.56%
Uploaded 91.03%
Uploaded 92.54%
Uploaded 94.07%
Uploaded 95.58%
Uploaded 97.14%
Uploaded 98.70%
Uploaded disk [Docker] of size 20.27g in 680.41 seconds (30.50m/s)
Finalizing transfer session...
Upload completed successfully
#- Create a new VM definition in OLVM making sure to at least set the following:
- Operating System – {what OS is the VM based on}
- Chipset/Firmware type – {if it was SeaBIOS in Proxmox, I use i440FX Chipset with BIOS)
- Name
- Attach your OS disk (don’t forget to check the bootable checkbox)
- Attach your network
DONE!
If your VM has multiple disks, just repeat the process for each disk. When you attach them to the VM, only the disk with the bootable partition should be set to boot in OLVM. At this point, you should be able to start your VM and jump on the console to make sure everything is working. Troubleshooting the boot process is outside the scope of a document like this, but if you do have problems leave a comment and I’ll do my best to give you pointers!
Cheers!
Fail2ban is an open source cross platform tool that leverages your firewall to block persistent threats from actors that are trying to break into your server. There are a number of services that typically run by default on most standard linux distros. I typically work mostly with Oracle Linux which is a derivative of Red Hat Enterprise Linux. I like it because it’s free to download and use, you only have to pay for support if you want it.
Eric Steed - Technology Evangelist 