I need some assistance with an issue I'm experiencing in my XenServer 7.6 environment.

I was performing a backup of a virtual machine (VM) using the Xackup tool. The backup process did not complete correctly, and as a result, the snapshot created for the backup has become stuck.

• Problem: The snapshot appears in the graphical interface (XenCenter) but I am unable to delete it from there.

• Status: The snapshot is not currently connected to any active VM.

• Impact: It's unnecessarily taking up disk space on my storage.

Does anyone know why this error might be occurring or how I can forcefully delete this snapshot from the command line, since the graphical interface isn't allowing it?

We are facing an issue with Citrix Hypervisor 8.4 related to local storage repositories (SRs), and I’d like to ask for your support or guidance.

Environment

• Tested on both VMware ESXi and VMware Workstation (nested environment).

• Same behavior observed across both platforms.

Issue Details

1. Clean Installation of 8.4
   When attempting to create a local SR using the following command:

   xe sr-create host-uuid=<valid_uuid> content-type=user name-label="Test SR" shared=false device-config:device=/dev/sdc type=lvm
   

   The following error occurs:

   Error code: SR_BACKEND_FAILURE_53
   Error parameters: , Logical Volume unmount/deactivate error [opterr=errno is 5],
   

2. Upgrade from 8.2 CU1 → 8.4

   • The local SR that was working in 8.2 becomes unavailable after the upgrade.

   • Attempts to activate the volume group fail:

     lvchange -ay /dev/XSLocalEXT-19b1f7c1-6c76-a31d-32eb-4479b9caebd4/19b1f7c1-6c76-a31d-32eb-4479b9caebd4
     Volume group "XSLocalEXT-19b1f7c1-6c76-a31d-32eb-4479b9caebd4" not found
     

   • Running fsck also results in:

     fsck.ext2: No such file or directory while trying to open /dev/XSLocalEXT-19b1f7c1-6c76-a31d-32eb-4479b9caebd4/19b1f7c1-6c76-a31d-32eb-4479b9caebd4
     Possibly non-existent device?
     

Observations

• On Citrix Hypervisor 8.2 CU1, local SR creation and usage works normally.

• After upgrading to 8.4, existing SRs are lost/unavailable.

• On a fresh 8.4 installation, SR creation fails with SR_BACKEND_FAILURE_53.

Request

Has anyone else encountered this issue with local LVM SRs in Citrix Hypervisor 8.4?

• Is this a known bug or change in SR handling between 8.2 and 8.4?

• Are there any workarounds to re-import or properly create local SRs in 8.4?

Any insights or guidance would be greatly appreciated.

Thank you,

I have a XenServer 7.6 installation and I'm looking to expand its storage by adding an NVMe disk. My motherboard is a Gigabyte GA-H77M-D3H, which does not have a native M.2 slot.

My plan is to use a PCIe to NVMe adapter and install a 1TB NVMe disk into one of the motherboard's PCIe x16 slots.

My question is: Does XenServer 7.6 support the detection and use of an NVMe disk connected via a PCIe adapter on a motherboard like the GA-H77M-D3H, for the purpose of being used as a Storage Repository (SR) for virtual machines?

Your assistance is greatly appreciated.

   I have configured a xenserver 8 on VMware VM and deployed two windows 2022 VMs in that xenserver. I have created two shared disks on the server and attached on both the VMs

The disks have volume created on them and are active

I need to create a failover cluster on one of nodes. But when I do "Add Disks" in the FailOver  cluster Manager, it gives this error

Powershell Commandline gives this error

I created the xendisk using the xe commands
 

xe vdi-create  virtual-size=$((20*1073741824)) sr-uuid=<sruuid> type=user name-label=<name> sharable=true sm-config:type=raw

xe vbd-create vm-uuid=<vmuuid> device=2 vdi-uuid=<vdi-uuid> bootable=false mode=RW type=Disk
xe vbd-plug uuid=<vbduuid>

The Failover cluster validation report suggests

Disk bus type does not support clustering. Disk is on the system bus. Disk partition style is MBR. Disk type is BASIC. 

I tried with MBR and GPT both. 

My question is that does XenServer support clustered disks . Do I need to change the way I createnthe shared disks on the XenServer

Thanks

Shobha

How to proceed to create a SR with Local Storage ? 

I am testing to create one with all Local Storage on the different Hosts.

I want to enable the HA on it.

Currently not possible.

Regards,

Francois

I want to use a PowerShell script, to each VM whose name starts with W10PRD, to add a 10 Gb disk that will be used as a persistent disk in an MCS scenario and expand the RAM to 16 Gb and the vCPU to 4.
 I created the script below for this purpose. This script works fine until creating the disks. So things start to go wrong from line 76 onwards.

I then get this error:
Error occurred while processing VM W10PRD1003: Cannot bind parameter 'SR'. Cannot convert the "@{uuid=<UUID of Local Storage SR>; name_label=Local storage}" value of type "Selected.XenAPI.SR" to type "XenAPI.XenRef`1[XenAPI.SR]".

# Variables
$hostName = "xenserver host name"
$Username = "admin_account"
$Password = "admin_password"
$vmPrefix = "W10PRD"
$storageRepoName = "Local storage"
$diskSize = 10GB # 10 GB disk
$diskLabel = "Persistent-disk"
$RAMinGB = 16
$RAMinBytes = $RAMinGB*1073741824
$vCPU = 4
$vCPUatStartup = 4
$vCPUMax = 4

# Import XenServer module and connect
Import-Module XenServerPSModule
  Try {
        $Session = Connect-XenServer -Url https://$hostName -UserName $username -Password $password -NoWarnCertificates -SetDefaultSession
    } Catch [XenAPI.Failure] {
        [string]$PoolMaster = $_.Exception.ErrorDescription[1]  
        Write-Host -ForegroundColor Red "$($Pools.$Pool) is slave, Master was identified as $PoolMaster, trying to connect"
        $Pools.Pool = $PoolMaster
        $Session = Connect-XenServer -url "https://$PoolMaster" -UserName $username -Password $password -NoWarnCertificates -SetDefaultSession
    }

# Search for XenServer host with specified name
$xenHost = Get-XenHost | Where-Object {
    $_.name_label -eq $hostName
} | Select-Object uuid, name_label

# Show saved Host UUID and name
$xenHost

if ($host -eq $null) {
    Write-Output "XenServer host not found."
    return
}

# Get all VM's on speciefied host that starts with the VMPrefix
$vms = Get-XenVM | Where-Object {
    $_.is_a_template -eq $False -and 
    $_.is_a_snapshot -eq $False -and
    $_.name_label -like "$vmPrefix*"
} | Select-Object uuid, name_label

# Show saved VM UUIDs and Names
$vms

# Get the storage repository by name
$storageRepoUUID = Get-XenSR | Where-Object {
    $_.name_label -like "$storageRepoName"
} | Select-Object uuid, name_label -First 1

if ($storageRepoUUID -eq $null) {
    Write-Output "Storage repository not found."
    return
}

# Show  saved SR UUID and Name
$storageRepoUUID

foreach ($vm in $vms) {
    $dynamicMin = $RAMinBytes
    $dynamicMax = $RAMinBytes
    $staticMin = $RAMinBytes
    $staticMax = $RAMinBytes

    Set-XenVM -Name $vm.name_label -Memory $RAMinBytes
 }

foreach ($vm in $vms) {
    Set-XenVM -Name $vm.name_label -VCPUsAtStartup $vCPUatStartup -VCPUsMax $vCPUMax
}

foreach ($vm in $vms) {
    # Create a new virtual disk (VDI)
    try {
    $vdi = New-XenVDI -NameLabel $diskLabel -VirtualSize $diskSize -SR $storageRepoUUID -ReadOnly $false

        if ($vdi -eq $null) {
            throw "Failed to create VDI for VM: $($vm.name_label)"
        }
      

        # Create a virtual disk drive (VBD) connecting the VDI to the VM
        $vbd = New-XenVBD -VM $vm.uuid -VDI $vdi.uuid -Device 2 -Bootable $false -Mode RW -Type Disk

        if ($vbd -ne $null) {
            Write-Output "Added a 2GB disk to VM: $($vm.name_label) on repository: $storageRepoName"
        } else {
            Write-Output "Failed to attach VDI to VM: $($vm.name_label)"
        }
    } catch {
        Write-Output "Error occurred while processing VM $($vm.name_label): $($_.Exception.Message)"
    }
}

# Close XenServer Connection
Disconnect-XenServer

Who knows what is wrong with this script, or knows a better option for creating 400 disks on 400 VMs?

I am facing the following issue after a system reboot to power issue,

I am using XenServer 7.6, which is not showing the SR in Xenceter.

using lsblk -f cmd I am able to see the SDA list and I don't see any output for pvdisplay or lvdisplay.

So I thought that vg is corrupted so tried to recover using "vgcfgrestore vol_grp --test -f /etc/lvm/backup/VG_XenStorage-c5645c0f-5a11-3ab1-e220-abd18a01aab0" getting TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.
  Couldn't find device with uuid 1m783S-mATE-T3TF-Pug8-jaNS-e3v3-9DHlmc.
  '/etc/lvm/backup/VG_XenStorage-c5645c0f-5a11-3ab1-e220-abd18a01aab0' does not contain volume group 'vol_grp'.
  Restore failed. error.

If anyone has an idea on this error please let me know.

Thanks in advance.

we want to remove/delete a Virtual Xe Recovery File System from an deleted VM but cannot remove it - also with xe pbd-unplug

xe pbd-unplug uuid=46dde978-7801-263d-27a3-7de6251d6a86
Error code: SR_BACKEND_FAILURE_74
Error parameters: , NFS unmount error [opterr=umount failed with return code 16], 

How to remove it?

robert

we are using XenServer Hypervisor 8.2 in a pool with 4 Hosts.

All of them are connected via FC to our SAN-Storage.

I created 4 Volumes ( FC SAN shared ) with 15TB. Now i wanted to know the max size for a Volume ? Is it 20 or 40 or 100TB.

What is best practice ?

Regards

Martin

I have a problem with an LVM storage in XenServer 8.1. The storage is 99.9% full and I can no longer take snapshots etc. All snapshots have been deleted but without success. There should be enough disk space since the disk is 1 TB and the virtual server is only 320 GB together (150 and 170 GB).

I would really like (paid_ support for this as soon as possible as I can also create backups of the virtual servers.

Thanks for the reaction.

remove-vdi deletes it ... how to just remove it ?  

using Citrix hypervisor 8.2
 

I just installed a couple of servers with Citrix Hypervisor and selected thin disk.

So when I deploy a VM with say 100GB of disk it always shows as 100GB.

Is that just the way its shown in the Citrix XenCenter?

Is it really only taking up with the VM says like 30GB on the host disk?
All storage is local RAID sets.

I chose not to select the VM's storage as I have added that in the past. (If I should do this during the install let me know).

It been a while but I have an HP DL360p Gen8  Yep its a bit older but works.

There is a single SCSI card which I have setup RAID1 (2 drives) for OS and RAID5 (5 drives) for VM's.

However I can't seem to get the RAID5 volume added to Xen.

What am I missing?

Recently we started experiencing issue while powering ON VM. The Power ON process  is failing with "Internal Error: xenopsd internal error: IDL Error:[S(Unix.Unix_error(Unix.EMFILE,"socket",""))]"

Any idea / advice ?

we want to configure new servers for our Citrix Hypervisor Farm.

We have a offer from DELL for a PowerEge R750 with a PERC H355 storage controller and a HPE DL385 with a MR408i-o controller. Both are not on the HCL for xenserver.

Has anybody experiences with one of these controllers an citrix hypervisor 8.2 ? 

We have a very old host running XenServer 7.0 for two Windows Server 2012R2 VM's, one a large DB server and the other a web server. Local LVM storage using SSD's. We were deploying Veeam Agent as a backup service on the large DB VM (160GB RAM, 4.5TB of storage across multiple partitions) and seemed to be the cause. Not long in to the initial backup it throw up some errors and the backup failed.

Windows reported event 153 warnings complaining about storage: The IO operation at logical block address 0xcaf0 for Disk 3 (PDO name: \Device\0000003e) was retried.

It reported this against multiple partitions. 

Digging further I've found in /var/log/daemon.log lots of errors like this:

May 22 21:07:31 hyp tapdisk[13252]: tap-err:guest_copy2: 31/51776, ring=0x2480010: req 7449327995633631470: failed to grant-copy segment 0: -3

May 22 21:07:31 hyp tapdisk[13252]: tap-err:tapdisk_xenblkif_complete_request: 31/51776, ring=0x2480010: req 7449327995633631470: failed to copy from/to guest: Input/output error

May 22 21:07:31 hyp tapdisk[13252]: tap-err:guest_copy2: 31/51776, ring=0x2480010: req 7449327995633631471: failed to grant-copy segment 0: -3

May 22 21:07:31 hyp tapdisk[13252]: tap-err:tapdisk_xenblkif_complete_request: 31/51776, ring=0x2480010: req 7449327995633631471: failed to copy from/to guest: Input/output error

May 22 21:07:31 hyp qemu-dm-31[13794]: XENVBD|PdoCompleteResponse:Target[4] : READ BLKIF_RSP_ERROR (Tag 31d80e0)

May 22 21:07:31 hyp qemu-dm-31[13794]: XENVBD|PdoCompleteResponse:Target[4] : READ BLKIF_RSP_ERROR (Tag 31d80e1)

May 22 21:07:31 hyp qemu-dm-31[13794]: XENVBD|PdoCompleteResponse:Target[4] : READ BLKIF_RSP_ERROR (Tag 31d80e2)

The hardware is a Dell R730 with H730P PERC and some Samsung SM883 SSD's. The disks do not report any issues in SMART and the controller log has not recorded anything. We ran our old backup solution afterwards which completed without any issues. 

Now obviously using outdated things like Xenserver 7.0 and soon to be EOL 2012R2 isn't particularly useful, and we'd like to upgrade to something newer, but we were hoping that by replacing out our old backup system with Veeam that it would provide us a path to doing that with better DR should there be issues as the current system is quite slow to restore. Veeam Agent runs fine on the small 150GB VM on this host without throwing up issues. . 

Any thoughts on the cause of this and what we could do?

The host has been up for 339 days, we could arrange a maintenance to reboot perhaps although it's brought up some nerves about issues if we do that. 

Thanks,

John

We are facing a problem with a new Pool. This new pool is entirely made out of Dell R720, furthermore this servers are all up to date, and firmware/hardware is exactly the same on all the pool hosts. The OS version is the last stable version of XCP-ng. The OS of the VM is the last stable version of Debian 12. 

The method we use is the following: We deploy a virtual machine through a custom template previously standardized by us (basic Debian 12 + cloud-init cloud-initramfs-growroot). Once the machine is deployed, we install and configure some additional software we provide, and we configure a IP. All this is done through the use of a custom ISO that have the user-data and meta-data files for cloud-init. This ISO is provided by an NFS server which is connected to the pool through an SR which is configured as an NFS ISO Library. As soon as cloud-init process is done, the same script unmounts the ISO ("eject" command), and a reboot is done ("/sbin/reboot" command). Here's when our error appears.

We have encountered that the following error does affect all VM's in one random host in the pool.

The moment the OS of the VM sends the reboot order, XCP-ng shows the console blank (white screen), preventing us from any further operation through console. The only actions available (making use of the buttons above) are "Force Reboot", "Force Shutdown", and "Pause", but these mentioned buttons don't execute any actions.

On the other hand, VM's on other hosts, work just fine (the process explained above finishes correctly and the VM is operative). 

It should be noted that this same method is being used for other pools that are working correctly, so it is an error that we are only experiencing in this pool.

While investigating the issue, we have noticed that timeouts are appearing repeatedly in the affected host logs only when the ISO needs to be ejected:

/var/log/kern.log
Aug 21 12:40:01 (host) kernel: [13417.136313] nfs: server (IP) not responding, timed out.

When this log is shown, if we try to "df -h" to check the mount point, the command gets stuck and never finishes. Doing a "strace df -h" we can see that it is trying to reach the mount point itself:

[12:41 (host) ~]# strace df -h
[...]
stat("/run/sr-mount/e4fcedb2-33f6-7f0f-8263-75416dd4412a",

Through Xen Orchestra, we can see that the SR is correctly attached, no errors whatsoever.

We also checked connectivity between the host itself and the NFS server and they are seeing each other correctly

.

The thing is, before starting installing VMs we can see the SR mounted:

[12:27 (host) ~]# df -h
Filesystem                              Size  Used Avail Use% Mounted on
devtmpfs                                3.9G   20K  3.9G   1% /dev
tmpfs                                   3.9G  272K  3.9G   1% /dev/shm
tmpfs                                   3.9G   11M  3.9G   1% /run
tmpfs                                   3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda1                                18G  3.4G   14G  21% /
xenstore                                3.9G     0  3.9G   0% /var/lib/xenstored
/dev/sda5                               3.9G  481M  3.2G  13% /var/log
(IP):/var/ISO                            33G   23G  9.1G  71% /run/sr-mount/e4fcedb2-33f6-7f0f-8263-75416dd4412a <-- Mount point
tmpfs                                   792M     0  792M   0% /run/user/0

Thanks in advance,

Regards.

I have problems with HBA storage. I can only run VMs once after clean installing or importing from XVA from still working older host machine.

After shutting down the VM machine or rebooting I get the message: VDI is not avaiable OR An emulator required to run this VM failed to start. When I try to rescan SR HBA: VDI is not avaiable.

The only solution is to delete volumen on MSA 2060, create again, attach in XenCenter and format. I have tried to create small LUNs ~100GB, 50GB and still the same. First start is ok, working with out any problems, but after restart VM I get error.

I have two servers HP ProLiant Gen10 in pool connected via FC to HP MSA 2060.

Citrix Hypervisior and XenCenter is updated.

I have similar configuration with HP ProLiant Gen9 with HP MSA 2040 on Citrix Hypervisor 8.2 and it's working fine for 5 years.

Please help me with that, I have no idea what to do...

XenCenter_log.zip

So there won’t be any critical data hosted and VMs could be rebuild quickly in case of hardware errors.

For this szenario what is an up to date recommendation for local storage layout?

A single NVMe SSD offers best performance along with lowest price.

But in case of an hardware failure host and VMs needs to be rebuilt.

Splitting OS and VMs to two SSDs does not provide any real advantage, does it?

Former best practice has been to use SAS based raid, this is what the vendor still proposed:

XenServer on 2 SATA SSDs in raid 1, VMs on 4x SAS SSDs in raid 5.

But is this the best soltions when using actual Enterpise SSDs?

Isn’t the bottleneck a raid card talking SAS?

One of my customers told me to have one SSD disk failed on 500 hosts in 3 years.

This is quite a could AFR and would not justify the money to spent for multiple SSDs running in raid.

So we appreciate any experience or recommendation on running XenServer on SSD based local storage.

Gradually error

    ExternalAPIException: SR_BACKEND_FAILURE_1200. SR_BACKEND_FAILURE_1200 Failed to read from journal clone_6b4db36b-fb10-4976-ad5b-efae1427335f_1

    ExternalAPIException: SR_BACKEND_FAILURE_1200. SR_BACKEND_FAILURE_1200 Failed to read from journal leaf_49563d68-9981-450a-bad4-b2c22a495b90_1

started to appear in most VMs and backup does not back up these VMs.

Where I look to find problem?

There is some information here.

https://discussions.citrix.com/topic/394653-xenserver-iscsiconf-and-multipathconf/#comment-2007016

This is what I received from Nimble support.

recommended settings in the /etc/iscsi/iscsid.conf file:

noop_out_interval = 5
noop_out_timeout = 10
replacement_timeout = 10
queue_depth = 64

AND

add the Nimble Storage device section to the mutipath.conf ( be sure to have user_friendly_names no  - this is disabled by default

defaults {
      user_friendly_names no
}
blacklist {
      devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
      devnode "^hd[a-z][[0-9]*]"
device {
      vendor "*"
      product "*"
      }
}
blacklist_exceptions {
      device {
      vendor "Nimble"
      product "Server"
      }
}
devices {
      device {
      vendor "Nimble"
      product "Server"
      path_grouping_policy group_by_prio
      failback immediate
      path_selector "service-time 0"
      path_checker "tur"
      hardware_handler "1 alua"
      prio "alua"
      dev_loss_tmo infinity
      fast_io_fail_tmo 5
      rr_min_io_rq    1
      rr_weight         uniform
      }
}

Hello,

For some time now, I can't take a snapshot. When I try to create it it gets stuck at 0% and never finishes. The only way to get out of that state is to force shutdown.

I don't know what to do, but I need to be able to make those backups.

I would appreciate if someone could help me.

I have tried moving the disk by going into storage, right clicking the disk and move and also tried using the migration wizard.

does anyone know of a workaround or fix for this.

Thanks in advance