CP-312874 SMAPIv3 storage-layer engine for SXM v3 snapshot migration - #7223
CP-312874 SMAPIv3 storage-layer engine for SXM v3 snapshot migration#7223LunfanZhang wants to merge 1 commit into
Conversation
| ~dest_sr:ctx.dest_sr ~mirror_vdi:working_vdi ~mirror_datapath:working_dp | ||
| ~verify_dest:ctx.verify_dest | ||
| in | ||
| Unix.sleepf mirror_poll_interval ; |
There was a problem hiding this comment.
nit: I think it is better to use a new var here as it is not in a poll. It just waits a while for readiness.
| in | ||
| D.info "%s nbd_proxy_path: %s nbd_url %s" __FUNCTION__ nbd_proxy_path | ||
| nbd_uri ; | ||
| let mk = Local.DATA.mirror dbg sr vdi image_format live_vm nbd_uri in |
There was a problem hiding this comment.
Need a sleep here like line 414?
There was a problem hiding this comment.
It's last round of MIRROR, so it is fine to without sleep.
| module MIRROR : SMAPIv2_MIRROR = struct | ||
| type context = unit | ||
|
|
||
| let send_start _ctx ~dbg ~task_id:_ ~dp ~sr ~vdi ~image_format ~mirror_vm |
There was a problem hiding this comment.
send_start now blocks synchronously for the whole tree. Have you estimated the time it takes is acceptable when the tree is deep and wide?
There was a problem hiding this comment.
QEMU mirror has better performance, so when the tree is deep and wide, the new SMAPIv3 will much faster then SMAPIv1.
There was a problem hiding this comment.
Were confounding removed from the comparison? Like using different hardware disks for both tests
There was a problem hiding this comment.
Good question - Both sources are now local SSD on hardware-identical hosts. The two hosts are the same model throughout: AMD EPYC 9124 16-Core (32 CPU, 3.0 GHz), 137 GB RAM, XenServer 9.0.0 same build, and — directly on your point — the local SR on both hosts is the same disk, sda = PERC H755N Front, 1.5 TB, non-rotational, behind an identical Broadcom MegaRAID SAS39xx controller. The xfs (V3) and ext4 (V1) SRs are filesystems on the same disk model and controller, not different hardware tiers.
Implement the new SXM v3 storage-layer migration for SMAPIv3 SRs, enabling a VDI to be mirrored together with its snapshot chain, including reverted/nested snapshot structures. storage_smapiv3_migrate gains snapshot-tree discovery (get_snapshot_tree), the DFS walk that mirrors/clones each node while keeping the storage chain advancing on the same leaf, NBD proxy export, and the send_start orchestration that drives the per-node mirroring. storage_migrate_helper.State is extended with a snapshot_relation type and set/get/remove_snapshot_mappings so the source->destination snapshot pairings recorded during mirroring can be retrieved later to restore snapshot metadata on the destination. Signed-off-by: Lunfan Zhang[Lunfan.Zhang] <Lunfan.Zhang@cloud.com>
f687782 to
c91f92c
Compare
| let set_snapshot_mappings mirror_id relations = | ||
| Xapi_stdext_threads.Threadext.Mutex.execute mutex (fun () -> | ||
| Hashtbl.replace snapshot_mappings mirror_id relations | ||
| ) | ||
|
|
||
| let get_snapshot_mappings mirror_id = | ||
| Xapi_stdext_threads.Threadext.Mutex.execute mutex (fun () -> | ||
| Hashtbl.find_opt snapshot_mappings mirror_id |> Option.value ~default:[] | ||
| ) | ||
|
|
||
| let remove_snapshot_mappings mirror_id = | ||
| Xapi_stdext_threads.Threadext.Mutex.execute mutex (fun () -> | ||
| Hashtbl.remove snapshot_mappings mirror_id | ||
| ) |
There was a problem hiding this comment.
Can we create a separate mutex for the snapshot_mappings table, rather than reuse the access_table mutex? It seems to me that they are independent, and this creates unnecessary contention.
4 participants
Implement the new SXM v3 storage-layer migration for SMAPIv3 SRs, enabling a VDI to be mirrored together with its snapshot chain, including reverted/nested snapshot structures.
storage_smapiv3_migrate gains snapshot-tree discovery
get_snapshot_tree, the DFS walk that mirrors/clones each node while keeping the storage chain advancing on the same leaf, NBD proxy export, and thesend_startorchestration that drives the per-node mirroring.storage_migrate_helper.Stateis extended with a snapshot_relation type andset/get/remove_snapshot_mappingsso the source->destination snapshot pairings recorded during mirroring can be retrieved later to restore snapshot metadata on the destination.Design refer: https://github.com/xapi-project/xen-api/blob/feature/sxm-v3/doc/content/xapi/storage/sxm/sxm-v3-with-snapshot.md