Saturday, June 7, 2008

How to manually merge Hyper-V snapshots into a single VHD

Okay, so you have to recreate your VM configuration and you absolutely know that your VM had a snapshot at some time.

You also realize that if you just link to the base VHD that you will lose the current state of your VM - what do you do?


You manually merge your snapshots into your base VHD before you boot your VM. (I am assuming that you know how to connect to an existing VHD using the new VM wizard).
Merging of snapshots can be performed manually. This is achieved by:

  1. On your Hyper-V host.Power off the Virtual Machine.
  2. Make a copy of the VHD and its corresponding AVHD files.
  3. Rename the AVHD extension to VHD.
  4. Write down the order of the disks from youngest to oldest (the oldest should be the root VHD). You can do this by looking at the last modified time stamp on the origional AVHD files, find the one that last changed. And find the last one that changed before it.
  5. In the Hyper-V manager, open the Edit Disk wizardBrowse to the youngest VHD in the chain, then choose 'reconnect' to point to the next youngest (the one that came before).
  6. Open the Edit Disk wizard a second time and merge.
  7. Then repeat the process until you have only a single VHD.

In a disaster case, you need to recover a copy of the root VHD prior to attaching it to a new VM and booting it (the act of booting it, modifies it)

Usually the most difficult part of this process is finding the last AVHD (differencing disk) in the chain.

The easiest way to do this is to find the configuration file for the VM.






Then open up that configuration file and locate the information for the virtual hard disk. In the screen shot below is the location of the current running state of the VM. The snapshot is a point in time to return to, the current running state is the "Now" and is contained in a differencing disk (AVHD) after a snapshot has been taken.


Now, find that AVHD file within the file system and rename it to VHD.

Now, go back tot he Hyper-V manager and open the "Edit Disk" wizard - Select the disk that you renamed above, and merge this disk into the one before it.

This process can be continued until all of the snapshots are merged back into a single VHD (the base VHD).

103 comments:

Omar said...

I ran into an issue where this was necessary and this processed worked great for me. I almost had a heart attack when the VM disappeared from the console and I recreated it with the original VHD and lost 23 days worth of data- luckily I knew to backup the entire VM guest folder before I tried anything and with this article I got back to the state right before the issue occurred.

BrianEh said...

Excellent.
Unfortunately, I predict that there will be more that experience the same pain.
I have my idea about what needs to be developed to fix the issue - my idea is simple in concept but very complex to implement, as no one has the solution today.

Yan said...

Omar had my exact issue except it about 30 days. I'm on the phone with PSS now and they are thinking its looking bad. the vhd was booted but according to the modified date was not changed. hopefully this method will work.

John_1 said...

We just had this issue. VM disappeared with a snapshot. Snapshots while convenient can bite you in the rear. If I have time I always run a diskshadow and copy the entire VHD so if you have to roll back you just copy the vhd back.

Here are the commands for diskshadow to back up the VHD using VSS (its kicks off vss inside the VM so the data is consistent)

Go to cmd prompt:
diskshadow
set context persistent
add volume d: alias ddrive
set verbose on
create
expose %ddrive% y:

This will take a VSS snapshop of my D drive (where my vhd lives) and then mounts it as drive letter Y: Now copy the contents of Y: to a safe place. When your done you can delete the snapshop and unmount Y:

Delete Shadows Volume D:
unexpose y:

BrianEh said...

Excellent idea and tip.
This is why, way early I wrote the post about snapshotting not being the same as checkpointing in VMware.
Running a VM with snapshots in production was not the design model. The model was snapshot, update, verify things are good, merge.
However, those of us who grew up with VMware (me included) are used to the VMware model, where a snapshot is a walk away copy of a vm at a point in time.
Totally different way of thinking.

Anonymous said...

You saved my butt. Thanks for taking the time to blog on this. May someone return the favor to you one day.

Tim

Aaron Segal said...

I am running my first production server on Hyper-V (Server 2008).

Once the virtual server was initially configured, I took a snapshot (just as I would have in VMWare).

Today, I went to increase the hard-disk size, and learned I couldn't do so on account of a snapshot.

So I shut down the VM and deleted the snapshot (which would have taken 2 seconds in VMWare).

The snapshot is deleted, but the system is now "merging" (a process taking too long - 14% after 15 minutes).

My heart is racing - I'm scared that this "merge" may be overwriting the more recent data.

Many regrets right now that I went with Hyper-V over VMware.

I at have least another hour before I can find out whether I just wiped out my office server, and there is no way to go back now.

I've come to trust VMware's snapshots so much, that I didn't think to backup before deleting a snapshot in Hyper-V.

Yikes - this is scary.
-Aaron

BrianEh said...

And, this is why I wrote this other article way early..

http://itproctology.blogspot.com/2008/03/snapshottingnot-your-checkpointing.html

And then this:

http://itproctology.blogspot.com/2008/05/hyper-v-snapshots-more-about-how-they.html

Now, growing your disk is actually a different issue.

Aaron Segal said...

Yes - well, had I found and read your essays before venturing into this project, I'd have experienced much less chest-pain.

Good news, though: The merge finished (was much faster after 60-80%) and the system appears intact.

Was able to expand the drive without any trouble. So I'm back where I need to be.

I guess the two big question I have are:
1) What is the best way to back up a Hyper-V system? (I AM able to shut down the system for backup if needed).
2) How easily can those backups be transferred to alternate hardware in the event of a system failure? (VMWare made that easy.)

-Aaron

BrianEh said...

Hyper-V uses VSS (Volume Snapshot Service) to backup running virtual machines.

Now, that being said, check with your backup software vendor to see if they support Hyper-V VSS (not just plain old Windows VSS) as they are different.

The other option is to export the VM, copy it, then import it to a new host. The negative is that your VM must be powered off. But it always works.

VMware made this all very easy, because VizionCore essentially built the backup for them and VMware later copied it. There was a time when VizionCore was the only backup utility for ESX server, other than building your own Perl scripts. That is where Hyper-V is today.

However, the backup vendors are coming up to speed pretty quickly on supporting Hyper-V VSS.

The Third option is to run a traditional backup agent within the VM. Then when you need to restore, restore into a new VM (just like you would with hardware). Yes, it does not use all the fancy, hnads off, VM stuff (just drop and go) but it works, and works on a budget.

Aaron Segal said...

1) Backing up anything live makes me nervous - seems to leave too many opportunities for data corruption. But I will investigate those options.

2) If I can just shut down the VM and then export it to a backup drive, that would be perfect.

But then the next question is whether that can all be automated.

Can I implement some sort of script or batch-file in Server 2008 to shut down the VM, export it, and then restart it?

BrianEh said...

check out this: Powershell script for exporting(backup) Virtual Machines script from Tore:
http://mindre.net/post/Powershell-script-for-snapshot-and-exporting(backup)-Virtual-Machines.aspx

Ignore tha snapshotting, you don't want to do that. And set it to shutdown (not power off nor pause) if you want a totally clean state on the OS.

ebe said...

Hello
That damn Offics.. AV lost its configuration (exclusions) and then I had the now attended merge/copy/remove error because of it !
one of the VM had a snapshot. your post helped me and it worked like a charm. Great Hyper-V :) I love it.
Thank again. Regards

Anonymous said...

This is great, thanks for taking the time to post. I lost an important 11 days worth of data and was able to recover following these steps.

snajke said...

this happend to me:
my vm was fine and i did a snapshot.
i powerd off my vm and extended the vhd.
after extending the vm doesnt boot.

so i made a new vm and added the vhd file to this machine.

i found the very old data on this vm - so about 2 weeks are missing.

now i have the server.vhd file and 5 avhd files.
the oldest (400kb) i can merge into the vhd file - but the important snapshot is about 9 GIG and i cant merge that snap.

any help welcome!
i am tryng for 2 days now to restore this vm.
thx

BrianEh said...

If I followed the last comment properly. You created a new VHD, then attempted to merge the AVHDs (differencing disks) back to it.

And you have a 9Gb differencing disks that won't merge.
This is a lot of change, and it could take a long time.

Because of the way differencing disks work if the base VHD does not match up properly (dirty sectors and all), then the merge won't work.
This is why it is very important to merge in the proper order and why it is imperative not to modify the base VHD.

I corrently don't know of a tool that will force this to happen.

However, there is a tool that can take a VHD and another VHD and compare them and produce a differencing disk that contains the differences between the two.
The caveot is that the OS must be the same, the patch level must be equal, etc.

That being said - it gives me an idea. You could use a similar way to recover data, for example.
Create a new VHD with just the OS installed, then merge to it. You won't have your application, and you 'might' only get the data back during your time frame.
But it is a possiblity.

The new base VHD must be the same size as your origional VHD that the AVHD was atached to.

Bunyamin said...

i did it like you wrote. And it worked.
Thanks,

Anonymous said...

Thanks. You saved my day :-)! Kr., Bernd.

Robert said...

I have been plagued with this issue more than once, shut down my Hyper-V VM using the Hyper-V manager native to server 2008 because of disk space issues. This VM in question holds our office Sharepoint Server, the Sharepoint Databases are backed up using Backup exec.
So the machine is now shut down after a 6 hour merge (What it is merging I have yet to figure out as it did not even seem to help me), I selected Edit Disk from the Actions menu on the right side of Hyper-V manager browse to the disk and select it, this gives a little disclaimer that you should not edit a Virtual Machine if it has a snapshot or differencing disk. I admit I sometimes skip reading small unemphasized print like this, It is obvious that Microsoft knows this is an issue judging from all the posts on the http://social.technet.microsoft.com/Forums and hopefully they will roll out a SP to fix this, many people have fallen into this trap and I am sure many more will until this is fixed. So here is where things went bad, made the disk 60GB from 50GB, thinking nothing of it I turned on the machine 5 minutes before quitting time and after 50% of starting I received this infamous error:

'SERVERNAME' Microsoft Emulated IDE Controller (Instance ID {83F8638B-8DCA-4152-9EDA-2CA8B33039B4}): Failed to power on with Error 'One or more arguments are invalid' (0x80070057). (Virtual machine CFB19551-4BB9-4B30-8EC7-1A461C2F6A33)

'SERVERNAME': Failed to open attachment 'D:\Images In Use\Server 2008 x64 Sharepoint\Snapshots\CFB19551-4BB9-4B30-8EC7-1A461C2F6A33\Server 2008 x64_C3EBA156-8CF4-4375-9DF3-43EF2C4B80AA.avhd'. Error: 'One or more arguments are invalid' (0x80070057). (Virtual machine CFB19551-4BB9-4B30-8EC7-1A461C2F6A33)

Luckily I am able to work from home but unfortunately it was during my evening that I would rather be out barbequing and playing with my kids.
After reading several posts on possible solutions to fix this error and trying some of them I was unable to resolve the issue. One of the articles “http://networkfoo.org/server-infrastructure/recovering-your-virtual-machine-how-manually-merge-hyper-v-snapshots-back-one-“ talked of renaming the last snapshot .AVHD to a .VHD and try to merge this as a differencing disk. Maybe in some situations it would work but not in my situation. There was one great piece of information here, it mentioned using WinImage 8.5 to read this newly renamed .AVHD ".VHD" file and copy out your data. I downloaded it and it seemed to read the VHD fine then I noticed that under the Disk tab there was an option to convert a virtual Hard Disk Image. I ran this option pointed to my renamed (.AVHD) .VHD file gave it a new name and ran it. An hour or two later when it finished my 20GB Snapshot was now a 44GB file.
I created a new VM using this VHD as the disk, fired it up and it actually started. Once it was up I was able to get to our Sharepoint and verify that all the data was present and intact! What a relief! If you decide to try this make sure you make copies of everything just in case this does not work for you but this seems like it was a great undocumented find.
There was one other option that I had not tried yet and that was to use the VHDTool at http://code.msdn.microsoft.com/vhdtool which actually is a command line tool which has a repair option: Repairs a broken Hyper-V snapshot chain where an administrator has expanded the size of the root VHD. The Base VHD will be returned to its original size. But has a warning that "This may cause Data Loss if the contents of the base VHD were changed after expansion". I was not sure I would like to take that chance as the VHD would fail after starting 50%. It would have been my last resort.
I hope this helps everyone as much as it helped me!

BrianEh said...

Without using WinImage you could have accomplished the exact same thing using Diskpart (or Disk Management) and mounting the last VHD in the chain.

Whenever you mount the last disk in a differencing disk chain you mount the entire chain, and therefore if you copy out the contents to a new VHD you get the entire "last running state"

Great feedback about how you managed to work through the puzzle.

BrianEh said...

IMHO - the problem is not that there is a bug in Hyper-V but hat many folks just don't understand snapshots before they begin using them.

This is particualarly true with Hyper-V v1. Where the .AVHD files were saved to the system volume instead of out with the VHD of the VM - this behavior was changed in R2 - specifically because folks kept filling up their system partitions.

Unfortunately many folks are still running Hyper-V v1 and still running into this problem.

The second problem that needs to be improved is that there needs to be some feedback in the UI that a snapshot was deleted and that you need to power off the VM in order for it to merge back.

I can't tell you how amny times I see folks in the forums that create a snapshot, then delete it (poof - it is gone from the UI) and they nover power it off - the end result is that they have been running on a differencing disk for months and were clueless aabout it (most frequently, some co-worker did it).

Robert said...

Brian, This is all good information how come it is so hard to come by I searched for days trying to find a solution that would work and I just stumbled into this one. You woult think there would be more reference to these types of solutions on the technet boards as well as in their KB articles this would save IT people with these problems a lot of pain, searching and typing to find a viable solution..

BrianEh said...

Most forum posts refer folks right to this article.

This is by far a very popular post, and I am trying to collect most of my information here.

TechNet forum posts also age and fall off the bottom, and frankly the forums talks of this less and less over time. Mainly becuase of behavior changes in R2 vs. v1 and that folks are learning more about the platform (it isn't so new anymore).

There is another source, the TechNet Wiki. I am posting much of what I glean from forum threads over there. The Wiki articles don't age and are being corrected and worked on over time. Some products even have their CSS team maintainging their wiki articles.

During Hyper-V's first year I (by far) posted the most information about snapshots trying to get folks to understand what they are and what they are getting themselves into by using them.

http://itproctology.blogspot.com/search?q=snapshot
This search string will give a bunch of background.

I have gotten to know a few of the Hyper-V folks pretty well, and they are well aware of the issue. But at the same time XenServer uses the exact same methodology of differencing disks. And most of us grew up with VMware, where a snapshot was a stand-alone copy of a VM, with no differencing disk chain involved.

As a person who tests software I recognize that there are many layers to the problem.

Don't even get me started on the trend I have seen in the skills of IT systems admins (my forum experiences)... As I always told my Jr. admins, that dialog box said something, do you know what it said? Did you understand what it said? Or did you just click Next?

This is all about helping folks understand the details of what is happening - that is important.

Frankly, I wish that this article was less popular. And the comments have been great, I am glad that folks are contributing back.

Dovid Gross said...

The problem isn't that people misunderstand what is going on. That's secondary to the problem of a poor UI and lack of on-line documentation, especially help. Hyper-V needs to mature.

The Good Samaritans said...

Brian,

I am not sure how often you read comments on this post, but hopefully I can get some help soon.

I have a SharePoint SQL server on a VM and accidentally took a few more snapshots than I needed and didn't realize I ran out of space. When I tried to delete the snapshots, it appears I don't have enough room for it to merge so it tries and nothing happens.

I still have all the snapshots at the original locations. Nothing changed except out of room. The question is:

Can I manually move the snapshots folders to an external drive, point the folder in the VM settings to it. Delete the old snapshot folders and have the merge take place on the bigger drive?

BrianEh said...

The best way to handle this situation (at this time) is to get some type of storage to the Hyper-V host and Export the virtual machine.

Then delete the origional VM, delete all of the virtual disks and snapshots.

Add storage to the host.

Then Import the Exported machine.

If you have Hyper-V R2 then it is a bit easier as we can take advantage of a new feature:
Power-off the VM, take a new snapshot, Export only that snapshot.
Then delete the origional VM and all of its snapshots.
Then Import only the snapshot.

The snapshot Export does not export the entire tree of snapshots but instead creates a new single VHD.

The Good Samaritans said...

Dear Bryan,

I so much appreciated your quick response. I followed your instructions but ran into some new problems and need more help if you could provide them. I run a very small non-profit and not well versed in these kind of things but have to do them.

My config:

Host: WS2008 Enterprise on Hyper-V SP2 only.
VM: WS2008 Standard x 2 machines.

I use the normal process to delete snapshots (as described above). When the VM merged on shut down, it said completed but nothing happens, all the .AVHDs were still there. Not only that it crashed and the disk which it resides disappear from Explorer. Restarting the host server gave an error that the Virtual Discs Service crashed and the RAID screen on reboot showed that disk is "foreign". I had to re-import the configs for it to boot up. There is this KB http://support.microsoft.com/kb/978897 and I installed it but didn't work so I thought it was due to only having 60GIG of space left. So I asked you if I could move the snapshot folder. You responded quickly.

Over the past 3 days, all my waking hours I tried to follow your instructions. It took 13 hours to export the VM and then it failed without reasons. Nothing in event logs.

I then backed them up with Backup Exec 12.5. It went to the verify stage and failed. I browsed the files tree in Backup Exec and saw that there are 3 .AVHD files but 2 are marked as corrupted and the date, time stamp were all exactly the same. The size of the files varies although the NORMAL one is bigger.

Do you think the corrupted files were the reasons for the merge to fail? If so could I simply just delete it out of the snapshot folders?

If all fails and I have to manually merge as you suggested, will it merge if I skip the corrupted files?

I am so desperate and appreciate any help available.

Vien Doan

BrianEh said...

Your host is Hyper-V v1 not Hyper-V R2 - so you can only export an entire VM (the entire snapshot tree).

13 hours for an export is an incredibly long time. Either you were exporting to a very slow disk or something is not right with your physical server.

Each snapshot is merged one by one (after you delete it and power off the VM). If you had multiple snapshots they all need to be merged back together.

Have you attempted the manual process that I outline (without exporting)?
Have you renamed the .AVHDs to .VHD and inspected them with the virtual disk wizard?
Do you have any good backups at all? - Possibly a way to re-create the VM in a manner similar to recovering a bare metal server.

Branko said...

After deleting the snapshot tree the all the .avhd files are still there.
But I have a .avhd file with an identical modified timestamp as .vhd file. I also have .avhd files older than .vhd file.
Should I merge this files with their parents too?

BrianEh said...

Without knowing the full history of this particular VM (or multiple VMs) it is really hard to say.

Questions I would consider:

why are you manually merging in the first place?
What is the history of this VM?
How are the virtual disks arranged in folders? (it is easy to determine who is the parent disk of any particular child disk)
Was a VM or the storage restored at any time?
Was the root VHD restored at some time?

There are a million things that can happen to cause strange time stamps - one might simply be that your VM had more than a single branch in the tree of snapshots.

Anonymous said...

Hi Brian - Looks like you have an army of followers here. Not sure how often you read and respond but:

Having done some reading I decided to take a nice cautious approach is tiding up my VM and snapshots, so I deleted a snapshot, started and then shutdown the VM, and as expected it started a merge, and quite quickly showed 100% and merge suceeded. Superb I thought and then merrily used the same procedure to remove the remaining snapshots, getting merge suceeded every time. Unfortunately I didn't check to see if the AVHD files had been removed, they haven't. Everytime I shutdown this VM I get a very quick merge succeeded, so it knows it has to do something but won't. Anyway I have plenty of disk space, running 2008 R1 SP2. I have tried stoping and starting the service, as this has been suggested as a way to kick start the merge.

Before I start a manual merge can you offer any advise?

Regards
Mike

BrianEh said...

I have seen stopping and restarting the VM Managmenet Service cause the merge to begin to happen again, only to leave a file lock hanging the background.

The only way we could get rid of the file lock was to reboot the entire Hyper-V Server.

This allowed the merge to actually succeed.

Have you checked the Hyper-V Event Logs for any error messages?

phil said...

Hi Brian

Hope you can help!

I have a server running server 2008 and hyper v

i have 4 VMs on the server but one is a file server that has multiple snapshots that i need to merge but i dont have enough disk space on the host server to complete the merge.

The size of the folder that contains the original vhd and the snapshots is 447GB

i have 3 x 300GB drives to put into the server which will give me 600Gb storage in RAID 5.

how can i use that space which will be a separate volume on the same server to help merge my server back to 1 VHD.

I am running out of time so you quick help would be appriciated

Thanks

Phil

BrianEh said...

it sounds to me that you are adding the new 300Gb drives as a new array.
(That is the understanding that I am going forward with - and it expands your options).

1) figure out how much storage your snapshots are taking up. Add up the size of the root VHD plus each AVHD snapshot of this particular VM.
2) If they fit on your new 600Gb volume you are golden and you can simply Export the VM to the new storage. Then rename the origional. Then Import the copy you exported. Test it. Then delete the origional - FYI - if you delete it through the Console the hypervisor is going to try and merge everything in the background.
At a future time you can worry about deleting the snapshots - but don't wait too long.

3) If it does not fit on your new storage volume - Begin by finding the smallest AVHDs and the snapshots they are prt of - and delete these one at a time - for little wins in recovering storage. Then you can eventually do step #2.

One side note - if you run a backup agent within the VM and your VM has dynamic disks - you could be using up a lot of the storage (for empty virtual disk), but the data in the VM is a lot less.
I have seen lots of folks get into situations like this.

This is one of those things that took some time to get into, so you should also plan that it will take some time to get out of.

Oh, and don't forget to have backups. Especially of the VM you are messing with.

philmillward said...

Hi Brian,

Thanks for your quick reply.

You were right to think that i was going to use the new drives as a new array. The size of the original VHD and the new AVHDs that have been created are 447GB in size which will fit on my new 600GB Array.

so can i now go ahead with exporting the VM like you said. If that is the case you tell me the step by step process of exporting the VM and what actually will happen? and also the steps i need to take after the export.

I have already deleted the snapshots using HYPERV Manager will this cause a problem with what i am trying to do because it will try to merge the snapshots on shutdown.

Thanks
Phil

BrianEh said...

I outlined what to do in my previous reply. And I will repeat here.

Add your storage, mount the volume (format, etc.), power off the VM, Export the VM to the new storage, rename the origional VM, Import the VM from the new new storage (you will see the VM appear in the console), power on the new VM. Delete the old VM. The VM will disappear from the console but its virtual disk will remain - any snapshots that existed will be merged into a single VHD. After this completes you can delete the VHD.

The hairy part is that you have already deleted your snapshots. I have not tried exporting a VM that has deleted snapshots that have not merged yet.

From the comments that I have seen, I know it works.
What I don't know for sure is the behavior of the exported VM.
Does the entire snapshot tree get exported? And then the merge process still has to happen? Or do you end up with a VM that is all merged into a single VHD (if you deleted all snapshots)?

It is that part that you would need to be aware of - as it could mean additional power off time for the merge to happen on the new storage.

If you power on a VM that is in the middle of a merge - the merge simply halts and waits until the vm is powered off again.

Thoase are all the gotchas that I can think of at the moment.

philmillward said...

Hi Brian,

Just thought i would update you on this issue.

I have built a new server today so i could test the merge and exporting process that you have described.

It seems because i have already deleted the snapshots it tries to merge on shutdown and if i export at that point it does the export including the snapshots but then tries to carry on with the merge.

if i import the exported machine as you said it does power up but still uses the last snapshot that was taken as the live disk.

the only way i can seem to get it all back into 1 VHD is to manually merge all the AVHDs back with the original VHD.

Is there anyway i can force the merge from taking place?

Thanks
Phil

BrianEh said...

I have asked the Hyper-V team about this and there is no way to stop, pause, affect a snapshot merge that has already been triggered.

The only thing that you can do is to allow it to finish.

On a side note - I would expect your export to equal the currently running VM - unless you have Hyper-V R2 and you are exporting the snapshot instead of the "Now".
(This is a new feature of R2)

That particular point is a bit puzzling.

philmillward said...

Hi Brian,

thought i would let you know the outcome of all my testing.

i added the new storage and created a new array.

i then shutdown the VM which has the snapshots already deleted in HYPERV Manager.

once the export has completed i take the original VM offline and import the previously exported VM.

at this point it doesn't carry out the merge so i have to merge all the disks manually.

also once this has done the only way to delete or stop the original VM from merging is when you have made sure that the new VM you have exported and imported and merged works properly then if you make all the snapshots and original VHS of the original VM read only then the merge doest happen and you can delete (hope that made sense)

thanks for all your help

Regards

Phil

Derek said...

I tried to expand a vhd that had a snapshot tree attached.

I broke it.

I followed your instructions to reconnect all drives. However I get an access error to the file renamed from avhd to vhd.

I do get a merg pending notice if I try to mount to a different vmc. When I try to start the original it tries to merge but then errors due to no access. I did also get an error that the child vhd does not recognize the parent vhd ID.

My Intranet is on this and I need it up, any suggestions

BrianEh said...

Have you seen this:
http://itproctology.blogspot.com/2009/04/never-resize-vhd-with-snapshots-or.html

Personally, I would be working at recovering a backup right now, instead of pushing forward.

I am getting the impression from your post that things are only getting worse.

Now, one thing to do is to create a new root VHD - if you recall what the root VHD contained (such as a bare OS) then you can create a new VHD of the same size and install an OS, repair the chain, and let the merge happen.

Also, the VM must be off for any merge to happen. And if you are getting an access denied, I am guessing that a background merge has the files locked - the only way to unlock is to stop the VMMS.exe service (the Hyper-V managment service).

TKM said...

Hi Brian, I would be going through your instruction today eve. I am just confused with the new VM thing. Do I need to create a new VM after the merge is completed or can I use the merged .vhd with the existing VM? Thanks.

BrianEh said...

The origional VM configuration will be pointing to the last AVHD in the snapshot chain.

At the very minimum you need to modify the settings of the VM to point to the merged VHD after everything is completed. Or else you will ahve an error that the VHD does not exist.

Jim said...

Brian,

Man...this saved my bacon. Thanks so much for posting!

-Jim

sarfo said...

For some reason, my virtual machine is actually running the vm from the avhd file. The path that reads the .vhd file is pointed to the avhd directory. So at this point, im almost certain that is what it is doing. My Question is how do i merge the avhd file with the vhd file, when the the avhd file is the primary one. This is a sql server so i cant afford to lose any data.

BrianEh said...

your description sounds like you have this: .avhd -> .vhd
not this: .vhd -> .avhd

And, I am assuming that at some point some one did not recover the VM and change the settings to point to .vhd (this happens, a lot) because the VM configuration was manually recreated and there was no knowledge of a created and deleted snapshot as it was removed from the GUI.
Check the setitngs of your VM to truely know where the "now" is.
A .avhd is simply a special extension for a .vhd of type differencing disk.
But, the only way that you could have .avhd at the root of your VHD disk tree is if someone manually renamed it that way. It will not prevent it from working.

sarfo said...

"your description sounds like you have this: .avhd -> .vhd
not this: .vhd -> .avhd

And, I am assuming that at some point some one did not recover the VM and change the settings to point to .vhd (this happens, a lot) because the VM configuration was manually recreated and there was no knowledge of a created and deleted snapshot as it was removed from the GUI.
Check the setitngs of your VM to truely know where the "now" is.
A .avhd is simply a special extension for a .vhd of type differencing disk.
But, the only way that you could have .avhd at the root of your VHD disk tree is if someone manually renamed it that way. It will not prevent it from working."


When i look at the file path, it has the Ares_674A00A8-AC18-4937-8089-ADD7CA302009.avhd and parent C:\ProgramData\Microsoft\Windows\Hyper-V\Ares\Ares.vhd. Is there a way to manually combine the two. I need the space bad.

BrianEh said...

Your path is perfect.
The Parent is the VHD that makes the child the AVHD.
First - power off the VM and see if within three minutes Hyper-v begins the merge by itself.
If not, then manually merge the two by renaming the VHD and manually merging with the Hyper-V Management Console, just as this really old, but popular article outlines.

sarfo said...

Your path is perfect.
The Parent is the VHD that makes the child the AVHD.
First - power off the VM and see if within three minutes Hyper-v begins the merge by itself.
If not, then manually merge the two by renaming the VHD and manually merging with the Hyper-V Management Console, just as this really old, but popular article outlines.

Thank You for the reply,sorry i opened such an old tutorial. My Server hosting my vms is running out of space.(not a good thing) So i cant do backups, or anything. I have to do them manually. And now there is a windows update and it takes the server down to 511 mb. This causes the virtual machines to pause. I have shut off the virtual machine. How do i check if it is merging? When you say rename the vhd. You mean the switch the two extentions. As in call the avhd-->vhd and the vhd-->>avhd. Does the actual name do anything?

BrianEh said...

Read the entire article that you are posting comments to. It answers your quentions and describes the process.
Plus there are good comments in the comments.
If you have the Hyper-V Management console open and select your VM - there is a far right column (you might need to scroll the center panel) that shows the Status. It is here that you will see if there is any merging happening.
But the VM must be powered off for the merge to happen.
The AVHD file extension means something to Hyper-V but the actual VHD is simply a differencing disk VHD.

Ray Barbieri said...

BrianEh,

Q: I ran out of disk space on the host so I deleted the snapshots. This didn't free any disk (as your post indicates). I didn't understand and the VM kept pausing so I shut it down hoping something would happen. I spent about 20 minutes trying to understand what happened and figured out that it was merging. But I couldn't leave the server offline while it merged (I have at least 40G in 5 differencing VHD's to merge).

Now I plan to shut down the VM tonight and hope that it will merge overnight and everything will be OK.

So, besides the "bug" you mentioned regarding the UI giving NO indication that the VM needs to be shut down in order to merge the snapshots, I also think it is a bug that you have to shut down the VM to merge the data. Clearly, there's tons of sectors of information that will never be accessed now that I deleted the snapshot. I see no reason (other than difficulty) not to start the recovery while the VM is live.

Alternatively, there should be a big warning when you go to create a snapshot saying don't do this on a production machine.

Ray Barbieri said...

Another comment, I told the system to put my snapshots on another drive but all of my snapshots are kept on the same drive as the guest. Is this a bug? Why do they give you the option to specify the folder to store the snapshot files if they are going to ignore it?

BrianEh said...

Hyper-V v1 allowed having snapshots in a different location than the other virtual disks.

This was great in theory but caused far more human errors than any idea should.

In R2 this behavior was changed. The option is in the GUI to support v1 systems, but the R2 Server does not allow this setting change.
It supports VMs already configured this way and you upgrade to R2, but R2 does not support this configuration beyond that.
So yes, the GUI is misleading. And MSFT knows.

Ray Barbieri said...

Follow up, I did stop the VM and it only took about 2hrs to free up 76GB!!!!

Now the VM's back up and I've learned not to EVER try messing with snapshots on a production system again!

Thanks again. Like others before me,this is the only place to find this info. Keep it up! And Thanks BrianEh for your support. You Rock!

morphwarp said...

Uh I see, dealing with snapshots can cause data loss. What happens exactly on clicking apply? I thought it would merge the snapshot into the image, but it does not intergrate it into the root VHD.

BrianEh said...

Clicking apply, it does just that, it applies that point in time.
The act of applying a point in the past is to toss away the current.
Much of the confusion is because of the use of differencing disks as the mechanism of snapshotting.

Johnny5 said...

This is the most active, informative post on VHD merging and management I have read so far. I wish I had read it 2 weeks ago. I have a server running 2008 R2 and Hyper V Manager "6.0.6002.18005". The disk where the snapshots were contained developed a bad block problem. The snapshots were pointed to an external USB drive - which was not backed up. Bad idea, I know. I rebooted the guest server to apply some updates and it failed to reboot. It said invalid boot media. I tried to merge the last AVHD file using steps much like these, but it failed with a Cyclical Redundancy Check error. I have tried to recover the AVHD files using data recovery software, and I may have a workable copy, but I need to try to merge it in order to know for sure.

I was able to recreate the production virtual server using the original base VHD but lost about 30 days of data. Now, in working through this, I have quite a few files pertaining to this guest server on this host server that have date stamps that are misleading, since I had made copies of things, trying to preserve as much data from this failing hard drive as I could. I am therefore unable to determine which of these AVHD files (there are 3) are for which dates. I would like to know whether there is a way to move all of the files: a copy of the VHD file as of the date of the problem as well as all 3 AVHD files, to a new disk or even a new server, and identify the order of AVHD files to merge in, and then I would like to try to define this merged VHD file as a new guest server and bring it up in order to export some data from it and import that into the currently running, restored server. Is there much hope of getting those missing 30 days, since there has been disk corruption and I am unable to determine the dates of the AVHD files?

BrianEh said...

it is possible to take a VHD and its children AVHDs and simply copy them somewhere else, and put them back together.
The fail proof way to do this is to move them to a totally different workstation.
I recommend this for one big (really big) reason. If you leave them on the production server and you are not careful with fixing the parent disk path - you can accidentially merge an AVHD into the wrong parent disk and wreck production.
The process is basically the same. Each child differencing disk knows its parent (it doesn't know anything else about its life as a snapshot). Simply restore the files to the date in question, then fix the chain, then merge away.
Any Server 2008 or newer system can handle this (or Win 7) natively.

Johnny5 said...

Thats great news - hopefully! I have just a couple questions then I will drive in and get the files:

How do I determine the dates of the AVHD files, in order to maintain the correct order when merging them back in, when the dates have all been changed due to a data recovery?

What program in Windows 7 do I need to use in order to merge and test running this virtual machine?

Will a 64bit virtual machine OS run on a 32bit Windows 7?

I have only used HyperV on 2008 server, never Windows 7, for virtualization.

BrianEh said...

Each AVHD is a differencing disk. It knows the name and literal path location of its parent.

Hyper-V Manager includes the "inspect" disk feature - this exposes this information without modifying any disks.

In the case of recovery the path of the parent disk is different, but the name is the same. I think you can work it out from that.

DISKPART contains the features to manage virtual disks. Fixing, mounting, etc. You can use Disk MAnager to mount a virtual disk (without booting it) to copy files in and out.

In fact, if all you need is files out of hte virtual disks, just restore the virtual disks, fix the differencing disk chain, and then mount the end of hte differencing disk chain (the latest one or last one) and you will be readin ghte latest files.

If you want to boot the VM in the VHD then you need an engine. VPC is 32bit only, so you would need Hyper-V. The alternate is VirtualBox (since MSFT does tno have an answer for 64bit VMs on Win7)

Johnny5 said...

Thanks for your interaction on here. I am moving forward, I have the VHD file and the 3 AVHD files (still not sure which order they would go in, as the dates are all the same day due to the recovery) and I have a Windows 7 x32 PC with the space to run a merge. I downloaded VirtualBox and also HyperV manager (see here: http://technet.microsoft.com/en-us/library/ee256062%28WS.10%29.aspx) but neither one seem to be able to allow me to edit the disks. Can you help me a little further in explaining exactly how I would go about editing / merging the VHD/AVHD files on Windows 7? Thanks so much!

Crab said...

I have this same issue, but stupid microsoft have filtered out the ability to select an avhd file in the edit disk wizard. I have renamed the avhd file to vhd but the wizard does not allow it to be selected .So I do not get presented with the merge option.
Anyone know a way to get the disk wizard to pick up the renamed avhd file ?

BrianEh said...

With Hyper-V R2 you can directly select a .avhd file. With Hyper-V v1 you can only select a .vhd file.

Renaming the .avhd to .vhd does work, I have never seen a case where it does not. Unless the system is hiding the .avhe extension from you because "hide extensions of known file types" is selected.

Manual merge is only an option if: you have selected a differencing disk and the differeincg disk chain is not in use (the VM is not running)

BrianEh said...

I do not know the VirtualBox interface well enough to walk you through merging the differencing disk chain using it - however the process is the same as outlined in the main article.

You can also use DiskPart - and again, the overall process is the same, except it is command line.

Using DiskPart you mount the differencing disk, then you can change its properties. Not as simple as the GUI wizard.

Anonymous said...

Thanx for the tip. It really wiped the sweat off my brow.
Well done.
John, NIC, India

Anonymous said...

Thanks for this post. It was really helpful.

I tested the method but in different way and wonder what you guys think about it. This article is very popular and with lots of feedback but I didn't see any post mentioning the method/or similar one I used. So I'm curious.

I was able to use avhd file in "Edit Disk" wizard and avoid "Reconnect" step altogether. I tested this on Windows server 2008 R2 with Hyper-V

What I had was backup of VM's configuration and vhd files along with their directory structure.
My VM files were in default location when I backed them up.
Config. in C:\ProgramData\Microsoft\Windows\Hyper-v\Snapshots (I had two snapshots)and C:\ProgramData\Microsoft\Windows\Hyper-v\Virtula Machines folders.
VHDs in C:\Users\Public\Documents\Hyper-V\Virtual hard disks folder (1 vhd and 2 avhd files - say Jul11.avhd and Aug11.avhd)

Then I restored those files as follows:
Config: C:\RestoreTest\ ProgramData\Microsoft\Windows\Hyper-v\Snapshots ; C:\RestoreTest\ ProgramData\Microsoft\Windows\Hyper-v\Virtula Machines
VHDs: C:\RestoreTest\ Users\Public\Documents\Hyper-V\Virtual hard disks

1. I started "Edit Disk" wizard and selected the youngest avhd file (here Aug11.avhd) in page "Locate Virtual Hard Disk" page.
2. On the next page "Choose Action", I saw only two options: Compact and Merge.
I didn't see "Reconnect" option. I believe I didn't see "Reconnect" option because the parent file, Jul11.avhd, was in same folder.
(Using original method, when I renamed the avhd to vhd, I always saw "Reconnect" option).

3. On next page "Configure Disk" / "Merge Changes from Differencing Disk" I saw "To the parent virtual hard disk selected".
4. On next "Summary page"/ "Completing the Edit Virtual Hard Disk Wizard" I saw confirmation as follows:

Virtual Hard Disk: VistaBus_E59D01D4-F8F7-46DD-BE5A-F4B7F8AC9272.avhd (differencing) // This is Aug11.avhd
Action: Merge
Configuration: Merge into parent VistaBus_67841386-9207-4CEF-A8C7-B43447AE441F.avhd // This is Jul11.avhd

5. Merge completed successfully.

6. I repeated above step, with Jul11.avhd and parent VistaBus.vhd and merged the snapshot.
7. At the end I was left with one big VistaBus.vhd file in folder C:\RestoreTest\ Users\Public\Documents\Hyper-V\Virtual hard disks
8. I created a new VM with that VHD and it looked OK.

I would appreciate the comments.

BrianEh said...

If all that you did was create a new folder on the same machine the copies of your snapshot files (AVHDs, or differencing disks) will see the origional parent disk at the origional file location.

The only way that the parent path is broken is if the ORIGIONAL file locaiton is missing. You must restore to a different host for this to happen.

Otherwise you are merging into your origional disks, not your copy.

Anonymous said...

Thanks Brian for your quick reply.

It doesn't appear that way. The avhd files in the folders were gone.
Looks like if the parent files are in same folder it would pick that and not in original location.

I did another test as follows.

I copied all 3 files, VistaBus.vhd and two avhd files to a temp. folder on different server which is windows 2008 r2 hyper-v server but with sp1. Then I started "Edit Disk" wizard and selected Aug11.avhd file. This time it showed me two options for "Merge" and "Compact". (I tried this step again after renaming Jul11.avhd to a different name and then the wizard showed me "Reconnect" option.) The summary page showed me the correct file names for child and parent (only path name. If summary page would show full path name, that would have been better!) avhd files. I was able to successfully merged two avhd files. Now the Jul11.avhd is bigger in size that prior to merge.

I saw similar behavior when I tried to mount/attach vhd in disk manager on windows 7 or 2008. I can mount an "avhd" as along as all ancestor files are in same folder. If any of the parent files is renamed or moved to different folder, disk manager would complain that chain is broken.

BrianEh said...

If you copy to a different server and the disk letter and the paths aremain the same, then the chain is not broken and will not need to be repaired.

Repair needs to happen when the path cannot be identified at all.
This is because it is a hard path ( ie. c:\folder\file.vhd ) not a soft path ( ie. %programfiles%\folder\file.vhd )

All this being said, there are some default behaviors that have been coded around this to handle things in a better way and mike the lives of folks easier.

In the end. Thanks for sharing!

Anonymous said...

Brian,

Thanks again.

I copied files to a different drive on another Hyper-v server. The files were in default folder "C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks" on the server where the VM was running. I copied only 3 files to E:\Test\ folder on another hyper-v server. Folder structure was completely different.

I wonder what algorithm Microsoft uses to locate parent files in the chain. Do you know of any documents about this.

BrianEh said...

The VHD format is well documented (the current implementation of it isn't).

AVHD files are simply special differencing disks. It is the differencing disk that you would look at in the VHD specification.

You can search for it at MSDN.

Zahid Siddique said...

i want to do manual merge but during reconnect phase i noticed that one of the avhd has 0 KB of size and i am unable to complete the merge process
is there any possible solution ?

BrianEh said...

An AVHD of "0" size generally results from a snapshot that was taken but the VM was never powered on, and thus nothing was ever written into the AVHD.

It can be due to two snapshots being taken while a VM is off, it can be the "now" state of a VM if it is powered off and the snapshot was taken.

By default this last AVHD should be locked (and owned) by the system. Especially if it is the "now".

Also, if it is "0" in size then there is nothing to merge.

Zahid Siddique said...

actually the "0" KB AVHD is not the latest file in my case this is oldest file in the chain.

even when i try to merge them manually this is the last file in the chain which Hyper-V is looking for and when i select this file it gives an error and the merge is not completed.

is there any way that i can skip this "0" KB file and complete the merge so that i will be able to recover my data which is very very critical.

BrianEh said...

You can always skip any differencing disk / AVHD in the chain. Doing so simply does not apply the data within that disk.

In your case the AVHD is 0k so it should be able to be skipped safely.

I know of a couple models that make two differencing disks off the root VHD. There are a couple reasons that they do this. But it is mostly to form a distinct and repalceable seperation between the running state and the base VHD.


I have seen it done in VDI deployments and I also see it done for storage migrations.

Beyond that, I have only seen it when someone quickly makes two snapshots.

SteveMontec said...

I deleted a Snapshot and lost one of my Virtual machines. I restarted the server hoping all would come back with no luck. I created a new Virtual machine using my old VHD which restores the server but only back to the Snapshot(AVHD) date which was a few days after I created the server which was 2 months ago. Now that I have started the VHD the date is different, anyone know how I can merge the AVHD and the VHD so as to have a Virtual Machine with current data.

Zahid Siddique said...

i am always getting
connect to parent virtual disk screen where it is looking for the VHD file which is "0" KB and there is no option to skip

BrianEh said...

Steve: How did you delete your snapshot? If you reverted you will toss away your "now" state. If you applied, you will also toss away your "now" state (unless you heded the dialog to create a snapshot before applying - revert does not give this warning).

If you did either of these actions, then your AVHD is gone. It has been deleted by the system.

If you simply deleted the snapshot, your machine should be intact.

That is why I am curious as to how you did the delete.

Zahid: Is the 0k disk the actual root parent VHD? Not a member of the differencing disk /snapshot chain - but possibly an empty VHD?

As in the scenario where the VM was created with a VHD and then immediately snapshotted?

You can try creating a copy of everything and then creating a new VHD of the exact same size and using that as the root and merge it in.

If you know what happened in the life of the machine then you could re-create it.

That is my best guess so far.

BADBOY said...

On my case
1. Deleted first snapshot on tree.

2. The VM dissapear after shutdown.

3. After reading you post, i've actually started a new VM pointing to Virtual Hard Disk .vhd. It won't boot. I've used a Win 7 Boot CD and checked that the exchange DB was 2 months ago.

4. Found your post.

5. Copied .avhd files and vhd with subsequent dates to External USB

6. On server: I've renamed avhd looking for date stamp order:
file4.vhd (vhd disk)
file3.vhd
file2.vhd
file1.vhd (youngest)

7. Run Disk Editor selected file1.vhd and reconnect to file2 (i've renamed all files so i've to browse to find file2.vhd)

8. Run Disk Editor again selected file1.vhd and select Combine. File1.vhd dissapeared

9. Run again the two steps 7 and 8 with file3.vhd with the file2.vhd and the file3.vhd with the original file4.vhd

10. Boot the VM, reconnect NICs, assigned old ip address, etc.

Great Job, I posted your experience on my personal blog.

Thanks

Fabio Gamboa said...

I found another way to fix that. In my case there was no snapshots and the some VMs had differencing hard disks (VHD and AVHD)files.
The solution was to:

- Stop the VM
- Create a Snapshot
- Delete the snapshot

Other way to fix it was to export the VM

BrianEh said...

So, if you have manually merged your snapshots and the system is still trying to merge them in the background (you get events that the merge is failing).

You will most likely need to manually edit the configuraiton file of the VM.

You will need to find its XML configuration file, its name will be a GUID.

Power off the VM. Open up the XML file and find the disk_merge_pending section and set the vaule to "False".

This should stop the background merge from attempting to happen.

Anonymous said...

On Oct 13/14 2011 I posted some comments/questions about how Hyper-V figure out the location of parent file of a snapshot AVHD file. I think I found the answer to the question.
The location of parent file of a differencing disk is inside the VHD dynamic header.
VHD Specifications(http://www.microsoft.com/download/en/details.aspx?DisplayLang=en&id=23850)
The Dynamice header has "Parent Unicode Name" and "Parent Locator Entry" (8 of them) field. On a windows 2008 r2 Hyper-V, I found two entires (out of 8) as follows.
C:\VMs\Win2K3Ent64\1GBFixedRoot.vhd
.\1GBFixedRoot.vhd

This tells me that, Hyper-V search both the paths to find the parent file. I think it would then also match "Parent Unique ID" to verify.

Let me know if my assumtion is not true.

BrianEh said...

AVHD / Snapshot files are no more than Differencing disks, so yes. In their header is the LiteralPath to the parent.

This is what the Inspect Disk utility of Hyper-V is using to determine its parent.

Does Hyper-V Srever also use this? I can't asner that one. As this information is also recorded in the XML configuration files of each snapshot - you will find the parent information.

I assume that different places int he code use these two pieces of information in differnet ways.

The merge process is built upon Windows Server and I assume uses the built-in differencing disk merge function which relies on the path in the header of hte differencing disk.

The disk fixup and Import funcitons (and others) I assume rely on the XML configuration files. At least this is what my experience as a software tester is leading me to believe and expect has been implemented.

I could be surprised, but I don't think so ;-)

DavidNg said...

Hi ,
I really need help , no space on drive c:
I don't have additional hard disk.
I don't have space on C:
I have space on f:

If I do copy exchangeserver vhd and avhd to another location and merge it there

Do I also have to do the exchangedata and exchangelog vhd and avhd as well ??.

Do I have to delete snapshots , how and when please ..?

Current disk space status is :-

Exchangeserver vhd 30GB , avhd 50GB on drive C: (no more space only 9GB left)

Exchangedate vhd 68GB , avhd 58GB on drive f: (1TB space left)

Exchangelog chd 85GB , avhd 16GB on drive f: (1 TB space left)

If I do the export vm way , how long to export please ... ?

How long to import again please ... ?

Please help ..

Thanks

BrianEh said...

As I have told many other folks - if you have the storage available:

Power off the VM, take a new snapshot, select this snapshot, export this snapshot, rename the first VM, then import the exported VM. power it up, verify it.

You get a second VM that is named the same as your origional. It will have flat VHDs (no AVHDs) and will contain all your data.

It sounds as if you have the storage on F: to do this. F: would be your export location.

If all your data is on VHDs, then it is really simple.

The second effect that this has is that you will be totally off C: and will not run into this problem again.
And you don't have to mess around with attempting to manually merge.

As to how long will it take. I cannot predict. It is all about the speed of your storage.

Doing the snapshot Export takes the time, the Import takes no time as the VM is mounted and runs from the folder it was exported to.

Dv said...

Great article. Saved me loads of time. Thanks again

ASHWINPAWAR said...

Hi Brian,

I must say it's a very popular thread.

I have the similar situation. In the last message, you mentioned about 'exporting'. Would that allow me to expand the disk size once it is single VHD. Is that the key factor in expnading the disk that once you have a single disk, it's easier.

Do you suggest - Once I import the VM, I shutdown the VM, and then click edit disk and expand it?

Thanks,
-Ashwin

BrianEh said...

The key is that you have a single VHD.
If you are running Hyper-V Server 2008 R2 or later - you can select and export a single snapshot.

The act of exporting a snapshot (instead of the entire machine) is that it gives you a single VHD.

Once you have the single VHD, you can safely modify its size.

http://itproctology.blogspot.com/2009/04/never-resize-vhd-with-snapshots-or.html

ASHWINPAWAR said...

Thank you Brian.

Another query here - I have a HYPER-V Full backup of this VM [Application consistent], if in case the VM is screwed up for some reason, Can I not just restore the VM back ? Just from the backup perspective. I read in other blogs, that you need to copy out the VHD before you do any such tasks. I guess if you have proper backup, then you are covered.

BrianEh said...

If you have a tested / verified back up, yes. Of course that is a way out of trouble.

But as always, make sure you can restore it, and that it is backing up the correct state first.

I have seen many folks over the years panic in the sudden realization that they totally broke something, and they never verified their backups to ensure that 1) it was 'right' and 2) that it was recoverable.

Personally, don't manually merge unless it is absolutely necessary. Delete the snapshots, be patient, power off the VM is pre-Hyper-V 2012, wait.

ASHWINPAWAR said...

Thanks, Brian. Absolutely true. I agree, without a regular testing, even backup remains a risk.

As per your advise - I have two options here:
1. Take a snapshot - Export the VHD, rename the old VM, create a new VM with original name and Import the VHD and then expand it.

or,

2. Delete snapshots, power-off VM and wait. Then, re-size the VHD.

3. Combination of both - Take the new snapshot and export it, [step2] then delete the snapshots and power-off VM and wait.

If something goes wrong, follow the step 1.

Does that make sense.

Thanks,
-Ashwin

BrianEh said...

1. Take a snapshot, select it, export it.

In the Exported destination folder, resize the VHD, copy the folder to your run location, import, boot, resize within the OS of the VM, move forward.
(Hyper-V 2008 R2 or later)

2. Delete existing snapshots. Follow merge procedure for Hyper-V version (if prior to 2012, power off VM).
- you can export an additional copy for safety.

#2 will take at least twice as long as #1, depending on the speed of the merge.

But, if you want a safe Export to go back to - do that prior to anything else. That will always put you back to where you are now.

ASHWINPAWAR said...

Thank you Brian for all the help. You are doing a great job. I appreciate it. Keep sharing these job saving article :)

Good luck!

Cheers!
-Ashwin

umer said...

I have a VM with two snapshots on Hyper V 2008 R2. Mistakenly, during the migration to Hyper V 2012, i added a new VM and attached the parent VHD and booted. Now merging is not working and i could not see any new data in merged VHD. It seems merge is not working. Any idea how to merge now ?

umer said...

i have a VM with with 2 snapshots in Hyper-v 2008R2,after migration in Hyper V 2012R2, mistakenly i created a new VM and attached parent hard disk to the the VM.
After login in the VM i realize that mine new data was missing.So i merge the snapshot(.AVHD) in the parent(base)VHD but after attaching this merged VHD Windows is not booting.
Moreover i also mount this VHD in other system but i could not see my latest data.

Seems merging is not successfully.
Please help me what to do now as i have very important data.

umer said...

i have a VM with with 2 snapshots in Hyper-v 2008R2,after migration in Hyper V 2012R2, mistakenly i created a new VM and attached parent hard disk to the the VM.
After login in the VM i realize that mine new data was missing.So i merge the snapshot(.AVHD) in the parent(base)VHD but after attaching this merged VHD Windows is not booting.
Moreover i also mount this VHD in other system but i could not see my latest data.

Seems merging is not successfully.
Please help me what to do now as i have very important data.

Abubakar said...

Hi , Excellent piece of information. This had helped in merging the VHD and i have deleted the snapshot subtree now.

I have even compacted the VHD which had saved around 20 GB space on the VHD file.

Great article.

Many thanks for your help

Clarkavelli said...

You saved my day when we had to recreate a VM that someone must have inadvertently took a snapshot of. Can't thank you enough!

iamnomad said...

Thank you very much for posting this! This trick works in Virtual PC as well (but the "undo disks" have the extension VUD instead). My computer had crashed while I was running it and somehow wiped out the VM settings file. After recreating the file, it didn't keep any of my changes. I was afraid that I had lost days of work, but luckily this post saved me from having to recreate all the changes I had made.

Ray Gross said...

Hi Brian,

You wrote - "I can't tell you how many times I see folks in the forums that create a snapshot, then delete it (poof - it is gone from the UI) and they never power it off - the end result is that they have been running on a differencing disk for months and were clueless about it (most frequently, some co-worker did it)."

This is the exact situation I'm in. The snapshot was deleted (a long time ago) and now the AVHD file is 95GB and the VHD is only 17 GB. The problem I have is there is only 20 GB of free space on the volume.

Now I am afraid to shutdown the Hyper-V for two reason - 1. the amount of time to merge the files 2. there is only 20 GB of free space

Any suggestions??

BrianEh said...

@Ray -
The merge process built into the system is designed to ensure that you don't have data loss. Giving that process time to happen is important (vs. trying to somehow go around it) if possible.

If you have no additional storage your options are truly limited. But I still cannot stress enough, allowing time to pass. Gaining enough time windows that you can get everything to safely happen.

There are a few tricks that can also be done, but they all require some type of additional storage / space.
For example, if your source VM is pretty static. Then you can attach a differencing disk to the snapshot and read through as a new VM. I know I did not describe all of the details in that statement. Just planting the idea.

Ray Gross said...

Thanks for the quick reply.

In your experience, will I need the combined size of the AVHD and VHD of free space for the merger to complete? Also, any idea how long a merger of a 95GB AVHD and 17GB VHD would take? Are we talking hours or days??

BrianEh said...

For it to complete? No, it will complete even if space constrained. It does make it take a bit longer, but it will complete.

The optimal is the two disks +50%. But this is not an optimal situation.

And there is no way to predict. It all depends on how may blocks have been changed. It could be lots of open space, I don't know.
Occupied blocks in a differencing disk (causing it to expand) are not always change. They could simply be due to paging out by some application.