Troubleshooting Windows Server VSS issues without rebooting

After being told by our backup crew that I'd probably have to reboot any Windows servers with VSS issues I had a look through all the information I could find.
Alright, I should have read about this in 2003 odd, but it's not really my field of IT expertise.

Having collated and written up some notes for work it seemed a shame to not post them here.

Contents

Volume Shadow copy Service component overview
Overiew of VSS troubleshooting
Resolve
Retry the operation and restart the writer if necessary
Retry the operation and restart the writer
Open Event Viewer and view events related to VSS
Verify
Resetting VSS Clients/Requestors
Resetting VSS Writers
Another random page
Deleting Shadows
Resetting Providers
Microsoft Software Shadow Copy Provider
Tools
VSSAdmin
VSSUIRUN
Powershell Get-VSSWriters
Other reading

Volume Shadow copy Service component overview

Writers: VSS aware add-ons that handle writing consistent state to shadow copies. Reset by restarting the relevant service.

Providers: Interface between VSS and disk writes, creating shadow copies and handling consistency.

Requestors: Software that makes requests for shadow copies. Usually backup software. Reset by stopping relevant program/service (eg. Networker client).

Shadows: Chunks of disk devoted to storing shadow copy data. If using the Windows software provider these will be copies of data that was modified after the shadow was taken.

For a full overview of VSS see: https://technet.microsoft.com/en-us/library/ee923636(v=ws.10).aspx

Overiew of VSS troubleshooting

From: https://technet.microsoft.com/en-us/library/ee264212(WS.10).aspx 20170419

Resolve

Retry the operation and restart the writer if necessary

This event indicates a problem with a Volume Shadow Copy Service (VSS) writer.

To perform this procedure, you must have membership in Administrators, or you must have been delegated the appropriate authority.

Retry the operation and restart the writer

To retry the operation and restart the writer:

Retry the backup or restore operation that caused the error condition.
If this does not resolve the issue, restart the service or process that hosts the writer, and retry the operation.
If this does not resolve the issue, open Event Viewer as described in the "Open Event Viewer and view events related to VSS" section and look for events related to the service or process that hosts the writer. If necessary, restart the service or process, and retry the operation.
If this does not resolve the issue, restart the computer, and retry the operation.
If restarting the computer does not resolve the issue, provide the Event Viewer information to the vendor whose application is indicated in the event text.

Open Event Viewer and view events related to VSS

To open Event Viewer and view events related to VSS:

Click Start, click Run, type eventvwr.msc, and then click OK. If the User Account Control dialog box appears, ensure that the action it displays is what you want, and then click Continue.
In Event Viewer, expand Windows Logs, and then click Application.
To filter the events so that only events with a Source of VSS are shown, in the Actions pane, click Filter Current Log. On the Filter tab, in the Event sources drop-down list, select the checkbox for VSS. Select other options as appropriate, and then click OK.
To sort the displayed events by date and time, in the center pane, click the Date and Time heading.

Verify

To verify that the Volume Shadow Copy Service (VSS) writers are operating correctly, perform the following procedure, and then retry the previous VSS operation.

To perform this procedure, you must have membership in Administrators, or you must have been delegated the appropriate authority.

To verify that the Volume Shadow Copy Service (VSS) writers are operating correctly:

Open an elevated Command Prompt window. Click Start, click All Programs, click Accessories, right-click Command Prompt, and then click Run as administrator.
Type vssadmin list writers, and then press Enter.
Ensure that all writers are in the Stable state and are operating without errors.

For more information about the vssadmin command, see http://go.microsoft.com/fwlink/?LinkId=103401.

Resetting VSS Clients/Requestors

Simply restart whatever client software is using VSS. This may be EMC “Networker Remote Exec Service”, Windows backup or other backup software.

Resetting VSS Writers

Resetting writers is done by restarting whatever service they’re part of.

The table below is from: http://www.planetcobalt.net/sdb/vss_writers.shtml 20170419,

The author updates the list regularly, so check it out for updates and some other tips on VSS writers.

VSS Writer	Service Name	Service Display Name
ADAM (<instance_name>) Writer	ADAM_<instance_name>	<instance_name>
ASR Writer	VSS	Volume Shadow Copy
BITS Writer	BITS	Background Intelligent Transfer Service
Certificate Authority	CertSvc	Active Directory Certificate Services
COM+ REGDB Writer	VSS	Volume Shadow Copy
Dedup Writer	ddpvssvc	Data Deduplication Volume Shadow Copy Service
DFS Replication service writer	DFSR	DFS Replication
DHCP Jet Writer	DHCPServer	DHCP Server
FRS Writer	NtFrs	File Replication
FSRM writer	srmsvc	File Server Resource Manager
IIS Config Writer	AppHostSvc	Application Host Helper Service
IIS Metabase Writer	IISADMIN	IIS Admin Service
Microsoft Exchange Replica Writer	MSExchangeRepl	Microsoft Exchange Replication Service
Microsoft Exchange Writer	MSExchangeIS	Microsoft Exchange Information Store
Microsoft Hyper-V VSS Writer	vmms	Hyper-V Virtual Machine Management
MSMQ Writer (MSMQ)	MSMQ	Message Queuing
MSSearch Service Writer	WSearch	Windows Search
NPS VSS Writer	EventSystem	COM+ Event System
NTDS	NTDS	Active Directory Domain Services
OSearch VSS Writer	OSearch	Office SharePoint Server Search
OSearch14 VSS Writer	OSearch14	SharePoint Server Search 14
OSearch15 VSS Writer	OSearch15	SharePoint Server Search 15
Registry Writer	VSS	Volume Shadow Copy
Shadow Copy Optimization Writer	VSS	Volume Shadow Copy
SharePoint Services Writer	SPWriter	Windows SharePoint Services VSS Writer
SMS Writer	SMS_SITE_VSS_WRITER	SMS_SITE_VSS_WRITER
SPSearch VSS Writer	SPSearch	Windows SharePoint Services Search
SPSearch4 VSS Writer	SPSearch4	SharePoint Foundation Search V4
SqlServerWriter	SQLWriter	SQL Server VSS Writer
System Writer	CryptSvc	Cryptographic Services
TermServLicensing	TermServLicensing	Remote Desktop Licensing
WDS VSS Writer	WDSServer	Windows Deployment Services Server
WIDWriter	WIDWriter	Windows Internal Database VSS Writer
WINS Jet Writer	WINS	Windows Internet Name Service (WINS)
WMI Writer	Winmgmt	Windows Management Instrumentation

Another page

Note: The following may also be worth a look – but re-registering all DLLs might be a bigger hammer than just restarting the server.

https://support.unitrends.com/UnitrendsBackup/s/article/ka0400000008ZiUAAU/000004085

Deleting Shadows

Remove excessive or corrupt shadows with VSSADMIN. If the shadows are created by dodgy backup software or backups being interrupted that is probably fine.

If they are being created on a schedule and there is no evidence that they are bad (and I’m not sure how to gather such evidence) try other things first.

For help : Vssadmin delete shadows /?

Example - To delete the oldest shadow on D:

vssadmin Delete Shadows /For=D: /Oldest

Resetting Providers

Providers are plug in DLLs that interact with writes to the filesystem to support VSS.

Providers can be listed using “vssadmin list providers”

For an overview of different providers that may be available see: https://technet.microsoft.com/en-us/library/ee923636(v=ws.10).aspx

I wouldn't advise messing around with providers, but (cowboy!) circumstances may warrant more than just restarting a provider service.

Microsoft Software Shadow Copy Provider

The default provider is the Microsoft Software Shadow Copy Provider. As it’s supposed to be doing a copy-on-write of the original blocks to the shadow there should be no data loss if stopping the service. Untested!!

A colleague noted:

Sometimes a Net Stop command is required to stop a service if it is hung in a stopping or starting state, but that has failed to stop the service before. The solution was to find its process and kill the process to stop the service and then start the service again.
This happened to the Microsoft shadow copy provider* service where the following command was used:
c:\tasklist /svc | findstr swprv
Once the process and its id was identified, the process was killed, the service had to be deleted.
sc delete swprv
The service then had to be created again
regsvr32 /i swprv.dll

The PID of the service (if running) can also be found using “SC QUERYEX swprv” if tasklist.exe isn’t cooperative.

c:\Windows\System32>sc queryex swprv

SERVICE_NAME: swprv

TYPE : 10 WIN32_OWN_PROCESS

STATE : 1 STOPPED

WIN32_EXIT_CODE : 0 (0x0)

SERVICE_EXIT_CODE : 0 (0x0)

CHECKPOINT : 0x0

WAIT_HINT : 0x0

PID : < PID if process is running >

FLAGS :

Check that SWPRV is running in its own process before killing the PID. This is visible in the “Services” tab of Task Manager or in the output of “SC QUERYEX swprv” (example above).

If that does not remedy VSS a reboot is probably in order.

In worst case, the following MS community post may be useful (re-registering a pile of windows DLLs and VSSVC):

https://answers.microsoft.com/en-us/windows/forum/windows_xp-update/volume-shadow-copy-service-errors/1822c1ee-b652-4ae9-af55-adb1443c3a32

Note: This is from 2010, may not apply to versions of Windows that are more recent.

Tools

VSSAdmin

The standard VSS tool on servers since 2003.

Can view writer state, providers and manage shadows.

For help: Vssadmin /?

VSSUIRUN

A GUI tool on Server 2008 and beyond! Not documented.

Powershell Get-VSSWriters

Powershell to pull “vssadmin list writers” into powershell. Works on remote servers via WinRM.

https://gallery.technet.microsoft.com/scriptcenter/Powershell-ScriptFunction-415e9e70

Search This Blog

Glorious Views