Purpose

This purpose of this Troubleshooting Guide is to provide you with techniques, best practices, and “quick fixes” to resolve the most frequently encountered Replibit questions and/or issues.  

This Guide is not intended to be a comprehensive, step-by-step manual, but rather a troubleshooting resource for common Replibit functions. For the full Replibit User manual on how to install, deploy, and manage Replibit’s Appliance software (including screenshots), please access the Replibit Administration Guide.

The article will cover the following topics:

• A. How to download your initial Replibit ISO Software
• B. How to Troubleshoot Failed Backup Jobs using the Replibit Backup Analysis Tool
• C. How to Troubleshoot Replibit VSS Issues 
• D. How to Troubleshoot your CPU Usage
• E. How to Troubleshoot the Appliance not communicating with the Replibit Licensing Portal 
• F. Appliance Migration Process
• G. How to Replace a Failed Disk in the Backup Storage Pool 

Troubleshooting Common Replibit Installation Functions

A. How to download your initial Replibit ISO Software

To install Replibit, you will need to download the Replibit ISO file and burn the ISO to a bootable flash drive.

1. Before you begin your Replibit Software download, note that Replibit may automatically start the installation without any prompts when using certain hardware; therefore, please use a machine that you do not need to preserve any data on, as Replibit may delete your computer’s contents during the install process.
2. Download the latest version of the Replibit Software using the link below: (select your download link under, “Downloads” from the Replibit page, agree to the “License Agreement” Terms and Conditions, and Open/Save):

https://axcient.com/replibit-downloads/

Note: This downloading process may take up to five minutes (the typical download is approximately 1.5 GB); so please check your “View Downloads” – internet explorer pop-up window for a status if needed.

For complete step by step instructions on:

• Determining Replibit Appliance Requirements and Optimal Hardware Specifications before you begin,
• Downloading the Linux Live USB Creator,
• Burning the ISO to the USB stick to create a Bootable Flash Drive,
• Installing the USB flashdrive on your computer,
• Preparing your system for installing the Replibit Appliance (Prepping your BIOS),
• Installing the Replibit OS,
• Configuring your Device as an Appliance,
• Performing Mandatory Registration in the Replibit Licensing Portal, and
• Adding Clients or Users in the Reseller’s Licensing Portal.

 B.  How to Troubleshoot Failed Backup Jobs using the Replibit Backup Analysis Tool

Once you have installed Replibit, it is highly recommended to check your backups. Test your backups by:

• mounting,
• booting as a VM,
• exporting as a virtual disk, and as a Bare Metal Restore. 

if you experience any failed backup jobs, please troubleshoot using the following procedures:

1. Download the latest copy of the Replibit Backup Analysis Tool from this KB article:

 The Replibit Backup Analysis Tool will analyze the health of a Protected System, capture logs, and creates a text file with a synopsis of any issues it finds. 

Notes:

• You will need Administrator Privileges to run the Replibit Backup Analysis Tool.
• You will need to run the tool on the Protected System from an elevated command prompt after stopping the Replibit service.
• The tool will create a .zip file in the C:\ReplibitAnalysisTool\Replibit folder along with all other files.
• Please send Axcient only the .zip file when directed.
• When you run the tool, please click the Enter button when prompted to keep the data local.

After the tool is finished, you can open up the text file it created (problems.txt file) and start troubleshooting the items it detected.  

Some of the Replibit Backup Analysis Tool integrity tests include: 

• Ensuring that Replibit can traverse the directory structure,
• Numerating out the directories,
• Reading-down the Master File Table and looking for orphan objects,
• Checking for a bad pointer within the Master File Table,
• Checking for high fragmentation levels (which affects virtualization from Windows),
• Checking for any DLL mismatches,
• Running a real-time tracking mechanism that creates a hash file,
• Running a full differential in the last backup day (automatically) to compare every block in the backup set to that of the production set,
• Real-time resilvering of data,
• Tracking Window Boot DLL files’ expectation of virtualization (proper modifications to Boot DLL files greatly impacts stability and performance on Virtualization).

If any piece of the data set exceeds the tolerances, Replibit will proactively fail the backup so you can troubleshoot.

C. How to Troubleshoot Replibit VSS Issues

When a Replibit backup fails, the problem is almost always caused by the failure of the Microsoft VSS provider, or one of the application-specific VSS writers, and this article above will help you to diagnose these failures.

Typically, the Top 10 reasons for Backup Failures associated with VSS include:

1. Snapshot creation failed due to Windows VSS failure.
2. There is not enough Shadow Copy storage space to create a backup.
3. A VSS writer is in a failed state.
4. A volume does not pass a CheckDisk (chkdsk).
5. The Master File Table is highly fragmented.
6. A volume is extremely fragmented.
7. Another application interrupted the snapshot creation.
8. An uninstalled backup application left behind its VSS provider.
9. The OS does not pass a System File Checker (SFC) scan.
10. A Windows Server Backup disk is online.

 D. How to Troubleshoot your CPU usage

If you are experiencing performance issues, please follow these steps:

1. Look at the system resources.

1. CPU and RAM usage are easy to monitor by opening Task Manager.
2. Additionally, the disk sub-system can greatly affect VSS performance.

 2. Monitor the CPU before and after the backup to set a baseline and to observe the change.

1. If the system is running an old or low-end CPU, you may already have a high CPU usage baseline, and running another service will exceed your CPU’s capacity.

2. If this is the case, please adjust the ‘Worker Threads’ or set the ‘Affinity’ to increase the backup duration using the following Knowledgebase article; the Replibit agent defaults to backup all volumes and uses 4 worker threads per processor core. The KB article below demonstrates how to specify which volumes are backups and how many worker threads are utilized.

Modifying the Replibit Agent - Set Volumes and Worker Threads - aristos.cfg or

Notes: Lowering the amount of worker threads will reduce the speed of the backup. If the backups start overlapping, you may need to increase the time between snapshots.

 As with the CPU, monitor the RAM and pagefile usage prior to, and during, a backup.

3. 1. If the Protected System is already using all of the available RAM, the VSS service will end up using the slower pagefile.
   2. Adding more RAM is the best option but you can also look at reducing the amount of RAM other services are using.
   3. Exchange and SQL by default, will use all available RAM.
   4. A quick web search will guide you on setting the maximum RAM that different services can use.
   5. Run chkdsk and then defrag all the partitions being backed up.
   6. Remember that VSS needs free disk space to create the ShadowCopies.

If changing worker threads to as low as [1] still does not work, you can try changing the affinity on aristosagent to lower CPU usage while a backup is running by executing these two commands from a raised CMD or .BAT file:

wmic process where name="AristosAgent.exe" CALL setpriority "below normal" PowerShell "$Process = Get-Process AristosAgent; $Process.ProcessorAffinity=1"

Note: As guidance, you can create a .BAT file with the two commands, then create a recurring scheduled task to run the .BAT file during or after a scheduled backup kicks off; this way you will know that AristosAgent.exe will be running.

   E.   How to Troubleshoot the Appliance not communicating with the Replibit Licensing Portal

1. When you set up your Replibit Appliance for the first time, you will need to logon with a Replibit Reseller Account so you can choose the Customer and Location.
2.  If you are having trouble connecting to: licesing.replibit.com, please follow the troubleshooting steps below:
   1. The URL should not contain http:// or https://
   2. Change the DNS setting to 8.8.8.8; try to log on again.
   3. Verify that outbound traffic is not being blocked.

  

F.   Appliance Migration Process

 Note: Perform each Protected System one at a time. DO NOT ATTEMPT TO DO ALL AT ONCE.

1. Stop and disable the Replibit service on the Protected System.
2. Disable replication of the Protected System on the original appliance.
3. Add 20 days to the Retention Policy on both the appliance and the vault so data will not be deleted during this process.
4. Migrate the Protected System via USB or Network in the Details page of that Protected System on the appliance.
5. Once the migration is complete, verify on the new Appliance that the data is there and is recoverable. Additionally, boot in Test
6. Change the IP address of the appliance in cfg located in c:\program files (x86)\replibit to the new appliance's IP.
7. Enable the Replibit service on the Protected System.
8. After several backups have run, repeat the verification process by ensuring that the data is present and is recoverable.
9. Enable Replication on the new Appliance to the Vault.
10. After the replication is back in sync, remove the 20 days on the retention policy.

G. How to Replace a Failed Disk in the Backup Storage Pool

 If your pool has lost a disk but the RAID is still intact (you have not lost two disks in a RAID5 or three disks in a RAID6), your pool will be in a degraded state.

1. Use the Storage Pool drop-down to find the Failed Disk(s) In this section you should be able to see your failed disk. 
2. After physically replacing the failed disk:
   1. Recreate the disk as a RAID0 (if the RAID controller is not passing the disks as JBOD) with a disk of equal or greater size. In this scenario you will need to reboot.
   2. Once complete, use the Storage Pool drop-down to find the Unused Disk(s) In this section, you should find an unused disk.
3. Return to the Failed Disk(s) section, and click Replace next to the failed disk. Select the correct Unused Disk and click
4. You will see a popup notifying you that the disk is being replaced.
5. While the disk is being replaced, you will see the resilvering status under Storage Status.
6. When the resilvering process is complete, the pool will return to an Online Status.