Service Advisory for 9.0.0 Release

Release Overview

x360Recover 9.0.0 is a Feature Release. The primary focus of this release is to prepare for the upcoming Ubuntu Xenial operating system upgrade.

Summary of New Features and Enhancements

• Multi-Factor Authentication support on Management Portal
• Management Portal centralized email alerting and ticketing
• Default retention policy changed to tiered model
• Pre-fetch of Xenial packages needed for upcoming OS upgrade
• OS Recovery Wizard now part of Provisioning Wizard
• AutoVerify improvements to reduce some failure cases
• Hardened OS via removal of unused packages
• Customer name added to email subject when delivering GMP reports

Summary of Bug Fixes 

• Fixed issue making it impossible to replace a failed disk in the same bus slot
• Fixed Installer ISO crash when x360Recover partitions already exist
• Fixed issue with re-enabling replication after a migration if the original base snapshot has been removed by retention cleanup
• Fixed invalid Reporting alerts when vault has been converted to Scale-Out Cloud
• Fixed Backup Status Check on Appliance Failed if volume has 64K+ cluster size
• USB Recovery progress is now displayed properly on vault
• AutoVerify Full Backup alert is now closed properly after full scan is completed
• Fixed issues preventing the use of ICANN-era generic top level domains in email configurations
• Fixed logos for newer Microsoft Operating systems displayed on Protected Systems page
• Fixed issue with replication progress being displayed inaccurately in some edge cases
• Fixed issues with replication jobs hanging or not starting when scheduled
• Fixed issues with incorrect timezone calculation in GMP reporting
• Fixed Billing reports not calculated for all customer locations

Management Portal ticketing

The Reporting Engine on the Management Portal is an enhancement that simplifies the daily validation and management of your expanding fleet of BDR devices and protected systems.  Since the release in version 6.0.0,  there has been a single-pane-of-glass view into the overall health and status of all managed devices and systems. 

Some of our partners have indicated that they prefer to manage and monitor their devices on a ticketing basis.  Release 9.0.0 delivers new enhanced functionality allowing the Report Engine to create email alerts based upon all of the scored metrics present within Reporting.

To enable this feature from the Settings -> Reporting page, click the check box to enable ticketing.  Fill in the POP server address information corresponding to the existing SMTP mailbox used for report delivery, and the email address to which alerts should be sent and save.

Report Engine ticketing is designed to be generically PSA friendly, rather than customized to any particular ticketing platform.  When an alert email message is sent, the backend also performs a POP mail pickup looking for any message replies from the ticketing system.  When replies are found, the subject line of the reply is stored and copied into any follow-up emails sent regarding the open alert.  In this way, most ticketing systems will be able to recognize responses and include the updates into the original ticket, rather than generate new disconnected tickets with every email.

When the alert condition has cleared, a final ‘close’ event message is sent.  Ticketing systems that allow for configurable automation (Autotask, for example) can be configured to look for keywords in the closing message and automatically close or otherwise change status of tickets, providing a generic round-trip ticket functionality.

Management Portal ticketing and email / ConnectWise ticketing on your managed devices are both complementary and overlapping.  Generally, Agent Status, Disk Usage, Success/Failed/Missed Backups, Pool Status, and Replication class ticketing can be disabled on your managed devices after enabling ticketing from the Management Portal. 

Additionally, you gain the ability to be alerted when your managed devices are offline, as this is monitored by the Report Engine on the Management Portal.

Virtually every metric monitored by Reporting has a configurable threshold that can be set within the backend.  You may contact the Axcient Support team for assistance in tuning these alerting metrics if the default behavior does not suite your SLA requirements or operating preferences.

Preparation for Ubuntu Xenial OS Upgrade

The next release of x360Recover will perform an upgrade of your devices from Ubuntu Trusty (14.04 LTS) to Ubuntu Xenial (16.04 LTS.) 

This release contains enhancements, features, bug fixes, and hardening to prepare for the upcoming upgrade.  Among the many changes in this release, a pre-fetch mechanism has been added that will download and pre-stage all the packages necessary to perform the OS upgrade.  By pre-staging these packages, we will significantly reduce the total time required to perform the upgrade and be able to monitor via telemetry when the majority of devices are ready to begin the upgrade process.

To alleviate this issue with the Ubuntu Xenial (16.04 LTS) and future OS upgrades, Axcient has created a dedicated Ubuntu software repository mirror.  Version 9.0.0 will perform all Xenial package pre-fetch operations solely with the dedicated Axcient mirror, and version 10.0.0 and newer will perform ALL package operations solely with this mirror going forward.  All package operations will occur over https to ensure operational security.  This will allow partners to configure a single firewall rule allowing traffic between the BDR appliance and a single internet target rather than having to manage rules between many public mirror servers.

Firewall Configuration

In order to ensure that the upgrade to Ubuntu Xenial (16.04 LTS) can complete successfully, it is vital that your x360Recover devices have unrestricted access to the Axcient Ubuntu software mirror at https://rb-mirror.slc.efscloud.net (198.73.17.51)  Please ensure that your firewall rules are configured to allow all HTTPS traffic between the Axcient Ubuntu repository and your x360Recover devices, with all antivirus/antimalware/antispam scanning, etc. disabled.  Failure to enable unrestricted access to the software repository could prevent prefetching of the necessary software packages from being successful and may block the upgrade to version 10.0.0.  

Other Feature Enhancements

Default Retention Policy Changed

With this release, the default retention policy is changed to a tiered model.

Important:  Existing protected systems that are currently configured for the old default retention policy of the basic 120 days will automatically be migrated to the new tiered retention policy with this update.  If you wish to continue to maintain a basic retention policy for your protected systems,  you must configure the setting to something other than 120 days BEFORE the update to version 9.0.0 is installed on your devices.  (For example, you can change the retention policy to 119 or 121 days or configure a tiered retention policy to your liking.)

We recommend that all protected systems be migrated to a tiered retention policy. This will provide a significant storage savings both locally and in the cloud.  In addition, tiered storage policies greatly reduce the total number of snapshots maintained in the ZFS pool, helping performance of pool operations.

OS Recovery via Provisioning Wizard 

Since its inception, x360Recover has had the capability to recover a previously running device by reloading the hard drive from ISO and performing an OS Recovery operation to retrieve the configuration and storage data from the storage pool backup. 

This process has been rather cumbersome and relied on assistance from the Support team. 

In this release, the OS Recovery process has been moved into the Provisioning Wizard, greatly simplifying the process.  Axcient Support assistance will still be required in order to retrieve your storage pool encryption keys, but root user access is no longer necessary, and device passwords will be reset along with the standard Provisioning procedure.

AutoVerify Enhancements

A number of changes have been made to the backend AutoVerify process to reduce the number of test failures caused by various known issues.  Many protected systems previously failing to pass an AutoVerify test should now be able to succeed.

Operating System Hardening

With this release, many software packages that were no longer In use, or only used in rare instances, have been removed.  This provides both hardening of the underlying environment and a reduction in the time required to complete the OS upgrade to Ubuntu Xenial (coming in the next release.)

GMP Reporting Enhancements

When delivering customer specific reports, the subject line of the email will now include the customer account name.

Billing reports have been fixed so they now correctly calculate usage statistics when customers have multiple locations.