Restore Backup

Hubble automatically backs up its data, including its repository and images, on the application server at the path /mnt/data/backups.

Before restoring ensure that Hubble is running. If you are uncertain, run the following to start Hubble :

/etc/hubble/start.sh

Restoring S3 objects

The restoration of S3 objects will be performed via the Hubble S3Service.

Before starting retrieve some information that will be need later.

On the Application Server, execute the following command and make note of the S3_STORAGE_ACCESS_KEY and S3_STORAGE_SECRET_KEY value.

If running Hubble 20.3 or earlier:

If running Hubble 20.4 or later:

There are 3 buckets that are backed up and need to be restored: imageview-files, user-photos and workflow-snapshots (<BUCKET_NAME>s).

The S3 port (S3_PORT) is 591 or 592 if using HTTPS.

Option 1 - Command Line

1. Execute the following command for each of the buckets named above, ensuring to replace placeholders:

   curl -X POST --header 'Accept: application/json'

   'http://<app_server>:5000/api/Bucket/

   restore?Host=<app_server>&Port=<S3_PORT>&AccessKey=<S3_STORAGE_ACCES S_KEY>&SecretKey=<S3_

   STORAGE_SECRET_KEY>&BucketName=hubble-<BUCKET_NAME>'

   e.g. for imageview-files:

   curl -X POST --header 'Accept: application/json' 'http://

   10.13.0.81:5000/api/Bucket/

   restore?Host=10.13.0.81&Port=591&AccessKey=aphnOhuT&SecretKey=LZ6to1 Dd&BucketName=hubble-

   imageview-files'

   The response of the command will be a JSON response of the following format:

   {"bucketName":"hubble-<BUCKET_NAME>"}

   e.g. for imageview-files:

   {"bucketName":"hubble-imageview-files"}

2. Each of the files in hubble-user-photos bucket need to have public read permissions for everyone.
   1. List the files that were restored to the hubble-user-photos bucket:

      ls /mnt/data/backups/hubble_s3_service/hubble-user-photos/

      9C51187E907163ABA5BD64E0CD20D416

      If no files were listed, the following steps can be skipped.

   2. Execute the following command for each of the files listed:

      curl -X PUT --header 'Accept: application/json' 'http://

      <app_server>:5000/api/File/ permission?Host=<app_server>&Port=<S3_PORT>&AccessKey=<S3_STORAGE

      _ACCESS_KEY>&SecretKey=<S3_STORAGE_SECRET_KEY>&FileName=<FILENAME

      >&BucketName=hubble-user-photos&S3PermissionKey=public-read'

      e.g.:

      curl -X PUT --header 'Accept: application/json' 'http:// 10.13.0.81:5000/api/File/

      permission?Host=10.13.0.81&Port=591&AccessKey=aphnOhuT&SecretKey=

      LZ6to1Dd&FileName=9C51187E907163ABA5BD64E0CD20D416&BucketName=hubble-user-photos&S3PermissionKey=public-read'

Option 2 - UI

1. The S3Service web UI can be accessed using a web browser. Navigate to:
   http://<app server>:5000/swagger.

Once the page is loaded, you should see a web page similar to:

2. Under the 'Bucket' grouping, find and click on “/api/Bucket/restore” to reveal the following:

   

3. Complete the above web form with the following details to restore each of the buckets named above:
   1. Host: <app server ip or DNS name>
   2. Port: <S3_PORT>
   3. AccessKey: <S3_STORAGE_ACCESS_KEY> (from above)
   4. SecretKey: <S3_STORAGE_SECRET_KEY> (from above)
   5. BucketName: hubble-<BUCKET_NAME> (from above)

      e.g.:

      

4. Clicking the Try it out! button should display the following:

   

   Ensure that a 'Response Code' of '200' was returned, which means it was successful.

5. Each of the files in hubble-user-photos bucket needs to have public read permissions for everyone.
   1. Execute the following command on the Application Server to list the files that were restored to the hubble-user-photos bucket:

      ls /mnt/data/backups/hubble_s3_service/hubble-user-photos/ 9C51187E907163ABA5BD64E0CD20D416

      If no files were listed, the following steps can be skipped.

   2. Under the 'File' grouping, find and click on “/api/File/permission” to reveal the following:

   

   3. For each of the files listed above, enter the appropriate details and click Try it out!.

      Most of the details are the same as above, except “BucketName” which should be “hubble- user-photos” and enter each file name in the “FileName” field.

      The response should look similar to the below for each file:

      

Restoring Hubble Repository

The restoration of the Hubble Repository will be performed via Hubble Repository API.

Option 1 - Command Line

1. Connect remotely using SSH to the Application Server and navigate to the backups subdirectory named “hubble_repository”,

   i.e.:

   cd /mnt/data/backups/hubble_repository/

2. From this directory, determine the backup you would like to restore.

   ls /mnt/data/backups/hubble_repository/

   hubble_repository-2018-02-16-00:00.bak

   Execute the following command, replacing <backupFileName> with the name of your chosen backup file from above.

   curl -I -X POST -T <backupFileName> 'http://<app_server>:5433/api/

   Repository/database/restore'

   e.g.:

   curl -I -X POST -T hubble_repository-2018-02-16-00:00.bak 'http://

   10.13.0.123:5433/api/Repository/database/restore'

   The expected response is:

   HTTP/1.1 100 Continue

   HTTP/1.1 200 OK

   Date: Tue, 06 Mar 2018 10:50:00 GMT

   Content-Length: 0

   Server: Kestrel

   In particular, the 3rd line above ending in '200 ok' indicates success.

Option 2 - UI

1. The 'hubble_repository' backups subdirectory is the location of repository backups, i.e.:

   /mnt/data/backups/hubble_repository

2. Retrieve the backup file you wish to restore from the Application Server.
3. Hubble Repository API is a REST API so you will require an application that is able to perform REST operations.

   Postman is the tool used in the following procedure.

   

4. We need to use a POST operation to:

   http://<app_server>:5433/api/Repository/database/restore

   ...providing the backup file from above.

   

   For Postman, select the drop down (1) and select the 'POST' entry (2). Then enter the appropriate URL (3).

5. The body of the request should be the backup file.

   

   Select the 'Body' tab (1) and click “Choose Files” (2) then select the backup file.

6. Clicking send will perform the operation. The response/status should be '200', which means the restore was successful.

   

Restoring Vault Secrets

1. Unlike when restoring the repository and S3 objects, you must ensure that the application server services are not running before restoring vault secrets. Run /etc/hubble/stop.sh if you are unsure.
2. Run docker ps. It should return empty as no services are running.
3. Check for the presence of a Vault backup by running ls /mnt/data/backups/ hubble_vault/. The output should be similar to this:

   

4. Remove the existing data by running:

   rm -rf /mnt/data/containers/hubble_vault

5. Restore the backup by running:

   cp -r /mnt/data/backups/hubble_vault /mnt/data/containers/

6. Start the application server again by running /etc/hubble/start.sh.