Upgrade from 5.9

General recommendations

Be sure to read the Everyware Cloud release notes and the planning the upgrade section.

It's recommended to upgrade to the latest version of the 5.9.x release before moving to 5.10.

Preparing to upgrade

Before upgrading to a patch release be sure to follow these steps:

• backup the databases (relational and Elasticsearch)
• backup your existing configuration exporting all project configurations

This is really important as in case of issues with the migration it would be possible to revert back the changes.

🚧

Upgrade Notice

An upgrade from version 5.9 or earlier requires a database migration. In order to ease the step, a migration a tool has been implemented to automate the process. The migration tool is available as a Docker Image and can be downloaded from the same repo of the Everyware Cloud component images. Check the upgrade steps below.

Upgrade steps

The process to upgrade Everyware Cloud is the following:

• Login to the Helm chart registry

HELM_EXPERIMENTAL_OCI=1
helm registry login registry.everyware-cloud.com

• Download the latest version of the Helm charts from the registry

cd charts

HELM_EXPERIMENTAL_OCI=1
helm chart pull registry.everyware-cloud.com/eurotech/charts/<chart-name>:<everyware-cloud-version>
helm chart export registry.everyware-cloud.com/eurotech/charts/<chart-name>:<everyware-cloud-version>

📘

Helm 3.7+

For those using Helm 3.7+ the last two lines in the snippet above have to be replaced by the following:

helm pull --untar oci://registry.everyware-cloud.com/eurotech/charts/<chart-name> --version <everyware-cloud-version>

Note that the schema oci:// needs to be added to the URL.

• Scale down all Everyware Cloud components

# For Kubernetes
COMPONENTS=("ec-events-broker" "ec-api" "ec-broker" "ec-console" "ec-vpn" "ec-job-engine" "ec-console-ng")
for COMPONENT in "${COMPONENTS[@]}"; do
  kubectl scale deployment/${COMPONENT} --replicas=0
done

# For OpenShift
COMPONENTS=("ec-events-broker" "ec-api" "ec-broker" "ec-console" "ec-vpn" "ec-job-engine" "ec-console-ng")
for COMPONENT in "${COMPONENTS[@]}"; do
  oc scale deploymentconfig/${COMPONENT} --replicas=0
done

Run the EC Migrator Tool

• Check all the default parameters of the migration tool and its usage in the readme. Check the Installation with Helm Charts section to see how to access the charts.

helm show readme charts/ec-migrator-tool

• Launch the Everyware Cloud Migration tool using the provided Helm chart

🚧

During the migration operation don't run any other operation (e.g. snaphshots) on the MariaDB instance.

helm install -n everyware-cloud -f ec-migrator-tool.yaml ec-migrator-tool charts/ec-migrator-tool

After the Migrator Tool has started open a terminal into the container.

# For Kubernetes
kubectl get pods -n everyware-cloud | grep ec-migrator-tool
kubectl exec --stdin --tty <ec-migrator-tool-pod> -- /bin/bash

# For OpenShift
oc get pods -n everyware-cloud | grep ec-es-migrator
oc rsh <ec-migrator-tool-pod>

• Once logged into the container shell switch to the Database Attribute Migration Tool script

cd /opt/ec/entity-secret-migration-tool

From here you can run the entitySecretMigrationTool.sh which is a helper shell script which runs the Database Attribute Migration Tool application.

./entitySecretMigrationTool.sh

Configuration values for the database target, username and password are taken from the ENVs of the environment.
After launching the shell script you'll be required to enter the old encryption key and the new one. When running the tool for the migration of the database from 5.9 release, this must be left empty.
Encryption uses the AES standard, encryption key values must have a size of 128, 192 or 256 bits (16, 24 or 32 bytes).
If you want to run the migration tool without doing the actual database migration, you can use the --dry-run option.

```
./entitySecretMigrationTool.sh [--dry-run]
Enter value for --old-encryption-key (The (old) key used to encrypt the current data. It can be prompted after executing the command): 
Enter value for --new-encryption-key (The new key used to use to encrypt the data. It can be prompted after executing the command): 
```

Once the tool has completed the container for the migration tool can be scaled down or removed.

# Removal
helm delete -n everyware-cloud ec-migrator-tool

# Scaling down Kubernetes
kubectl scale deployment/ec-migrator-tool --replicas=0

# Scaling down OpenShift
oc scale deployment/ec-migrator-tool --replicas=0

Update Components and configurations

• Set the database schema update property to true in the ec-configs values override file and run the Helm upgrade procedure

helm upgrade -n everyware-cloud -f ec-configs.yaml ec-configs charts/ec-configs

🚧

Extend health check timeouts

When running EC Services for the first time the startup may take longer than usual because of some database optimizations that have been introduced with this release. The time required may vary depending on the amount of data stored. We advise to extends health check thresholds for services ready-ness in the container orchestrator. After the first run thresholds can be set back to the previous values since the optimization will run just once.

• Upgrade the standard EC components using Helm, changing the version of the container images in the values override file. Before launching the upgrade command it's necessary to remove the deployment as described in the following snipped (use the oc command instead of kubectl for OpenShift installations)

# Deploy the Events Broker
kubectl delete -n everyware-cloud deployment/ec-events-broker
helm upgrade -n everyware-cloud -f ec-events-broker.yaml ec-events-broker charts/ec-events-broker

# Deploy the Message Broker
kubectl delete -n everyware-cloud deployment/ec-broker
helm upgrade -n everyware-cloud -f ec-broker.yaml ec-broker charts/ec-broker

# Deploy the Job Engine
kubectl delete -n everyware-cloud deployment/ec-job-engine
helm upgrade -n everyware-cloud -f ec-job-engine.yaml ec-job-engine charts/ec-job-engine

# Deploy the RESTful API
kubectl delete -n everyware-cloud deployment/ec-api
helm upgrade -n everyware-cloud -f ec-api.yaml ec-api charts/ec-api

# Deploy the Web Console
kubectl delete -n everyware-cloud deployment/ec-console
helm upgrade -n everyware-cloud -f ec-console.yaml ec-console charts/ec-console

• Repeat the upgrade operation for the optional components ec-vpn and ec-monitoring if installed.

# Upgrade the VPN Service if installed
kubectl delete -n everyware-cloud deployment/ec-vpn
helm upgrade -n everyware-cloud -f ec-vpn.yaml ec-vpn charts/ec-vpn

# Upgrade the Monitoring Service if installed
kubectl delete -n everyware-cloud deployment/ec-monitoring
helm upgrade -n everyware-cloud -f ec-monitoring.yaml ec-monitoring charts/ec-monitoring

• Set the database schema update property to false in the ec-configs values override file and run the Helm upgrade procedure

helm upgrade -n everyware-cloud -f ec-configs.yaml ec-configs charts/ec-configs

🚧

Restore health check timeouts

If health checks thresholds have been extended, restore the previous configurations.

Rollback

In case of problems with the upgraded version it's possible to revert to a previous version.

The steps for the rollback are the following:

• stop all the services scaling down the deployments to 0
• restore the databases to the backed-up version
• execute the Helm charts for the previous installed version with the old values override files
• start the services scaling up the deployments to 1 and then, once the service is available again, to the original number of pods. Follow the order described in the planning section.

Troubleshooting Database Encryption migration

If any error occurs while migrating the data the tool can be run again after the issue has been resolved.
You need to provide the encryption key used on the failed run as --old-encryption-key.
The --old-encryption-key and --new-encryption-key can have the same value.

Encryption key values must have a size of 128, 192 or 256 bits (16, 24 or 32 bytes).

Depending on the error you observe you may:

• Check the list of launch parameters as explained above. Changing default values may allow to adapt the behaviour of the tool to your operational conditions
• Contact the support team