Installation with Helm Charts

This page describe Everyware Cloud installation process using Helm Charts on Kubernetes and OpenShift.

Namespace creation

The first step is to create a dedicated namespace. In this documentation, we will use the namespace everyware-cloud. This is the default value used on the helm charts; if a different one is used, the configuration passed to the helm charts should be changed accordingly.

# For Kubernetes
kubectl create namespace everyware-cloud
kubectl label namespaces everyware-cloud name=everyware-cloud

# For OpenShift
oc new-project everyware-cloud \
    --description="Everyware Cloud deployment" \
    --display-name="Everyware Cloud"
oc label namespaces everyware-cloud name=everyware-cloud

It's important to add the namespace's name label if enforcing NetworkPolicies.

Docker registry secret

Firstly create a docker-registry secret containing the credentials for the Everyware Cloud docker registry. Replace username and password with the values provided by Eurotech.

# For Kubernetes
kubectl -n everyware-cloud create secret docker-registry eurotech-docker-registry \
    --docker-server=registry.everyware-cloud.com \
    --docker-username=<USERNAME> \
    --docker-password=<PASSWORD>

# For OpenShift
oc -n everyware-cloud create secret docker-registry eurotech-docker-registry \
    --docker-server=registry.everyware-cloud.com \
    --docker-username=<USERNAME> \
    --docker-password=<PASSWORD>

Service Secrets

Before deploying Everyware Cloud it is advised to configure few secrets that will be used by the containers. In particular, EC uses secrets for both the credentials and the certificates.

Database

First, configure the secrets for the database using the command shown below. Change the values according to your own environment. The default name for the secret in the charts is ec-credentials-db .

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-db \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>
    
# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-db \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

Encryption of Data

Some informations need to be stored in an encrypted form. Create the secret used to encrypt and decrypt back these information. The default name for this secret in the charts is ec-crypto. Encryption uses the AES standard, values of secretKey(see command below) must have a size of 128, 192 or 256 bits (16, 24 or 32 bytes).

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-crypto \
    --from-literal=secretKey=<SECRET-KEY>

# For OpenShift
oc -n everyware-cloud create secret generic ec-crypto \
    --from-literal=secretKey=<SECRET-KEY>

Events Broker

Configure the secrets for the events broker. These are used for the internal communication between the EC components and the events broker itself. The default name for the secret in the charts is ec-credentials-events-broker .

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-events-broker \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-events-broker \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

Message Broker

Create the secret used for the communication between the various services and the message broker. The default name for this secret in the charts is ec-credentials-transport .

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-transport \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-transport \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

Datastore

When the Datastore service is enabled you may need to configure extra secrets to properly manage connection credentials and or TLS protocol depending on the security configurations of the Elasticsearch engine instance. This secrets should be referenced in all the Everyware Cloud components as described in the Helm chart documentation.

Username and password.

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-storage \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-storage \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

Logstore

When the Logstore service is enabled, you may need to configure extra secrets to properly manage connection credentials and or TLS protocol depending on the security configurations of the Elasticsearch engine instance. This secrets should be referenced in all the Everyware Cloud components as described in the Helm chart documentation.

Username and password.

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-logstore \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-logstore \
    --from-literal=username=<USERNAME> \
    --from-literal=password=<PASSWORD>

Cache

When caching is enabled, an extra secret with the credentials for the Redis service needs to be created. This secret should be referenced in all the Everyware Cloud components as described in the Helm chart documentation.

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-cache \
    --from-literal=password=<PASSWORD>

# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-cache \
    --from-literal=password=<PASSWORD>

Kafka Routes

If planning to use message routes to a Kafka cluster, the JaaS authentication file should be stored in a secret. This secret should then be referenced in the configuration file of the Message Broker Helm chart as described in the chart documentation.

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-jaas-config-kafka \
    --from-file=jaas.conf=<PATH-TO-FILE>
    
# For OpenShift
oc -n everyware-cloud create secret generic ec-jaas-config-kafka \
    --from-file=jaas.conf=<PATH-TO-FILE>

The content of the JaaS file varies depending on the authentication method used. The following snippet shows the content format for username and password authentication.

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="ec_kafka_username"
    password="ec_kafka_password";
};

Monitoring

Lastly, to expose the monitoring endpoint with basic authentication, create a secret containing an htpasswd file with the credentials:

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-credentials-monitoring \
    --from-file=htpasswd=<PATH-TO-FILE>
    
# For OpenShift
oc -n everyware-cloud create secret generic ec-credentials-monitoring \
    --from-file=htpasswd=<PATH-TO-FILE>

Certificates

This step creates a secret containing the certificate, the private key and the CA chain for the Everyware Cloud services. This can be done multiple times if not using wildcards certificates or in case separate certificates are used for each service. The default is a single secret ec-certificate containing a wildcard certificate. This step can be skipped if SSL/TLS is not used (not recommended).

The following files are necessary to create certificate secrets:

file containing the PEM encoded certificate
file containing the PEM encoded private key
file bundling the certificate's parent CAs
file bundling the concatenation of the certificate and certificate's parent CAs

With these files run the following command:

# For Kubernetes
kubectl -n everyware-cloud create secret generic ec-certificate \
    --from-file=crt=<PATH-TO-CERTIFICATE-FILE> \
    --from-file=key=<PATH-TO-KEY-FILE> \
    --from-file=ca=<PATH-TO-CA-BUNDLE-FILE> \
    --from-file=full=<PATH-TO-CERTIFICATE-AND-CA-CHAIN-FILE>
    
# For OpenShift
oc -n everyware-cloud create secret generic ec-certificate \
    --from-file=crt=<PATH-TO-CERTIFICATE-FILE> \
    --from-file=key=<PATH-TO-KEY-FILE> \
    --from-file=ca=<PATH-TO-CA-BUNDLE-FILE> \
    --from-file=full=<PATH-TO-CERTIFICATE-AND-CA-CHAIN-FILE>

If the services are going to be exposed via an NGINX ingress controller, the following secret of type TLS should also be created:

# For Kubernetes
kubectl -n everyware-cloud create secret tls ec-certificate-ingress \
    --key=<PATH-TO-KEY-FILE> \
    --cert=<PATH-TO-CERTIFICATE-AND-CA-CHAIN-FILE>
    
# For OpenShift
oc -n everyware-cloud create secret tls ec-certificate-ingress \
    --key=<PATH-TO-KEY-FILE> \
    --cert=<PATH-TO-CERTIFICATE-AND-CA-CHAIN-FILE>

Getting the Helm charts

Helm charts are hosted on the Eurotech docker registry. To retrieve them follow the next steps.

Firstly login to the registry. This is a one time operation before doing the next ones. The credentials are the same used for the eurotech-docker-registry secret.

HELM_EXPERIMENTAL_OCI=1
helm registry login registry.everyware-cloud.com

Then, retrieve and save locally the Helm charts. This command should be executed for every chart that appears in the install sections below. For this documentation we are going to store and use the charts in the charts directory

cd charts

HELM_EXPERIMENTAL_OCI=1
helm chart pull registry.everyware-cloud.com/eurotech/charts/<chart-name>:<everyware-cloud-version>
helm chart export registry.everyware-cloud.com/eurotech/charts/<chart-name>:<everyware-cloud-version>

📘
Helm 3.7+
For those using Helm 3.7+ the last two lines in the snippet above have to be replaced by the following:
helm pull --untar oci://registry.everyware-cloud.com/eurotech/charts/<chart-name> --version <everyware-cloud-version>
Note that the schema oci:// needs to be added to the URL.

Common configuration

This procedure installs a ConfigMap containing the common configurations used by all the EverywareCloud components like the database endpoint URL. The first step is to configure the values file ec-configs.yaml containing all the custom values for the environment. The description of the accepted options can be obtained with the following command:

helm show readme charts/ec-config

To create the ConfigMap run the following command:

helm install -n everyware-cloud -f ec-configs.yaml ec-configs charts/ec-configs

Everyware Cloud components

Everyware Cloud is composed by five different parts: events broker, RESTful API, Web Console, message broker, VPN and basic monitoring.

The installation is pretty straightforward. First create the configuration files for the various components. A description of the possible options can be obtain with the following command:

helm show readme charts/<component>

Once the configuration files have been created it's just a matter of applying the Helm chart. The script below lists the commands in the suggested order of execution. Wait for a component to be fully up before starting the deployment of the next one.

# Deploy the Events Broker
helm install -n everyware-cloud -f ec-events-broker.yaml ec-events-broker charts/ec-events-broker

# Deploy the Message Broker
helm install -n everyware-cloud -f ec-broker.yaml ec-broker charts/ec-broker

# Deploy the RESTful API
helm install -n everyware-cloud -f ec-api.yaml ec-api charts/ec-api

# Deploy the Web Console
helm install -n everyware-cloud -f ec-console.yaml ec-console charts/ec-console

# Deploy the Job Engine
helm install -n everyware-cloud -f ec-job-engine.yaml ec-job-engine charts/ec-job-engine

Remote Access Service

The Remote Access Service is an optional component that allows remote connections to the devices via an on-demand VPN tunnel. This component requires some extra policies to be configured at cluster level. You'll need cluster-admin role in order procede with the next steps.

Firstly create a service account at project level. This can be done by a normal project administrator.
It's not necessary to repeat the following prerequisite steps when installing multiple Remove Access Service components.

# For Kubernetes
kubectl create sa ec-vpn-sa

# For OpenShift
oc create sa ec-vpn-sa

Then, as a cluster-admin install the new policy for the VPN and assign it to the service account. For the ec-vpn-role chart you'll need to customise the chart passing the service account name. You can create a different project to store the helm chart status instead of using the default project.

# Login a clust-admin user

# For Kubernetes
helm install -n default ec-vpn-policy charts/ec-vpn-policy
helm install -n everyware-cloud -f ec-vpn-role.yaml ec-vpn-role charts/ec-vpn-role

# For OpenShift
helm install -n default ec-vpn-policy charts/ec-vpn-policy
oc adm policy add-scc-to-user anyuid-ec-vpn -z ec-vpn-sa -n everyware-cloud

At this point it's possible to install the VPN Service itself using a normal project administrator user.

# Deploy the VPN Server
helm install -n everyware-cloud -f ec-vpn.yaml ec-vpn charts/ec-vpn

Basic Monitoring

This step installs basic monitoring tools to retrieve metrics from the other components. It's composed of a service account with read permission on some of the resources of the namespace and the monitoring container themselves.

# Deploy the Monitoring Service
helm install -n everyware-cloud -f ec-monitoring.yaml ec-monitoring charts/ec-monitoring

Scaling

Everyware Cloud supports horizontal scaling and workload partitioning.

Horizontal scaling

With horizontal scaling it's possible to expand or reduce the capacity of a service instance, the service instance is the same but the amount of computational resources assigned to it gets increased or decreased. Horizontal scaling is available for the following services:

RESTful API
Message Broker
Admin Console services (require stickiness).

To expand or reduce the capacity increase or decrease the replication count property in the values of the Helm Charts.

Workload partitioning

With workload partitioning it's possible to create multiple service instances and assign to each of them an account account or a group of accounts. Workload partitioning is available for the following services:

Message Broker
VPN Service

Message Broker supports both horizontal scaling (within the same service instance) and workload partitioning, the VPN service, instead, supports just workload partitioning.

To deploy a distinct service instance just set a name for the parameter release of the helm command. When no value is provided for the release parameter, the default names are assumed (see the defaults for the CLUSTERNAME and VPN_SERVER_NAME below). The name set for the _release parameter of the helm command must match with the value set for the following properties:

Message Broker service: CLUSTER_NAME Container Properties
VPN Service: VPN_SERVER_NAME Container Properties

Each EC account must be assigned the the correct service instance so that client devices or applications of the account will use the instance when they connect to the service. The procedure to execute the assignment of the accounts is described in the Messaging and VPN Servers sections.

Affinity and Node Selector

It's possible to specify node affinity and/or node selector using the affinity and nodeSelector values in the chart property file. Simply define the affinity rules in YAML format. More documentation can be found on the official Kubernetes documentation.

Rolling Deployment Strategy

By default the components are installed with a replace deployment strategy. This means that when a new version is installed or a configuration is changes all the existing pods are terminated and new ones are then launched. This is the safest approach as pods of different version are not running at the same time.

In some cases it may be better to opt in for a rolling deployment strategy. This way the containers are replaced in batches in order to avoid downtimes of the system. To enable the rolling deployment strategy change the Helm values override file adding the following lines.

deployment:
...
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
...

The possible options are described in the Helm chart readme.

Namespace creation

Docker registry secret

Service Secrets

Database

Encryption of Data

Events Broker

Message Broker

Datastore

Logstore

Cache

Kafka Routes

Monitoring

Certificates

Getting the Helm charts

📘Helm 3.7+

Common configuration

Everyware Cloud components

Remote Access Service

Basic Monitoring

Scaling

Horizontal scaling

Workload partitioning

Affinity and Node Selector

Rolling Deployment Strategy

📘
Helm 3.7+