HCL DX docker-compose

This little scripting enables you to run a fully fledged DX environment with minimal footprint on your local machine. It uses docker-compose to start/stop and manage Docker containers. Docker-compose an addon on top of Docker. On Mac OS Docker desktop docker-compose is available out of the box. On other OS, you might need to manually install docker-compose even if you have docker installed already. For installation instructions see: https://docs.docker.com/compose/install/

Setup your environment

Start by cloning this repository locally and cd into the dx-docker-compose directory.

All you need to do is to load the HCL DX docker images into your local docker repository and set up your local environment with some environment variables.

Loading DX docker images

The load.sh script expects a path to a directory containing the docker image archives as a command line argument.

NOTE: If you already loaded the DX docker images into a docker repository of your choice, you may skip executing load.sh or load.bat. Please make sure to update the image names in the dx.properties file appropriately.

Linux/MAC:

cd ./dx-docker-compose
bash load.sh <docker-image-archives-directory>

Windows:

cd ./dx-docker-compose
load.bat <docker-image-archives-directory>

Set up local environment variables

If the docker compose is not running on local, then DX_HOSTNAME value in set.sh/set.bat needs to be modified accordingly.

Linux/MAC:

cd ./dx-docker-compose
source ./set.sh

Windows:

cd ./dx-docker-compose
set.bat

NOTE: The second command is source ./set.sh and not just executing set.sh directly.

If you want to unset your DX docker-compose environment, you can do so by running unset.sh:

Linux/MAC:

cd ./dx-docker-compose
source ./unset.sh

Windows:

cd ./dx-docker-compose
unset.bat

Starting the environment

After setting your environment, you can start the DX docker-compose environment by running. Important is that you need to be using a minimum version 1.27.4 for docker-compose.

docker-compose up

This will start all services defined in dx.yaml and logs will be printed directly go to your bash. You can stop docker-compose in this situation by pressing CTRL+C.

If your user does not have permission to write to the persistent volumes location (folder dx-docker-compose/volumes) specified in the docker-compose file dx.yaml, you will see errors and the system will not start properly. If necessary, change the permissions of this folder so that the user running the docker process can read from and write to it.

Here are some useful command line arguments to run docker-compose up:

-d, --detach: detached mode
--remove-orphans: this cleans up orphaned containers

For more information on startup parameters for docker-compose up, please see https://docs.docker.com/compose/reference/up/.

Stopping the environment

If you didn't start docker-compose in detached mode, you can stop by pressing CTRL+C. If you started docker-compose in detached mode, you can stop your environment by issuing

docker-compose stop

This will securely stop all running docker containers. If you want to properly clean up your system and even purge stopped docker containers, you can do so by issuing

docker-compose down

Looking at logs and metrics

Logs

If you want to look at logs for all of the DX services, you can easily do so by running

docker-compose logs

This will show you all system out logs of all services of all running containers (might be quite a lot - see tips and tricks below).

Metrics

You can also look at CPU, memory and network consumption using

docker stats

Example output:

NAME                 CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
dx-haproxy           0.00%     7.008MiB / 31.21GiB   0.02%     11.1MB / 10.8MB   0B / 0B           9
dx-dam               0.27%     493.4MiB / 31.21GiB   1.54%     263MB / 381MB     8.19kB / 19.5kB   78
dx-dam-db-pool       0.05%     116.8MiB / 31.21GiB   0.37%     586MB / 659MB     0B / 1.44MB       36
dx-ringapi           0.17%     104.2MiB / 31.21GiB   0.33%     1.45MB / 1.2MB    0B / 24.1kB       23
dx-dam-db-node-0     0.07%     63.26MiB / 31.21GiB   0.20%     414MB / 233MB     0B / 15.6MB       14
dx-core              0.80%     2.137GiB / 31.21GiB   6.85%     1.62MB / 6.71MB   436MB / 563MB     375
dx-cc                0.19%     71.73MiB / 31.21GiB   0.22%     7.7kB / 0B        0B / 11.3kB       23
dx-image-processor   0.17%     426.3MiB / 31.21GiB   1.33%     17.5MB / 4.27MB   0B / 23kB         23

To get an overview of running docker-compose services, you can run

docker-compose ps

Example output:

IMAGE                                                      COMMAND                  CREATED      STATUS                PORTS                                                                                                                                                                                                                                                                                                                                                                                                                      NAMES
hcl/dx/haproxy:v1.0.0_20220713-0158                        "/bin/bash entrypoin…"   3 days ago   Up 3 days             0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp                                                                                                                                                                                                                                                                                                                                                   dx-haproxy
hcl/dx/digital-asset-manager:v1.12.0_20211213-1448         "/bin/bash entrypoin…"   3 days ago   Up 3 days             0.0.0.0:80->8081/tcp, :::80->8081/tcp                                                                                                                                                                                                                                                                                                                                                                                      dx-dam
hcl/dx/persistence-connection-pool:v1.13.0_20211213-1457   "/scripts/entrypoint…"   3 days ago   Up 3 days (healthy)   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                                                                                                                                                                                                                                                                                                                                                                  dx-dam-db-pool
hcl/dx/ringapi:v1.13.0_20211213-1457                       "/opt/app/start_all_…"   3 days ago   Up 3 days             0.0.0.0:4000->3000/tcp, :::4000->3000/tcp                                                                                                                                                                                                                                                                                                                                                                                  dx-ringapi
hcl/dx/persistence-node:v1.3_20211213-1454                 "/start_postgres.sh"     3 days ago   Up 3 days (healthy)   0.0.0.0:5433->5432/tcp, :::5433->5432/tcp                                                                                                                                                                                                                                                                                                                                                                                  dx-dam-db-node-0
hcl/dx/core:v95_CF200_20211213-1442                        "sh -c /opt/app/entr…"   3 days ago   Up 3 days             0.0.0.0:7777->7777/tcp, :::7777->7777/tcp, 0.0.0.0:10020->10020/tcp, :::10020->10020/tcp, 10032/tcp, 0.0.0.0:10033->10033/tcp, :::10033->10033/tcp, 10034-10038/tcp, 0.0.0.0:10039->10039/tcp, :::10039->10039/tcp, 10040/tcp, 0.0.0.0:10041->10041/tcp, :::10041->10041/tcp, 10042/tcp, 0.0.0.0:10200->10200/tcp, :::10200->10200/tcp, 0.0.0.0:10202-10203->10202-10203/tcp, :::10202-10203->10202-10203/tcp, 10201/tcp   dx-core
hcl/dx/content-composer:v1.13.0_20211213-1443              "/opt/app/start_all_…"   3 days ago   Up 3 days             0.0.0.0:5000->3000/tcp, :::5000->3000/tcp                                                                                                                                                                                                                                                                                                                                                                                  dx-cc
hcl/dx/image-processor:v1.13.0_20211213-1446               "/home/dx_user/start…"   3 days ago   Up 3 days             0.0.0.0:3500->8080/tcp, :::3500->8080/tcp                                                                                                                                                                                                                                                                                                                                                                                  dx-image-processor

Tips and tricks

Performance Tuning

The performance for local docker volumes on some operating systems like Mac OS and Windows is quite slow. To improve especially the startup time of DX Core, you may choose to remove the persistent volume configuration. To do so, remove the following lines from the docker-compose file:

    volumes:
      - ./volumes/core/wp_profile:/opt/HCL/wp_profile

NOTE: When removing the volumes at all, any change you apply in DX Core will not be persisted. All your changes will be lost as soon as the container is stopped. If you need any specific folders or files been persistent, then please try to reduce it as much as possible. For example by just saving the server config of the WebSphere_Portal JVM to your local volume.

Debugging WebSphere_Portal JVM on dx-core container

It is possible to enable debugging mode on the WebSphere_Portal JVM to monitor/debug the whole server process.
For that, please make sure to enable port 7777 for the core service and in your firewall. Below you will find an example to expose that port 7777 in the dx.yaml file:

    services:  
      core:  
        container_name: dx-core  
        image: ${DX_DOCKER_IMAGE_CORE:?'Missing docker image environment parameter'}  
        ports:  
          - "7777:7777"

In addition to that it is needed to enable the WebSphere Application Server debugging services in the IBM Integrated Solutions Console (admin console). For details, please check: Debugging Service details

NOTE: Make sure that the changes are persistent for the current dx-core container. A WebSphere_Portal server restart is needed, as soon as the debugging services are enabled!

After debugging is enabled you can use any IDE like Microsoft Visual Studio Code, Eclipse or IBM Rational Application Developer to connect to that remote debugging port. For details, please check: HDX-DEV-300 HCL Digital Experience for Developers (Advanced)

Adding Shared Libraries

HCL highly recommends to add shared libraries in the folder /opt/HCL/wp_profile/PortalServer/sharedLibrary/ of the dx-core container to avoid performance problems with shared libraries.

Docker-compose services and load balancing

The core of a docker-compose environment are its services. In the case of DX, each of the different DX components (Core, CC, DAM, ...) is a individual docker-compose service. The services are all described and configured in dx.yaml. Amongst other configurations, each service has a external port defined.

Inside a docker-compose environment all containers of a particular service are reachable via their service name. If you connect into a docker container running in docker-compose, you'll be able to resolve the service name via dns. You could do so by just pinging the image processor (service name "image-processor") from any other container. See below on how to bash into a docker-compose container.

Running DX docker-compose in a hybrid setup

In the case that you already have a fully configured DX Core (e.g. an on premise installation) up and running, you can choose to configure docker-compose to connect to the on premise environment. The below mentioned changes in dx.yaml need to be applied to make this work.

NOTE: You will also have to configure your DX Core environment to connect to the services running docker-compose (e.g. configuration of DAM and Content Composer portlets). Please have a look in the official HCL DX Help Center to understand which changes need to be done, if necessary.

Update the Ring API service configuration as described:

Disable the depends_on parameter.
```
ringapi:
  # depends_on:
  #   - core
```
Update the PORTAL_HOST parameter values.

environment:
  - PORTAL_HOST=example.com

The result of the changes to the ringapi service should look similar to the snippet below:

ringapi:
  # depends_on:
  #   - dx-core
  image: ${DX_DOCKER_IMAGE_RINGAPI:?'Missing docker image environment parameter'}
  environment:
    - DEBUG=ringapi-server:*
    - PORTAL_PORT=10039
    - PORTAL_HOST=example.com
  ports:
    - "4000:3000"
  networks:
    - default

Update the Content Composer service configuration as described:

environment:
  - PORTAL_HOST=example.com

Starting and stopping individual services

Docker-compose up

docker-compose up allows you to start only individual services. To only start the DAM service, you could run

docker-compose up -d dam

For more information see https://docs.docker.com/compose/reference/up/

Docker-compose stop

docker-compose stop allows you to stop only individual services. To only stop the DAM service, you could run

docker-compose stop dam

For more information see https://docs.docker.com/compose/reference/down/

Docker-compose logs

To only look at logs for an individual service you can run

docker-compose logs dam

For more information see https://docs.docker.com/compose/reference/logs/

Installing Applications CC, DAM and DXConnect in DX Core

To install CC, DAM and DXConnect applications in DX Core and to enable , you can run

Linux/MAC:

cd ./dx-docker-compose
source ./installApps.sh

Windows:

cd ./dx-docker-compose
installApps.bat

NOTE: For any change in DX_HOSTNAME it's a must to re-execute installApps.sh / installApps.bat

Connecting to your DX and applications

To access your dx environment, navigate to http://<PORTAL_HOST>/wps/portal

Example: http://example.com/wps/portal

To access dx admin console, navigate to https://<PORTAL_HOST>:10041/ibm/console

Example: https://example.com:10041/ibm/console

To access the ConfigWizard Server Admin console https://<PORTAL_HOST>:10203/ibm/console

Example: https://example.com:10203/ibm/console

Connecting into a docker-compose service via bash

To bash into a docker container of a service, you can directly connect using the service name

docker-compose exec dam bash

To connect into a specific container of a service (if there is multiple containers running for a service), you have to look up the name of the container e.g. using docker-compose ps and then run

docker exec -it dx_dam bash

Running Prerequisite Checks to your DX and applications

To perform checks to the mounted volumes, you can directly connect using the dx-prereqs-checker container

docker-compose exec prereqs-checker /bin/bash /usr/local/sbin/run_test.sh

To display the logs of the check results, run

docker-compose logs prereqs-checker

DX Development with Visual Studio Code & Docker-Compose

Please check the course content:
HDX-DEV-300 HCL Digital Experience for Developers (Advanced)

Best practice

Always use docker-compose up command to start your environment
Make sure that the installApps.sh / installApps.bat script is already completed, before accessing the environment. Only then the additional extentions like Content Composer and DAM will be available!
Try to avoid using volumes (at best don't use volumes at all to get the best speed)
Avoid restarting the dx-core container for performance reasons (If needed, just stop/start the server directly in the dx-core container)
Add shared libraries into folder /opt/HCL/wp_profile/PortalServer/sharedLibrary/ of the dx-core container
In the Web-Browser:
- Access your Portal environment with URL: http://<hostname>/wps/portal.
- Don't access the portal over the direct port! (for example: http://localhost:10039/wps/portal). If you do so, then you might not be able to access addons like the Content Composer or DAM. The whole communication works via the embeded http-proxy!
- By default accessing the portal server environment via https is not configured out of the box. Additional steps need to be done to enable SSL.

Enabling Secured Socket Layer (SSL )

Prerequisites

OpenSSL need to be installed, if own self-signed certificates will be used. Installation instructions can be found at Install openSSL.

NOTE:
The ssl folder contains a localhost.pem file that can be used with the haproxy service running on a local environment. The certificate is created for the hostname localhost and it never expires. Please modify the create_certificates scripts for your needs, if you want to use your own certificates.

Instructions

Create own self signed certificates
Navigate to the ssl folder and execute the create_certificates script

Windows:
```
    create_certificates.bat  
```
Linux/Mac:
```
    ./create_certificates.sh  
```

Change the haproxy.cfg file in the following section:

  frontend dx
    bind :8081

    use_backend dam if { path -m reg ^/dx/(api|ui)/dam/ }
    use_backend content if { path_beg /dx/ui/content/ }
    use_backend image-processor if { path_beg /dx/api/image-processor/ }
    use_backend ring-api if { path_beg /dx/api/core/ }

    default_backend core-dx-home

  backend core-dx-home
    server core dx-core:10039 check resolvers nameserver init-addr none

to use:

  frontend dx  
    #BIND SSL and HTTP PORT
    bind :8083 ssl crt /etc/ssl/private/localhost.pem  
    bind :8081

    use_backend dam if { path -m reg ^/dx/(api|ui)/dam/ }
    use_backend content if { path_beg /dx/ui/content/ }
    use_backend image-processor if { path_beg /dx/api/image-processor/ }
    use_backend ring-api if { path_beg /dx/api/core/ }

    #DEFAULT BACKEND CONNECTS OVER HTTPS
    default_backend core-dx-home-ssl

  backend core-dx-home
    server core dx-core:10039 check resolvers nameserver init-addr none


  backend core-dx-home-ssl 
    server core dx-core:10041 check resolvers nameserver init-addr none ssl verify none

Modify the dx.yaml and change the haproxy service from:

haproxy:
  image: ${DX_DOCKER_IMAGE_HAPROXY:?'Missing docker image environment parameter'}
  container_name: dx-haproxy
  volumes:
    - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
  ports:
    - 80:8081
  networks:
    - default

to:

haproxy:
  image: ${DX_DOCKER_IMAGE_HAPROXY:?'Missing docker image environment parameter'}
  container_name: dx-haproxy
  volumes:
    - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
    - ./ssl/localhost.pem:/etc/ssl/private/localhost.pem      
  ports:
    - 80:8081
    - 443:8083
  networks:
    - default

Run docker-compose up to start the environment
Install or update CC, DAM and DXConnect applications in DX Core to use SSL

Linux/MAC:
```
./installApps_SSL_Enabled.sh
```
Windows:
```
installApps_SSL_Enabled.bat
```
Restart the dx-core container

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

HCL DX docker-compose

Setup your environment

Loading DX docker images

Set up local environment variables

Starting the environment

Stopping the environment

Looking at logs and metrics

Logs

Metrics

Tips and tricks

Performance Tuning

Debugging WebSphere_Portal JVM on dx-core container

Adding Shared Libraries

Docker-compose services and load balancing

Running DX docker-compose in a hybrid setup

Starting and stopping individual services

Docker-compose up

Docker-compose stop

Docker-compose logs

Installing Applications CC, DAM and DXConnect in DX Core

Connecting to your DX and applications

Connecting into a docker-compose service via bash

Running Prerequisite Checks to your DX and applications

DX Development with Visual Studio Code & Docker-Compose

Best practice

Enabling Secured Socket Layer (SSL )

Prerequisites

Instructions

About

Releases

Packages

Contributors 11

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 105 Commits
ssl		ssl
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
LICENSE		LICENSE
README.md		README.md
SUPPORT.md		SUPPORT.md
dx.properties		dx.properties
dx.yaml		dx.yaml
haproxy.cfg		haproxy.cfg
installApps.bat		installApps.bat
installApps.sh		installApps.sh
installApps_SSL_Enabled.bat		installApps_SSL_Enabled.bat
installApps_SSL_Enabled.sh		installApps_SSL_Enabled.sh
load.bat		load.bat
load.sh		load.sh
set.bat		set.bat
set.sh		set.sh
unset.bat		unset.bat
unset.sh		unset.sh

License

HCL-TECH-SOFTWARE/dx-docker-compose

Folders and files

Latest commit

History

Repository files navigation

HCL DX docker-compose

Setup your environment

Loading DX docker images

Set up local environment variables

Starting the environment

Stopping the environment

Looking at logs and metrics

Logs

Metrics

Tips and tricks

Performance Tuning

Debugging WebSphere_Portal JVM on dx-core container

Adding Shared Libraries

Docker-compose services and load balancing

Running DX docker-compose in a hybrid setup

Starting and stopping individual services

Docker-compose up

Docker-compose stop

Docker-compose logs

Installing Applications CC, DAM and DXConnect in DX Core

Connecting to your DX and applications

Connecting into a docker-compose service via bash

Running Prerequisite Checks to your DX and applications

DX Development with Visual Studio Code & Docker-Compose

Best practice

Enabling Secured Socket Layer (SSL )

Prerequisites

Instructions

About

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

Packages 0

Contributors 11

Languages

Packages