Note that this setup might take as long as 15 minutes so please run these steps before the workshop!
If you'd like help with these commands you can reach out on this Liferay community Slack channel: #devcon-2023-client-extensions-101-workshop
-
Clone repo:
git clone https://github.com/LiferayCloud/client-extensions-deep-dive-devcon-2023.git
-
Change into workspace
cd client-extensions-deep-dive-devcon-2023
-
Initialize the bundle (this downloads dependencies so it might take a while)
./gradlew initBundle
-
Start DXP
-
Make sure you have java8 or java11 installed and in the system path (
java -version
) to confirm -
Linux/Mac:
./bundles/tomcat-9.0.73/bin/catalina.sh run
-
Windows:
.\bundles\tomcat-9.0.73\bin\catalina.bat run
-
-
DXP should pop up automatically on http://localhost:8080
-
Log in with user
test@liferay.com
and passwordtest
Note - if the page is unresponsive you might have to refresh the page once before logging in (known issue).
-
Change the password to something you can remember
-
Build all projects
./gradlew build
-
That's it! You can leave DXP running if you are about to complete the workshop exercise, or shut it down
What are Client Extensions again?
Client extensions are a generic mechanism for customizating or extending DXP which run outside of DXP.
The extensions are defined in aclient-extension.yaml
file where we specify their properties.
This config file is deployed to DXP in order to give it the configuration information necessary to communicate with our Client Extension. For secure communcation between DXP and your client extension, OAuth2 can be used easily by specifing oAuth2Application* type of client extension. To learn more about Client Extensions visit the reference documentation.
In this workspace we will use Client Extensions to build the following use case:
-
A ticket management system
-
Requirements:
- Defines a Customized Data Schema
- Applies the Corporate brand and style
- Provides a Customized User Application
- Implements Programmatic Documentation Referral
In the end it should look like the following image:
Our domain model is Ticket but DXP doesn't have the concept of a Ticket. Traditionally, we would have used Service Builder to model this but today we will use the Objects feature of DXP to define it.
We will start the definition of our domain model by creating some Picklists. A picklist is a predetermined list of values a user can select, like a vocabulary. We can use Picklists when modelling Objects where an attribute needs to be constrained to specific values. For instance; status, priority, region and so on.
The Picklists we need are already defined in the project client-extensions/list-type-batch
.
This project's client-extension.yaml
declares a client extension of type: batch
(see Batch Client Extensions) which is used to import DXP resources without requiring us to write any code. Resources are exported from DXP's Import/Export Center (in the JSONT
format required for client extensions) and placed in the project's batch
directory. Note that batch engine data files are not generated by hand. However, they are intended to be editiable by humans.
Execute the following commmand from the root of the workspace to deploy the picklists:
./gradlew :client-extensions:list-type-batch:deploy
Watch the tomcat logs to see that the client extension deployed.
Now we can deploy our Ticket object defeinition which we have already definded for you and put into this project client-extensions/ticket-batch
.
Again, this project's client-extension.yaml
declares a client extension of type: batch
. It's batch
directory contains the batch engine data file where the Ticket object definition is defined.
Execute the following commmand from the root of the workspace to deploy the Ticket object:
./gradlew :client-extensions:ticket-batch:deploy
Watch the tomcat logs to see that the client extension deployed.
When defining a domain model using Objects a set of headless APIs are automatically generated for you without any additional effort.
You can view these APIs in DXP's built in headless API browser by following this link: Tickets Headless API
Action item: Please view the endpoints of the headless API now.
We created the first ticket by hand, but in the scenario where you have pre-existing data, you can import it using batch (several of these operations do need to be performed in order)
- Lets deploy some pre-existing tickets
./gradlew :client-extensions:ticket-entry-batch:deploy
- Now you can see these ticket entires in the DXP UI
We've acheived our first business requirement: Define a Customized Data Schema. Let's move onto the next.
Most organizations, after some level of maturity, have established a brand and style which, ideally, is carried through each new project. There are a number of existing client extensions available to support this use case as opposed to traditional Theme module. These are a subset of the client extensions referred to collectively as Front-End Client Extensions.
The client-extensions/ticket-theme-css
project's client-extension.yaml
declares a client extension of type: themeCSS
(see Theme CSS Client Extension) which is used to replace the two core CSS resources from the portal's OOTB themes without modifying DXP.
Execute the following commmand from the root of the workspace to deploy the tickets-theme-css project:
./gradlew :client-extensions:ticket-theme-css:deploy
Watch the tomcat logs to see that the client extension deployed.
At this point let's return to the main page of our site. Let's apply the tickets-theme-css to the home page as demonstrated in the following video:
We've acheived our second business requirement: Apply the Corporate brand and style. Let's move onto the next.
Our next business requirement is to build a customized user application. Today in DXP, there are low code mechanisms for doing this which directly support objects but which are not yet enabled as client extensions. So today we are going to solve this using the Custom Element Client Extension which enables use to build portal applications based on HTML 5 Web Components. In this case, using React.
The project is the client-extensions/current-tickets-custom-element
. This project is a Javascript project with a package.json
file that has a .scripts.build
property which allows it to be seamlessly integrated into the workspace build (the workspace handles this integration for you and even handles the precise Javascript build tool installation and initialization & build tasks like $pkgman install
and $pkgman run build
. Here in this workspace $pkgman
is yarn
out of preference only.)
While we inspect this project, let's take a short sidebar and consider the client-extension.yaml
's assemble
block (see Assembling Client Extensions).
Note that in each of the previous projects we did already have the assemble block but let's take a minute to review it here. As was eluded to in the previous paragraph the workspace build knows how to seamlessly integrate certain non-Gradle builds. This is true of most Front End client extensions. However it doesn't know what to include in the LUFFA.
assemble:
- from: build/assets
into: static
The assemble block allows you to declare what resources need to be included in the LUFFA.
Execute the following commmand from the root of the workspace to deploy the current-tickets-custom-element project:
./gradlew :client-extensions:current-tickets-custom-element:deploy
Watch the tomcat logs to see that the client extension deployed.
At this point let's return to the main page of our site. Let's remove the main grid section and add the current-tickets-custom-element in place of it as demonstrated in the following video
Note that this app uses the auto-generated Ticket headless APIs.
We've acheived our third business requirement: Provide a Customized User Application. Let's move onto the last.
Our last business requirement is to implement a business logic that will improve the speed of resolving tickets so that we can serve customers more efficiently using an programmatic strategy to assess ticket details and adding information directly for the customer and maybe reducing the amount of research support agents need to perform in order to resolve the issue.
The client-extensions/ticket-spring-boot
project's client-extension.yaml
declares a client extension of type: objectAction
(see Object Action Client Extension) which enables Object event handler which is implemented as a REST endpoint to be registered in Liferay.
Before we proceed we will make one small change to one of the previously deployed client extensions and redeploy it. Edit the file client-extensions/ticket-batch/batch/ticket-object-definition.batch-engine-data.json
.
On line 46
change the value of "active"
from false
to true
. Save the file and then (re)execute the command:
./gradlew :client-extensions:ticket-batch:deploy
One small sidebar about this notion of redeployment. It is intended that all deployment operations from the workspace should be idempotent (or that redeployments should both be effective and not result in error). This is not only important as a mechanism to speed up iterative development, but as a means to move changes between environments; such as moving future changes from a DEV to UAT or UAT to PROD.
Back to the business logic.
Please take a moment to look at the file
client-extensions/ticket-spring-boot/src/main/java/com/liferay/ticket/TicketRestController.java
The key takeaways should be that:
- the body of the request is the payload which contains all the information relevant to the object entry for which the event was triggered
- the endpoint receives and validates JWT tokens which are signed by DXP and issued specifically for the clientId provisioned for the OAuth2Application also specified in the
client-extension.yaml
using the client extension oftype: oAuthApplicationUserAgent
In a separate terminal, execute the following commmand from the root of the workspace to deploy the ticket-spring-boot project and at the same time start the microservice:
(cd client-extensions/ticket-spring-boot/ && ../../gradlew deploy bootRun)
Watch the tomcat logs to see that the client extension deployed.
To witness that the microservice will not allow unauthorized requests run the following curl command in a separate terminal while the microservice is running:
curl -v -X POST http://localhost:58081/ticket/object/action/documentation/referral
Note the response returns an error.
Finally, return to the main page of our site and click the Generate a New Ticket
button. Review the outcome and verify that:
- a ticket was created
- the documentation referrals are added
We've acheived our third business requirement: Implement Programmatic Documentation Referral.
Try making other changes to the projects and redeploying the changes. In the case of the microservice make sure not only to execute the deploy task but also to restart it after any changes.
Now that we can create tickets, at some point, we need to clean them up. Let's create a cron job that will delete all tickets that are marked 'done' or 'duplicate'. To do this we will use a spring boot application that when executed, it will connect to DXP using the generated headless/REST API for ticket objects using another type of client extension type: oAuthApplicationHeadlessServer
. This type of OAuth2 application is using the client credentials flow and is associated with a special account defined for this purpose (the current default uses the instance admin). One thing that we need to know is that client credential flow in OAuth2 require both a client_id and client_secret, so there will be some additional steps to perform in order to get this working locally.
The client-extensions/ticket-cleanup-cron
project's client-extension.yaml
declares a client extension of type: oAuth2ApplicationHeadlessServer
(see OAuth2ApplicationHeadlessServer Client Extension) which defines an OAuth2 Application using client credntials flow.
Execute the following command
./gradlew :client-extensions:ticket-cleanup-cron:deploy
Note: See tomcat log for when the client extension is deployed. Now the oAuthApplication has been created in DXP.
If this were an LXC deployment, the cron schedule is specified in the LCP.json and would be scheduled accordingly. Since we are using a local deployment, we will simulate the cron execution by executing the application ourself. However, since this is a client_credentials type of OAuth2 Application we must provide both the client_id and client_secret. In our sample the code already gets the client_id by looking it up via the external reference code. However, we must copy the secret from the DXP UI.
- Go to the DXP UI and navigate to Control Panel > Security > OAuth 2 Administration
- Select the
Ticket Cleanup Oauth Application Headless Server
application and click on theEdit
button for the Client Secret field. Copy the value. - In the terminal run the following command:
./gradlew :client-extensions:ticket-cleanup-cron:bootRun --args='--ticket-cleanup-oauth-application-headless-server.oauth2.headless.server.client.secret=<PASTE_IN_CLIENT_SECRET>'
Note: when you run the application you should see a message about the number of tickets that were deleted.
2023-06-14 18:18:23.027 INFO 29047 --- [ main] c.l.t.TicketCleanupCommandLineRunner : Amount of tickets: 11
2023-06-14 18:18:23.028 INFO 29047 --- [ main] c.l.t.TicketCleanupCommandLineRunner : Deleting ticket: 44767
2023-06-14 18:18:23.134 INFO 29047 --- [ main] c.l.t.TicketCleanupCommandLineRunner : Deleting ticket: 44795
In order to deploy to LXC, we need the following as requirements:
- LXC extension environment with LCP credentials
- Access to DXP Virtual Instance connected to the LXC extension enviroment
Assuming you have everything above, we can now deploy our extensions to LXC. The following steps will deploy the client extensions to LXC:
- From root workspace run this command:
./gradlew clean build
- Execute
lcp login
and enter your credentials - Execute
lcp deploy --extension <path_to_cx_zip>
and select the LXC environment for each client extension zip
First lets deplay the list-type-batch extension which is the first one we need to deploy since ticket-batch depends on it.
lcp deploy --extension client-extensions/list-type-batch/dist/list-type-batch.zip
In the LCP console logs for this extension wait until you see
Jun 16 16:53:26.429 build-58 [listtypebatch-vhp9k] Execute Status: STARTED
Jun 16 16:53:27.228 build-58 [listtypebatch-vhp9k] Execute Status: COMPLETED
Next lets deploy the ticket-batch extension
lcp deploy --extension client-extensions/ticket-batch/dist/ticket-batch.zip
In the LCP console logs for this extension wait until you see
Jun 16 16:59:24.734 build-59 [ticketbatch-cnhtt] Execute Status: STARTED
Jun 16 16:59:25.532 build-59 [ticketbatch-cnhtt] Execute Status: COMPLETED
Now that we have deployed both of the batch type extensions, lets verify in the DXP UI that our object has been imported.
- Go to the DXP UI and navigate to Control Panel > Object > Objects
- Verify that the Ticket object is listed
Next we can deploy both of the frontend client extensions at the same time.
lcp deploy --extension client-extensions/current-tickets-custom-element/dist/current-tickets-custom-element.zip
lcp deploy --extension client-extensions/ticket-theme-css/dist/ticket-theme-css.zip
Since these are frontend client extensions, the resources will be loaded by the browser, so we need to make sure the client extension workloads (a Caddy fileserver) are visible on the network (which means the dns entries and global loadblancer will resolve the requests). You can view this using the network tag of the LCP Console:
https://console.liferay.cloud/projects/<your_ext_project>/network/endpoints
Wait until you see both the ingress endpoints are green.
Now we can deploy the microservice client extension.
lcp deploy --extension client-extensions/ticket-spring-boot/dist/ticket-spring-boot.zip
If it isn't working, see the troubleshooting section down below. If it is working you should see the servie available and in the logs you should see a message like this:
Jun 16 17:46:26.730 build-65 [ticketspringboot-74fcf56d76-tll5v] 2023-06-16 22:46:26.729 INFO 8 --- [ main] rayOAuth2ResourceServerEnableWebSecurity : Using client ID id-99677fc4-b15d-5968-4a1b-88e63897f9
This means your microservice is correctly talking with DXP and will be able to verify JWT tokens.
If you deploy the batch client extension to the local tomcat/osgi/client-extensions or dockerDeploy before you start the server, you may see an error when it tries to process the batch client extension. This is a known issue where the batch client extension is processed too soon by the headless batch import process. To fix this, simply reploy the batch client extension using gradlew deploy
again.
If you try to deploy ticket-batch
or ticket-entry-batch
client extensions before you deploy the list-type-batch
this will result in an error because ticket-batch
depends on list-type-batch
resources that must be deployed first. This is a known issue that will be addressed in the future.
If you receive a HTTP 401 error or 403 not allowed, this may be because the OAuth2 scopes were not properly applied. To fix this you must edit the OAuthApplication in the DXP control panel UI and go to the "Scopes" tab and make sure the scopes that you are expected to be set, have indeed be set. In this example application is ths Ticket User Agent application
and the Scopes that should be set are the C_Ticket.everything
Here are some possible problems you may run into when deploying to LXC and how to try to troubleshoot them.
If you do not see your microservice client extension is starting (lcp deployment never finishes), it is likely because DXP did not process your client-extension configuration correctly or had some error. You can check the DXP logs to see if there is an error processing your client extension configuration.
It is possible that the DXP environment in the cloud is not configured correctly, namely the DXP virtual instance may work in the UI but the headless apis are not working, perhaps because of some middleware. Ensure that the /o/oauth2
headless apis are working by executing the following command:
curl https://dxp-env.lfr.cloud/o/oauth2/jwks
This should return the JSON Web Key Set (JWKS) for the DXP environment. If it does not, then the headless apis are not working and you will need to troubleshoot the DXP environment.
{"keys":[{"kty":"RSA","kid":"authServer","alg":"RS256","n":...}]}
Here you could use the internal diagnostics tool to try to determine why the microservice is not starting once it is generally available.
If you the LCP console logs for the spring-boot microservice you see that is starts, but it shows that the spring-boot process is being killed like this:
Jun 16 17:22:22.897 build-62 [ticketspringboot-7c9d7f4999-pqcv2] [INFO tini (1)] Spawned child process '/usr/local/bin/liferay_jar_runner_entrypoint.sh' with pid '7'
Jun 16 17:22:22.897 build-62 [ticketspringboot-7c9d7f4999-pqcv2] [INFO tini (1)] Main child exited with signal (with signal 'Terminated')
It could be because the pod does not have enough memory. Edit the client-extensions/ticket-spring-boot/LCP.json
and set the memory to a higher amount and redploy.
./gradlew :client-extensions:ticket-spring-boot:build
lcp deploy --extension client-extensions/ticket-spring-boot/dist/ticket-spring-boot.zip
If the spring boot microservice is starting but is immediately killed, you may see a message like this:
Jun 16 17:22:22.897 build-62 [ticketspringboot-7c9d7f4999-pqcv2] [INFO tini (1)] Spawned child process '/usr/local/bin/liferay_jar_runner_entrypoint.sh' with pid '7'
Jun 16 17:22:22.897 build-62 [ticketspringboot-7c9d7f4999-pqcv2] [INFO tini (1)] Main child exited with signal (with signal 'Terminated')
This may be because LCP could not detect that the service was ready. Review the LCP.json and notice the /ready
path. Ensure that this path is able to respond to the platform within the specified timeout.