After setting up the Wolfram Application Server, there are various modifications that may be applied to customize your deployment. This file documents the commonly used configurations for mofifying the applicaiton server.
It is highly recommended to change the password for the restart API.
The restart api used to force the restart of the Wolfram Engine pools is protected with basic HTTP authentication requiring a user name and password. Wolfram Application Server deploys a default password for the user applicationserver specified in the file Source/ingress/auth on the deployment machine. The format of the password file is generated by the Apache htpasswd applicaiton. To generate a new encrypted password navigate to the ingress directory and run the following command:
htpasswd -c ./auth applicationserver
You will be prompted for a new password. The htpasswd command may additionally be used further customize the password file with new users and passwords as desired.
To deploy the modified auth password file to the cluster run the following command:
kubectl create secret generic basic-auth --from-file=auth -n was --ignore-not-found=true
to apply the changes.
Developers may wish to run Wolfram Language code when Wolfram Engine kenels are initialized prior to handling user requests. By convention this code should be deployed as a node file named init.wl to one of the following paths
.Wolfram/Kernel/init.wl.Wolfram/[pool name]/Kernel/init.wl
Configuration files deployed to the first location will be run for all kernels in all pools. Configurations deployed to the second will be run only for the pool specified by [pool name] to permit customizing initialization code for each pool.
- Active Web Elements Server
- Endpoint Manager
- Resource Manager
- Node Files Manager
- Minio
- Kafka
The auto-scaling of kubernetes upscales and downscale automatically based upon load. Additional pods are started when the pod average cpu usage exceeds 85% or memory usage exceeds 90%. Every 5 minutes the cluster may shut down pods if average load falls below these scaling limits. Up scaling is generally immediate while downscaling is more infrequent to limit the variability in cluster size.
In addition to the default above, the scaling of Active Web Elements Server is based on the pools managed by this service. Pod average statistics of the recent peak percentage of kernels in use as well as the queue length of users waiting to access a kernel are collected for each kernel pool managed by the Active Web Elements Server. If for any pool the average kernel usage exceeds 90% or the average queue length exceeds 2 requests, additional services are launched (as long as the service instance maximum has not been reached). If all the pools average less than 80% kernel usage and queue lengths less than one then pods will be shut down.
Note: "Average" here means average across active pods of that particular type.
By default, there are 2 replicas and they scale up to a maximum of 10.
How does one change these?
For each service you can find the appropriate configruation file in Source/hpa/[service name]-hpa-autoscaler.yaml. You may alter these configuration values, ensuring a minimum of 2 replicas for each pod.
From the configuration file directory run the following command:
kubectl apply -f .
Restart to apply any configuration changes.
A number of cluster environment variables are exposed to configure various aspects of the cluster and its services. The source files containing may be found in source/deployments on the deployment machine. To modify any environment settings modify the value in te appropriate file. Prior to installing the cluster, these modifications will be used during the installation process. To change a value after the cluster is running, navigate to the directory of the updated source file and run the following command:
kubectl apply -f .
These variables may be changed to configure the public URL's to various services offered by the Wolfram Application Server.
applicationserver.servername: "http://domain.com/" - Base URL
applicationserver.resourcemanager.url: "http://domain.com/resources/" - Resource manager URL
applicationserver.nodefilesmanager.url: "http://domain.com/nodefiles/" - Node files manager URL
applicationserver.endpointmanager.url: "http://domain.com/endpoints/" - Endpoint manager URL
applicationserver.restart.url: "http://domain.com/.applicationserver/kernel/restart" - Restart API URL
These variables may be changed to configure the Kafka streaming service.
KAFKA.BOOTSTRAP-SERVERS: kafka-persistent-kafka-bootstrap:9092 - Service and communication port of kafka client
These variables may be changed to modify the internal behavior of the Active Web Elements Server.
APPLICATIONSERVER_CACHEDIRECTORY: /tmp/.wolframcache - Root directory for locally cached resources
applicationserver.nodefiles.cachedirectory: /opt/.wolframcache/nodefiles/ - Root directory for all deployed node files
These variables are used to configure pools of Wolfram Engines in the Active Web Elements Server. These kernel pool configurations are stored in a zero offset array where values for each pool follow the naming convention poolconfiguration_kernelpool_<n>__ with <n> being the indext of pool being configured. A minimum of one pool must be configured and it is recommended that at least one pool be named Public as this is the pool which will be selected by default when deployed resources do not explicitly declare a pool to use.
The following variables configure a default Public pool suitable for serving active web element resources:
poolconfiguration_kernelpool_0__KernelNumber: "2" - Number of Wolfram Engines in the pool
poolconfiguration_kernelpool_0__KernelPoolName: Public - Name of this pool
poolconfiguration_kernelpool_0__JLinkEnabled: "false" - Required for serving active web element resources
This second set of variables configure a pool named MSP to serve MSP pages. These resources cannot be served by the previous pool (note that the JLinkEnabled value must be set to "true" for MSP content).
poolconfiguration_kernelpool_1__KernelNumber: "2" - Number of Wolfram Engines in the pool
poolconfiguration_kernelpool_1__KernelPoolName: MSP - Name of this pool
poolconfiguration_kernelpool_1__JLinkEnabled: "true" - Required for serving MSP resources
Additional pools may be configured if desired using sequentially increasing index values.