Deephaven Community Core is a real-time, time-series, column-oriented analytics engine with relational database features. Queries can seamlessly operate upon both historical and real-time data. Deephaven includes an intuitive user experience and visualization tools. It can ingest data from a variety of sources, apply computation and analysis algorithms to that data, and build rich queries, dashboards, and representations with the results.
Deephaven Community Core is the open version of Deephaven Enterprise, which functions as the data backbone for prominent hedge funds, banks, and financial exchanges.
This README is intended to provide a high-level overview of the installation and use of Deephaven Community Core. For more detailed guides on the topics presented below, see our Community documentation.
Language | Server Application | Client Application |
---|---|---|
Python | Yes | Yes |
Java / Groovy | Yes | Yes |
C++ | No | Yes |
JavaScript | No | Yes |
Go | No | Yes |
R | No | Yes |
Deephaven's client APIs use gRPC, protobuf, Apache Arrow Flight, and Barrage to handle ticking data. Users who wish to build their own client APIs can use these tools to do so.
The following list contains documentation links for installation instructions and more:
- Python
- Groovy
- Python client
- Java client
- JS client
- Go client
- R client
The Deephaven server can be installed and instantiated from Docker, from Python, or from source code.
This is the easiest way to get started with Deephaven. For complete instructions, see our quickstart for Docker. The table below shows installation dependencies.
Dependency | Version | OS | Required/Recommended |
---|---|---|---|
Docker | ^20.10.8 | All | Required |
Docker compose | ^2 | All | Recommended |
Windows | 10+ | Windows | Required |
WSL | ^2 | Windows | Required |
The quickest way to install and run Deephaven from Docker is with a single Docker command:
Python without Docker Compose
# Python
docker run --rm --name deephaven -p 10000:10000 ghcr.io/deephaven/server:latest
Groovy without Docker Compose
# Groovy
docker run --rm name deephaven -p 10000:10000 ghcr.io/deephaven/server-slim:latest
Users who wish to customize their deployment should use Docker Compose. Deephaven offers a multitude of pre-made docker-compose.yml files to choose from. To get started, all that's required is to download a file, pull the images, and start the server.
Python with Docker Compose
The base Python docker-compose.yml
file can be found here.
mkdir deephaven-deployment
cd deephaven-deployment
curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml
docker compose pull
docker compose up
Groovy with Docker Compose
The base Groovy docker-compose.yml
file can be found here.
mkdir deephaven-deployment
cd deephaven-deployment
curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy/docker-compose.yml
docker compose pull
docker compose up
Users who wish to use Python but not Docker should use pip-installed Deephaven. For users with Windows operating systems, WSL is not required to use Deephaven this way.
pip install --upgrade pip setuptools wheel
pip install deephaven-server deephaven-ipywidgets
Then, from Python:
from deephaven_server import Server
s = Server(port=10000, jvm_args=["-Xmx4g"]).start()
The input arguments to Server
specify to bind to the Deephaven server on port 10000
and to allocate 4GB of memory to the server JVM.
Users who wish to modify source code and contribute to the project should build Deephaven from source. For complete instructions, see How to build Deephaven from source.
Building and running Deephaven requires a few software packages.
Package | Version | OS | Required/Recommended |
---|---|---|---|
git | ^2.25.0 | All | Required |
java | >=11, <=22 | All | Required |
docker | ^20.10.8 | All | Required |
docker compose | ^2 | All | Recommended |
Windows | 10 (OS build 20262 or higher) | Only Windows | Required |
WSL | 2 | Only Windows | Required |
You can check if these packages are installed and functioning by running:
git version
java -version
docker version
docker compose version
docker run hello-world
NOTE: Internally, the Java build process will use Gradle Auto Provisioning to download and use the appropriate Java version for building and testing.
NOTE: On Windows, all commands must be run inside a WSL 2 terminal.
A Python virtual environment is highly recommended for building Deephaven from source. Additionally, the wheel is installed with pip and built with Gradle.
git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
python3 -m venv /tmp/my-dh-venv
source /tmp/my-dh-venv/bin/activate
./gradlew py-server:assemble
pip install "py/server/build/wheel/deephaven_core-<version>-py3-non-any.whl[autocomplete]
./gradlew server-jetty-app:run
The Groovy server is built with Gradle. -Pgroovy
builds the Groovy server instead of Python.
git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
./gradlew server-jetty-app:run -Pgroovy
You can debug the server by adding the -Pdebug
flag, and then attaching a debugger to port 5005. This can be used in conjunction with other flags. For example, if you wanted to debug a server and startup with Groovy:
./gradlew server-jetty-app:run -Pgroovy -Pdebug
Deephaven, by default, uses pre-shared key authentication to authenticate against unauthorized access.
The pre-shared key is printed to the Docker logs when the server is started. Set your own key with the configuration parameter -Dauthentication.psk=<YourKey>
. For users running Deephaven via Docker, this is set in the environment
section of a docker-compose.yml
file, or as a space-separated configuration parameter at the end of the docker run
command.
To find the pre-shared key in the Docker logs:
docker compose logs -f | grep "access through pre-shared key"
When a Deephaven server is started from Python, executing Deephaven queries from Python does not require the key. However, if you wish to connect to the IDE via your web browser, you will need the pre-shared key. You will not be able to get the pre-shared key unless you set it yourself. To set the pre-shared key, add "-Dauthentication.psk=<YourKey>"
as an additional JVM parameter to the server. The following example sets the key to MyPreSharedKey
:
from deephaven_server import Server
s = Server(port=10000, jvm_args=["-Xmx4g", "-Dauthentication.psk=MyPreSharedKey"]).start()
Clients that attempt to connect to a server using pre-shared key authentication will need to supply the key to complete the connection. The key is the same for a client connection as it is for connecting directly to the server. For instance, in the above example, the key for a client connection would also be MyPreSharedKey
.
The Deephaven UI is accessible from a web browser. For a server running locally on port 10000, it can be connected to via https://localhost:10000/ide
. For a server running remotely on port 10000, it can be connected to via https://<hostname>:10000/ide
. If using authentication, enter credentials to gain access to the IDE. For information on supported browsers, see here.
From the Deephaven IDE, you can perform your first query.
The scripts below create two small tables: one for employees and one for departments. They are joined on the DeptID
column to show the name of the department where each employee works.
from deephaven import new_table
from deephaven.column import string_col, int_col
from deephaven.constants import NULL_INT
left = new_table([
string_col("LastName", ["Rafferty", "Jones", "Steiner", "Robins", "Smith", "Rogers"]),
int_col("DeptID", [31, 33, 33, 34, 34, NULL_INT]),
string_col("Telephone", ["(347) 555-0123", "(917) 555-0198", "(212) 555-0167", "(952) 555-0110", None, None])
])
right = new_table([
int_col("DeptID", [31, 33, 34, 35]),
string_col("DeptName", ["Sales", "Engineering", "Clerical", "Marketing"]),
string_col("Telephone", ["(646) 555-0134", "(646) 555-0178", "(646) 555-0159", "(212) 555-0111"])
])
t = left.join(right, "DeptID", "DeptName, DeptTelephone=Telephone")
left = newTable(
string_col("LastName", "Rafferty", "Jones", "Steiner", "Robins", "Smith", "Rogers"),
int_col("DeptID", 31, 33, 33, 34, 34, NULL_INT),
string_col("Telephone", "(347) 555-0123", "(917) 555-0198", "(212) 555-0167", "(952) 555-0110", null, null)
)
right = newTable(
intCol("DeptID", 31, 33, 34, 35),
stringCol("DeptName", "Sales", "Engineering", "Clerical", "Marketing"),
stringCol("Telephone", "(646) 555-0134", "(646) 555-0178", "(646) 555-0159", "(212) 555-0111")
)
t = left.join(right, "DeptID", "DeptName, DeptTelephone=Telephone")
- Help!
- Deephaven Community Slack
- Discussions
- Deephaven Community Core docs
- Java API docs
- Python API docs
See CONTRIBUTING for full instructions on how to contribute to this project.
This project has adopted the Contributor Covenant Code of Conduct. For more information see the Code of Conduct or contact opencode@deephaven.io with any additional questions or comments.
Copyright © 2016-2023 Deephaven Data Labs and Patent Pending. All rights reserved.
Provided under the Deephaven Community License.