It can also be more elaborated: e.g. binary compilation, installation of subroutines...
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "**NOTE:** It replaces the dependencies on the HPC (e.g. `modules`) and makes it more transferable to other e.g. institutions"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# Git and Gitlab"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What is git? \n",
+ "Is a free and open source *Distributed* **Version Control System**:\n",
+ "- **Version Control System**: a system that tracks and provides control over changes to source code, documentation and configuration files.\n",
+ "- *Distributed*: a form of version control in which the complete codebase, including its full history, is mirrored on every developer's computer\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What is gitlab? \n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
GitLab is a web-based tool: provides version control, issue tracking, CI/CD etc.
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva:\n",
+ "\n",
+ "- One central local repository for the plugged plugins\n",
+ "- Other local repositories for different developers\n",
+ "- One remote repository for the plugin at gitlab\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: location of the plugins\n",
+ "\n",
+ "- Under [https://gitlab.dkrz.de/freva/plugins4freva](https://gitlab.dkrz.de/freva/plugins4freva) project:\n",
+ "\n",
+ "- Other projects have also their own, e.g.: \n",
+ " - climxtreme: https://gitlab.dkrz.de/bm1159/plugins4freva\n",
+ " - regiklim-nukleus: https://gitlab.dkrz.de/ch1187/plugins4freva\n",
+ "- some plugins can be found in different projects (see [forks](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html)).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: cloning a repository\n",
+ "To download a repository there are 2 ways: via `https` and via `ssh`:\n",
+ "\n",
+ "- `https`: straightforward, requires to enter (dkrz) username and password (usually). \n",
+ "- `ssh`: convenient, requires to provide a ssh key previously.\n",
+ "\n",
+ "In both cases:\n",
+ "\n",
+ "```\n",
+ "$ ssh @levante.dkrz.de\n",
+ "> cd /path/to/my/work/\n",
+ "> git clone https://gitlab.dkrz.de/freva/plugins4freva/plugin_workshop/example_plugin.git\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: setting your SSH key\n",
+ "1. generate a ssh a private and public (`.pub`) key locally (e.g. laptop or levante):\n",
+ "```\n",
+ "$ ssh-keygen -t ed25519\n",
+ "$ cd ~/.ssh/\n",
+ "$ ls\n",
+ "id_ed25519 id_ed25519.pub\n",
+ "```\n",
+ "2. copy the **public** key: \n",
+ "```\n",
+ "$ cat id_ed25519.pub\n",
+ "ssh-ed25519 AAAAC3NzaC1lFDI1YTE5AAAAICUqpXx8AyrNH7HQgPSq9LuB7sTEmXP4ODqOnaHVPPQ6 username@levante\n",
+ "```\n",
+ "3. add the public key in gitlab (`user>preferences>ssh keys`): "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: creating a new project\n",
+ "- If we already had an existing Git repository:\n",
+ "```shell\n",
+ "$ cd NEW_PLUGIN # for example\n",
+ "$ git remote rename origin old-origin\n",
+ "$ git remote add origin git@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git\n",
+ "$ git push -u origin --all\n",
+ "$ git push -u origin --tags\n",
+ "```\n",
+ "\n",
+ "- If we did not already have an existing Git repository:\n",
+ "```shell\n",
+ "$ cd NEW_PLUGIN # for example\n",
+ "$ git init\n",
+ "$ git remote add origin git@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git\n",
+ "# create your new files and folders\n",
+ "$ git add .\n",
+ "$ git commit -m \"Initial commit of my new project\"\n",
+ "$ git push -u origin master\n",
+ "```\n",
+ "\n",
+ "Now you could (hypothetically) go to https://gitlab.dkrz.de/freva/plugins4freva/new_plugin"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# License of Plugins"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What kind of License should I follow?\n",
+ "\n",
+ "[The Freva code](https://github.com/FREVA-CLINT/freva/blob/main/LICENSE.md) follows a [BSD3 license](https://opensource.org/license/bsd-3-clause), a permissive open-source license with minimal restrictions:\n",
+ "- It allows for wide usage, including in proprietary software.\n",
+ "- it imposes very few restrictions on how the software can be redistributed.\n",
+ "\n",
+ "In principle, any derived product from Freva (i.e. plugins), should follow this license. *Unless* stated otherwise by the developer.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What kind of License should I follow?\n",
+ "\n",
+ "- We encourage open-source software to foster collaboration within and outside your projects.\n",
+ "- That said, common courtesy should be expected, e.g.:\n",
+ " - reference to the first manuscript where the developer used the plugin (the plugin documentation is useful for this!).\n",
+ " - contacting him/her in case there is no such manuscript."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# Connecting your IDE / Code Editor to a remote session using SSH"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 1. VS-Code \n",
+ "\n",
+ "Open a new window in VS Code. There are several ways to connect to a remote session, as indicated in the picture below:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Clicking either of the buttons opens a prompt at the top of the window, giving you several options to start a remote session:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Click \"Connect Current Window to Host...\" (\"Connect to Host...\" opens a new window for the connection).\n",
+ "\n",
+ "You can type in the address of the remote host that you want to connect to or, in case the host is specified in your ssh config file (.ssh/config or ssh_config), select them from the list of hosts. To connect to levante, enter [k-number@]levante.dkrz.de :\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "The first time connecting to a host, you will be prompted to specify the OS of the remote host (i.e. Linux / Windows / macOS).\n",
+ "\n",
+ "Depending on the authentication set up for the remote host, you might be prompted to type in a password. Once the connection has been established, your can open a folder on the remote host, like you would do on your local machine (by pressing Ctrl + K, Ctrl + O or by pressing \"Open Folder\"):\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "You may be prompted again to type in the password at this point. The window should refresh and give you access to the remote folder that you selected in the previous step, similar to what can be seen in the following image:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "From here you can use the editor to work with remote files basically as you would with files on your local machine. Any terminals opened in this window will also connect to the remote host.\n",
+ "\n",
+ "For detailed documentation on remote development using VS Code, including development on dev containers or WSL, click [here](https://code.visualstudio.com/docs/remote/remote-overview)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 2. PyCharm \n",
+ "\n",
+ "\n",
+ "> 😱\n",
+ ">Remote Development is currently in beta and only supported for the Professional Version of PyCharm!\n",
+ "\n",
+ "Open your Plugins (Ctrl+Alt+S) and confirm that \"Remote Development Gateway\" is installed and that it is enabled:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "To open a remote session in PyCharm, go to the welcome screen and click on the \"Remote Development\" Tab in the left side bar. This should give you a number of options, including the option to connect to a remote session using SSH:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "To establish a new SSH Connection, click \"New Connection\" below the SSH Connection option. This should open the following dialogue:\n",
+ "\n",
+ "\n",
+ ">⚠\n",
+ ">Note that PyCharm currently only supports connecting to remote hosts running Linux."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Again, depending on the type of authentication set up by the remote host, you may be prompted to enter a password to establish the remote connection.\n",
+ "\n",
+ "In the next step (pending the successful connection to the remote host), you will be asked to set the IDE version (i.e. the version of the IDE that will be installed on the remote host) and the project directory (i.e. the directory on the remote host that contains the project files that you will be working with):\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Click \"Download IDE and Connect\". The installation of the selected IDE on the remote host might take a while. Once the installation is complete, a window containing the remote IDE should open.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "The remote IDE gives you the ability to work with project files, much like you would with local project files. Any terminal sessions started in this IDE will also be connected to the remote host.\n",
+ "\n",
+ "For detailed documentation on remote development using PyCharm, click [here](https://www.jetbrains.com/help/pycharm/remote-development-starting-page.html)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 3. R Studio \n",
+ "\n",
+ "There is currently only one method of using R Studio on a remote host that is officially supported by R, namely by connecting to an instance of R Studio Server running on the remote host. The previously available alternative, by creating a remote connection in [R Studio Desktop Pro](https://docs.posit.co/ide/desktop-pro/2022.02.1+461.pro1/remote-desktop.html), has been removed since version [2023.06.2](https://docs.posit.co/ide/desktop-pro/remote_desktop/remote_desktop.html).\n",
+ "\n",
+ "However, R Studio Server is not available on Levante, so creating a remote development environment instance this way is not possible. However, it is possible to remotely develop and execute R-code by using the [VS Code R Extension](https://code.visualstudio.com/docs/languages/r). "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "First, connect to Levante as outlined in the VS Code section of this guide. Once the connection is established, you will want to install the R (`REditorSupport.r`) and R Debugger (`RDebugger.r-debugger`) Extensions, as shown below:\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Make sure that you have installed the extensions to the remote host, and not your local machine, by going to Extensions (Ctrl+Shift+X):\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "In a terminal that is connected to levante via ssh (e.g. in the VS Code window connected to levante), type in the following command:\n",
+ "```bash\n",
+ "$ module load r\n",
+ "```\n",
+ "The result should be something along the lines of:\n",
+ "```bash\n",
+ "[k202195@levante3 ~]$ module load r\n",
+ "Autoloading openjdk/17.0.0_35-gcc-11.2.0\n",
+ "\n",
+ "Loading r/4.1.2-gcc-11.2.0\n",
+ " Loading requirement: openjdk/17.0.0_35-gcc-11.2.0\n",
+ "[k202195@levante3 ~]$\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "We'll need to find out, where R is located on the remote host. We do this by using the command:\n",
+ "```bash\n",
+ "$ which R\n",
+ "```\n",
+ "The output should be something like:\n",
+ "```bash\n",
+ "[k202195@levante3 ~]$ which R\n",
+ "/sw/spack-levante/r-4.1.2-eprwea/bin/R\n",
+ "[k202195@levante3 ~]$\n",
+ "```\n",
+ "Copy this path to your clipboard."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Install the [httpgd](https://github.com/nx10/httpgd) package to embed live R graphics into the VS Code environment. You can do this by running the following commands in a terminal on the remote host:\n",
+ "```bash\n",
+ "# Start and R command line session\n",
+ "$ R\n",
+ "```\n",
+ "The following command is executed in the R command line session\n",
+ "```R\n",
+ "install.packages(\"httpgd\")\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Open the settings (Ctrl + ,) and type `r.rterm.linux` in the search bar as shown in the image below. Make sure that you paste the previously copied path into the field, as depicted:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Once that is done, open any R file on the remote host. Once an R terminal is ready, you could either select the code or put the cursor at the beginning or ending of the code you want to run, press (Ctrl+Enter), and then code will be sent to the active R terminal.\n",
+ "\n",
+ "If you want to run an entire R file, open the file in the editor, and press Ctrl+Shift+S and the file will be sourced in the active R terminal:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "For more information on working with R in VS Code, click [here](https://code.visualstudio.com/docs/languages/r)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# What is a Freva Wrapper?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "
Freva Wrapper Main Classes
\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
PluginAbstract Class Functions
\n",
+ " \n",
+ "
parameters.py Classes
\n",
+ " \n",
+ "
_ConfigWrapper Class
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Special Variables:\n",
+ "\n",
+ "| Variables | Description |\n",
+ "|--------------------|------------------------------------------------------------------------|\n",
+ "| USER_BASE_DIR | Absolute path to the central directory for this user. |\n",
+ "| USER_OUTPUT_DIR | Absolute path to where the plugin outputs for this user are stored. |\n",
+ "| USER_PLOTS_DIR | Absolute path to where the plugin plots for this user are stored. |\n",
+ "| USER_CACHE_DIR | Absolute path to the cached data (temp data) for this user. |\n",
+ "| USER_UID | The users’ User Identifier. |\n",
+ "| SYSTEM_DATE | Current date in the form YYYYMMDD (e.g. 20120130). |\n",
+ "| SYSTEM_DATETIME | Current date in the form YYYYMMDD_HHmmSS (e.g. 20120130_101123). |\n",
+ "| SYSTEM_TIMESTAMP | Milliseconds since epoch (i.e. e.g. 1358929581838). |\n",
+ "| SYSTEM_RANDOM_UUID | A random Universal Unique Identifier (e.g. 912cca21-6364-4f46-9b03-4263410c9899). |\n",
+ "\n",
+ "Example: \n",
+ "\n",
+ "```bash\n",
+ "output_file='$USER_OUTPUT_DIR/myfile_${SYSTEM_DATETIME}blah.nc'\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: class_basedir\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " \"\"\"Base class that is used as a template for all Freva plugins.\n",
+ " Any api wrapper class defining Freva plugins must inherit from this class.\"\"\"\n",
+ "\n",
+ " @deprecated_method(\"PluginAbstract\", \"class_basedir\")\n",
+ " def getClassBaseDir(self) -> Optional[str]:\n",
+ " \"\"\"Deprecated method for class_basedir.\"\"\"\n",
+ "\n",
+ " def class_basedir(self) -> str:\n",
+ " \"\"\"Get absolute path to the module defining the plugin class.\"\"\"\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: call\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " \"\"\"Base class that is used as a template for all Freva plugins.\n",
+ " Any api wrapper class defining Freva plugins must inherit from this class.\"\"\"\n",
+ "\n",
+ " def call(\n",
+ " self,\n",
+ " cmd_string: Union[str, list[str]],\n",
+ " check: bool = True,\n",
+ " **kwargs,\n",
+ " ) -> sub.Popen[Any]:\n",
+ " \"\"\"Run command with arguments and return a CompletedProcess instance.\"\"\"\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: call and class_basedir - Example\n",
+ "```python\n",
+ "from evaluation_system.api import plugin\n",
+ "class TemplatePlugin(plugin.PluginAbstract):\n",
+ " def run_tool(self, config_dict=None)\n",
+ " self.call(\n",
+ " f\"{self.class_basedir}/main_code.sh {config_dict}\"\n",
+ " )\n",
+ " # or\n",
+ " self.call(\n",
+ " f\"{Path(__file__).parent}/main_code.sh {config_dict}\"\n",
+ " )\n",
+ " # or\n",
+ " self.call(\n",
+ " f\"{os.getenv(USER_BASE_DIR)}/main_code.sh {config_dict}\"\n",
+ " ) \n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: add_output_to_databrowser\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " @deprecated_method(\"PluginAbstract\", \"add_output_to_databrowser\")\n",
+ " def linkmydata(self, *args, **kwargs): # pragma: no cover\n",
+ "\n",
+ " def add_output_to_databrowser(\n",
+ " self,\n",
+ " plugin_output: os.PathLike,\n",
+ " project: str, # user-\n",
+ " product: str, # project.product\n",
+ " *,\n",
+ " model: str = \"freva\",\n",
+ " institute: Optional[str] = None,\n",
+ " ensemble: str = \"r1i1p1\",\n",
+ " time_frequency: Optional[str] = None,\n",
+ " variable: Optional[str] = None,\n",
+ " experiment: Optional[str] = None,\n",
+ " index_data: bool = True,\n",
+ " ) -> Path:\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: add_output_to_databrowser - Example\n",
+ "```python\n",
+ "from evaluation_system.api import plugin\n",
+ "class TemplatePlugin(plugin.PluginAbstract):\n",
+ " def run_tool(self, config_dict=None)\n",
+ " self.add_output_to_databrowser(\n",
+ " plugin_output=config_dict[\"outputdir\"],\n",
+ " project=config_dict.get(\"project\", \"default_project\"),\n",
+ " product=config_dict.get(\"product\", \"default_product\"),\n",
+ " model=config_dict.get(\"model\", \"default_model\"),\n",
+ " institute=config_dict.get(\"institute\", \"default_institute\"),\n",
+ " time_frequency=\"mon\",\n",
+ " )\n",
+ "```\n",
+ "\n",
+ "### How to find it in Freva:\n",
+ "\n",
+ "```bash\n",
+ "freva databrowser product=project.product project=user-\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: prepare_output\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " \"\"\"Base class that is used as a template for all Freva plugins.\n",
+ " Any api wrapper class defining Freva plugins must inherit from this class.\"\"\"\n",
+ "\n",
+ " @deprecated_method(\"PluginAbstract\", \"prepare_output\")\n",
+ " def prepareOutput(self, *args) -> dict[str, dict[str, str]]:\n",
+ " \"\"\"Deprecated method for :class:`prepare_output`. \"\"\"\n",
+ "\n",
+ " def prepare_output(\n",
+ " self, output_files: Union[str, list[str], dict[str, dict[str, str]]]\n",
+ " ) -> dict[str, dict[str, str]]:\n",
+ " \"\"\"Prepare output for files supposedly created.\"\"\"\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: prepare_output - Example\n",
+ "```python\n",
+ "from evaluation_system.api import plugin\n",
+ "class TemplatePlugin(plugin.PluginAbstract):\n",
+ " def run_tool(self, config_dict=None)\n",
+ " return self.prepare_output(config_dict[\"outputdir\"])\n",
+ "```\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# ParameterType Class\n",
+ "```python\n",
+ "class ParameterType(initOrder):\n",
+ " \"\"\"Base class for all parameter types.\n",
+ " All available parameter types inherit from this class. The class creates\n",
+ " a parameter object holding the following user defined information.\n",
+ " \"\"\"\n",
+ " def __init__(\n",
+ " self,\n",
+ " name=None,\n",
+ " default=None,\n",
+ " mandatory=False,\n",
+ " max_items=1,\n",
+ " item_separator=\",\",\n",
+ " regex=None,\n",
+ " version=1,\n",
+ " help=\"No help available.\",\n",
+ " print_format=\"%s\",\n",
+ " impact=Parameter.Impact.affects_values,\n",
+ " ):\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# ParameterDictionary Class\n",
+ "```python\n",
+ "class ParameterDictionary(dict):\n",
+ " \"\"\"Directory holding all plugin parameters for a ``Freva`` plugin.\n",
+ " This class behaves like a built-in ``dict`` with additional features.\n",
+ " The most prominent feature is that the order of added items is preserved,\n",
+ " as opposed to a normal build-in ``dict``.\n",
+ " \"\"\"\n",
+ " def __init__(self, *parameters: ParameterType) -> None:\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# _ConfigWrapper class\n",
+ "```python\n",
+ "class _ConfigWrapper:\n",
+ " \"\"\"Convenience class that helps to dynamically set the location of the\n",
+ " evaluation system config file.\"\"\"\n",
+ " _env: str = \"EVALUATION_SYSTEM_CONFIG_FILE\"\n",
+ " def __init__(\n",
+ " self,\n",
+ " default_file: str,\n",
+ " ):\n",
+ " self.default_file = default_file\n",
+ " def __fspath__(self) -> str:\n",
+ " return os.environ.get(self._env, self.default_file)\n",
+ " def __repr__(self) -> str:\n",
+ " return self.__fspath__()\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Plug a tool into Freva\n",
+ "\n",
+ "In this step we need to clone a plugin in the user directory of freva-dev and replace `` with your `k-number` or `b-number` username:\n",
+ "\n",
+ "```sh\n",
+ "ssh @levante.dkrz.de\n",
+ "#After login to Levante\n",
+ "cd work/ch1187/clint/freva-dev/share/work//freva-dev/\n",
+ "git clone https://gitlab.dkrz.de/freva/plugins4freva/plugin_workshop/example_plugin.git\n",
+ "```\n",
+ "In this step you need to install the dependencies. Have a look at the previous section (How To Install All Needed Libraries Of A Plugin)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Plug a tool into Freva\n",
+ "\n",
+ "\n",
+ "
It can also be more elaborated: e.g. binary compilation, installation of subroutines...
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "**NOTE:** It replaces the dependencies on the HPC (e.g. `modules`) and makes it more transferable to other e.g. institutions"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# Git and Gitlab"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What is git? \n",
+ "Is a free and open source *Distributed* **Version Control System**:\n",
+ "- **Version Control System**: a system that tracks and provides control over changes to source code, documentation and configuration files.\n",
+ "- *Distributed*: a form of version control in which the complete codebase, including its full history, is mirrored on every developer's computer\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What is gitlab? \n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
GitLab is a web-based tool: provides version control, issue tracking, CI/CD etc.
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva:\n",
+ "\n",
+ "- One central local repository for the plugged plugins\n",
+ "- Other local repositories for different developers\n",
+ "- One remote repository for the plugin at gitlab\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: location of the plugins\n",
+ "\n",
+ "- Under [https://gitlab.dkrz.de/freva/plugins4freva](https://gitlab.dkrz.de/freva/plugins4freva) project:\n",
+ "\n",
+ "- Other projects have also their own, e.g.: \n",
+ " - climxtreme: https://gitlab.dkrz.de/bm1159/plugins4freva\n",
+ " - regiklim-nukleus: https://gitlab.dkrz.de/ch1187/plugins4freva\n",
+ "- some plugins can be found in different projects (see [forks](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html)).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: cloning a repository\n",
+ "To download a repository there are 2 ways: via `https` and via `ssh`:\n",
+ "\n",
+ "- `https`: straightforward, requires to enter (dkrz) username and password (usually). \n",
+ "- `ssh`: convenient, requires to provide a ssh key previously.\n",
+ "\n",
+ "In both cases:\n",
+ "\n",
+ "```\n",
+ "$ ssh @levante.dkrz.de\n",
+ "> cd /path/to/my/work/\n",
+ "> git clone https://gitlab.dkrz.de/freva/plugins4freva/plugin_workshop/example_plugin.git\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: setting your SSH key\n",
+ "1. generate a ssh a private and public (`.pub`) key locally (e.g. laptop or levante):\n",
+ "```\n",
+ "$ ssh-keygen -t ed25519\n",
+ "$ cd ~/.ssh/\n",
+ "$ ls\n",
+ "id_ed25519 id_ed25519.pub\n",
+ "```\n",
+ "2. copy the **public** key: \n",
+ "```\n",
+ "$ cat id_ed25519.pub\n",
+ "ssh-ed25519 AAAAC3NzaC1lFDI1YTE5AAAAICUqpXx8AyrNH7HQgPSq9LuB7sTEmXP4ODqOnaHVPPQ6 username@levante\n",
+ "```\n",
+ "3. add the public key in gitlab (`user>preferences>ssh keys`): "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: creating a new project\n",
+ "- If we already had an existing Git repository:\n",
+ "```shell\n",
+ "$ cd NEW_PLUGIN # for example\n",
+ "$ git remote rename origin old-origin\n",
+ "$ git remote add origin git@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git\n",
+ "$ git push -u origin --all\n",
+ "$ git push -u origin --tags\n",
+ "```\n",
+ "\n",
+ "- If we did not already have an existing Git repository:\n",
+ "```shell\n",
+ "$ cd NEW_PLUGIN # for example\n",
+ "$ git init\n",
+ "$ git remote add origin git@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git\n",
+ "# create your new files and folders\n",
+ "$ git add .\n",
+ "$ git commit -m \"Initial commit of my new project\"\n",
+ "$ git push -u origin master\n",
+ "```\n",
+ "\n",
+ "Now you could (hypothetically) go to https://gitlab.dkrz.de/freva/plugins4freva/new_plugin"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# License of Plugins"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What kind of License should I follow?\n",
+ "\n",
+ "[The Freva code](https://github.com/FREVA-CLINT/freva/blob/main/LICENSE.md) follows a [BSD3 license](https://opensource.org/license/bsd-3-clause), a permissive open-source license with minimal restrictions:\n",
+ "- It allows for wide usage, including in proprietary software.\n",
+ "- it imposes very few restrictions on how the software can be redistributed.\n",
+ "\n",
+ "In principle, any derived product from Freva (i.e. plugins), should follow this license. *Unless* stated otherwise by the developer.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What kind of License should I follow?\n",
+ "\n",
+ "- We encourage open-source software to foster collaboration within and outside your projects.\n",
+ "- That said, common courtesy should be expected, e.g.:\n",
+ " - reference to the first manuscript where the developer used the plugin (the plugin documentation is useful for this!).\n",
+ " - contacting him/her in case there is no such manuscript."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# Connecting your IDE / Code Editor to a remote session using SSH"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 1. VS-Code \n",
+ "\n",
+ "Open a new window in VS Code. There are several ways to connect to a remote session, as indicated in the picture below:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Clicking either of the buttons opens a prompt at the top of the window, giving you several options to start a remote session:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Click \"Connect Current Window to Host...\" (\"Connect to Host...\" opens a new window for the connection).\n",
+ "\n",
+ "You can type in the address of the remote host that you want to connect to or, in case the host is specified in your ssh config file (.ssh/config or ssh_config), select them from the list of hosts. To connect to levante, enter [k-number@]levante.dkrz.de :\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "The first time connecting to a host, you will be prompted to specify the OS of the remote host (i.e. Linux / Windows / macOS).\n",
+ "\n",
+ "Depending on the authentication set up for the remote host, you might be prompted to type in a password. Once the connection has been established, your can open a folder on the remote host, like you would do on your local machine (by pressing Ctrl + K, Ctrl + O or by pressing \"Open Folder\"):\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "You may be prompted again to type in the password at this point. The window should refresh and give you access to the remote folder that you selected in the previous step, similar to what can be seen in the following image:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "From here you can use the editor to work with remote files basically as you would with files on your local machine. Any terminals opened in this window will also connect to the remote host.\n",
+ "\n",
+ "For detailed documentation on remote development using VS Code, including development on dev containers or WSL, click [here](https://code.visualstudio.com/docs/remote/remote-overview)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 2. PyCharm \n",
+ "\n",
+ "\n",
+ "> 😱\n",
+ ">Remote Development is currently in beta and only supported for the Professional Version of PyCharm!\n",
+ "\n",
+ "Open your Plugins (Ctrl+Alt+S) and confirm that \"Remote Development Gateway\" is installed and that it is enabled:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "To open a remote session in PyCharm, go to the welcome screen and click on the \"Remote Development\" Tab in the left side bar. This should give you a number of options, including the option to connect to a remote session using SSH:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "To establish a new SSH Connection, click \"New Connection\" below the SSH Connection option. This should open the following dialogue:\n",
+ "\n",
+ "\n",
+ ">⚠\n",
+ ">Note that PyCharm currently only supports connecting to remote hosts running Linux."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Again, depending on the type of authentication set up by the remote host, you may be prompted to enter a password to establish the remote connection.\n",
+ "\n",
+ "In the next step (pending the successful connection to the remote host), you will be asked to set the IDE version (i.e. the version of the IDE that will be installed on the remote host) and the project directory (i.e. the directory on the remote host that contains the project files that you will be working with):\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Click \"Download IDE and Connect\". The installation of the selected IDE on the remote host might take a while. Once the installation is complete, a window containing the remote IDE should open.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "The remote IDE gives you the ability to work with project files, much like you would with local project files. Any terminal sessions started in this IDE will also be connected to the remote host.\n",
+ "\n",
+ "For detailed documentation on remote development using PyCharm, click [here](https://www.jetbrains.com/help/pycharm/remote-development-starting-page.html)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 3. R Studio \n",
+ "\n",
+ "There is currently only one method of using R Studio on a remote host that is officially supported by R, namely by connecting to an instance of R Studio Server running on the remote host. The previously available alternative, by creating a remote connection in [R Studio Desktop Pro](https://docs.posit.co/ide/desktop-pro/2022.02.1+461.pro1/remote-desktop.html), has been removed since version [2023.06.2](https://docs.posit.co/ide/desktop-pro/remote_desktop/remote_desktop.html).\n",
+ "\n",
+ "However, R Studio Server is not available on Levante, so creating a remote development environment instance this way is not possible. However, it is possible to remotely develop and execute R-code by using the [VS Code R Extension](https://code.visualstudio.com/docs/languages/r). "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "First, connect to Levante as outlined in the VS Code section of this guide. Once the connection is established, you will want to install the R (`REditorSupport.r`) and R Debugger (`RDebugger.r-debugger`) Extensions, as shown below:\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Make sure that you have installed the extensions to the remote host, and not your local machine, by going to Extensions (Ctrl+Shift+X):\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "In a terminal that is connected to levante via ssh (e.g. in the VS Code window connected to levante), type in the following command:\n",
+ "```bash\n",
+ "$ module load r\n",
+ "```\n",
+ "The result should be something along the lines of:\n",
+ "```bash\n",
+ "[k202195@levante3 ~]$ module load r\n",
+ "Autoloading openjdk/17.0.0_35-gcc-11.2.0\n",
+ "\n",
+ "Loading r/4.1.2-gcc-11.2.0\n",
+ " Loading requirement: openjdk/17.0.0_35-gcc-11.2.0\n",
+ "[k202195@levante3 ~]$\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "We'll need to find out, where R is located on the remote host. We do this by using the command:\n",
+ "```bash\n",
+ "$ which R\n",
+ "```\n",
+ "The output should be something like:\n",
+ "```bash\n",
+ "[k202195@levante3 ~]$ which R\n",
+ "/sw/spack-levante/r-4.1.2-eprwea/bin/R\n",
+ "[k202195@levante3 ~]$\n",
+ "```\n",
+ "Copy this path to your clipboard."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Install the [httpgd](https://github.com/nx10/httpgd) package to embed live R graphics into the VS Code environment. You can do this by running the following commands in a terminal on the remote host:\n",
+ "```bash\n",
+ "# Start and R command line session\n",
+ "$ R\n",
+ "```\n",
+ "The following command is executed in the R command line session\n",
+ "```R\n",
+ "install.packages(\"httpgd\")\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Open the settings (Ctrl + ,) and type `r.rterm.linux` in the search bar as shown in the image below. Make sure that you paste the previously copied path into the field, as depicted:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "Once that is done, open any R file on the remote host. Once an R terminal is ready, you could either select the code or put the cursor at the beginning or ending of the code you want to run, press (Ctrl+Enter), and then code will be sent to the active R terminal.\n",
+ "\n",
+ "If you want to run an entire R file, open the file in the editor, and press Ctrl+Shift+S and the file will be sourced in the active R terminal:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "\n",
+ "For more information on working with R in VS Code, click [here](https://code.visualstudio.com/docs/languages/r)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# What is a Freva Wrapper?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "
Freva Wrapper Main Classes
\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ "
PluginAbstract Class Functions
\n",
+ " \n",
+ "
parameters.py Classes
\n",
+ " \n",
+ "
_ConfigWrapper Class
\n",
+ "
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Special Variables:\n",
+ "\n",
+ "| Variables | Description |\n",
+ "|--------------------|------------------------------------------------------------------------|\n",
+ "| USER_BASE_DIR | Absolute path to the central directory for this user. |\n",
+ "| USER_OUTPUT_DIR | Absolute path to where the plugin outputs for this user are stored. |\n",
+ "| USER_PLOTS_DIR | Absolute path to where the plugin plots for this user are stored. |\n",
+ "| USER_CACHE_DIR | Absolute path to the cached data (temp data) for this user. |\n",
+ "| USER_UID | The users’ User Identifier. |\n",
+ "| SYSTEM_DATE | Current date in the form YYYYMMDD (e.g. 20120130). |\n",
+ "| SYSTEM_DATETIME | Current date in the form YYYYMMDD_HHmmSS (e.g. 20120130_101123). |\n",
+ "| SYSTEM_TIMESTAMP | Milliseconds since epoch (i.e. e.g. 1358929581838). |\n",
+ "| SYSTEM_RANDOM_UUID | A random Universal Unique Identifier (e.g. 912cca21-6364-4f46-9b03-4263410c9899). |\n",
+ "\n",
+ "Example: \n",
+ "\n",
+ "```bash\n",
+ "output_file='$USER_OUTPUT_DIR/myfile_${SYSTEM_DATETIME}blah.nc'\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: class_basedir\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " \"\"\"Base class that is used as a template for all Freva plugins.\n",
+ " Any api wrapper class defining Freva plugins must inherit from this class.\"\"\"\n",
+ "\n",
+ " @deprecated_method(\"PluginAbstract\", \"class_basedir\")\n",
+ " def getClassBaseDir(self) -> Optional[str]:\n",
+ " \"\"\"Deprecated method for class_basedir.\"\"\"\n",
+ "\n",
+ " def class_basedir(self) -> str:\n",
+ " \"\"\"Get absolute path to the module defining the plugin class.\"\"\"\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: call\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " \"\"\"Base class that is used as a template for all Freva plugins.\n",
+ " Any api wrapper class defining Freva plugins must inherit from this class.\"\"\"\n",
+ "\n",
+ " def call(\n",
+ " self,\n",
+ " cmd_string: Union[str, list[str]],\n",
+ " check: bool = True,\n",
+ " **kwargs,\n",
+ " ) -> sub.Popen[Any]:\n",
+ " \"\"\"Run command with arguments and return a CompletedProcess instance.\"\"\"\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: call and class_basedir - Example\n",
+ "```python\n",
+ "from evaluation_system.api import plugin\n",
+ "class TemplatePlugin(plugin.PluginAbstract):\n",
+ " def run_tool(self, config_dict=None)\n",
+ " self.call(\n",
+ " f\"{self.class_basedir}/main_code.sh {config_dict}\"\n",
+ " )\n",
+ " # or\n",
+ " self.call(\n",
+ " f\"{Path(__file__).parent}/main_code.sh {config_dict}\"\n",
+ " )\n",
+ " # or\n",
+ " self.call(\n",
+ " f\"{os.getenv(USER_BASE_DIR)}/main_code.sh {config_dict}\"\n",
+ " ) \n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: add_output_to_databrowser\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " @deprecated_method(\"PluginAbstract\", \"add_output_to_databrowser\")\n",
+ " def linkmydata(self, *args, **kwargs): # pragma: no cover\n",
+ "\n",
+ " def add_output_to_databrowser(\n",
+ " self,\n",
+ " plugin_output: os.PathLike,\n",
+ " project: str, # user-\n",
+ " product: str, # project.product\n",
+ " *,\n",
+ " model: str = \"freva\",\n",
+ " institute: Optional[str] = None,\n",
+ " ensemble: str = \"r1i1p1\",\n",
+ " time_frequency: Optional[str] = None,\n",
+ " variable: Optional[str] = None,\n",
+ " experiment: Optional[str] = None,\n",
+ " index_data: bool = True,\n",
+ " ) -> Path:\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: add_output_to_databrowser - Example\n",
+ "```python\n",
+ "from evaluation_system.api import plugin\n",
+ "class TemplatePlugin(plugin.PluginAbstract):\n",
+ " def run_tool(self, config_dict=None)\n",
+ " self.add_output_to_databrowser(\n",
+ " plugin_output=config_dict[\"outputdir\"],\n",
+ " project=config_dict.get(\"project\", \"default_project\"),\n",
+ " product=config_dict.get(\"product\", \"default_product\"),\n",
+ " model=config_dict.get(\"model\", \"default_model\"),\n",
+ " institute=config_dict.get(\"institute\", \"default_institute\"),\n",
+ " time_frequency=\"mon\",\n",
+ " )\n",
+ "```\n",
+ "\n",
+ "### How to find it in Freva:\n",
+ "\n",
+ "```bash\n",
+ "freva databrowser product=project.product project=user-\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: prepare_output\n",
+ "```python\n",
+ "class PluginAbstract(abc.ABC):\n",
+ " \"\"\"Base class that is used as a template for all Freva plugins.\n",
+ " Any api wrapper class defining Freva plugins must inherit from this class.\"\"\"\n",
+ "\n",
+ " @deprecated_method(\"PluginAbstract\", \"prepare_output\")\n",
+ " def prepareOutput(self, *args) -> dict[str, dict[str, str]]:\n",
+ " \"\"\"Deprecated method for :class:`prepare_output`. \"\"\"\n",
+ "\n",
+ " def prepare_output(\n",
+ " self, output_files: Union[str, list[str], dict[str, dict[str, str]]]\n",
+ " ) -> dict[str, dict[str, str]]:\n",
+ " \"\"\"Prepare output for files supposedly created.\"\"\"\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# PluginAbstract Class\n",
+ "## Function: prepare_output - Example\n",
+ "```python\n",
+ "from evaluation_system.api import plugin\n",
+ "class TemplatePlugin(plugin.PluginAbstract):\n",
+ " def run_tool(self, config_dict=None)\n",
+ " return self.prepare_output(config_dict[\"outputdir\"])\n",
+ "```\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# ParameterType Class\n",
+ "```python\n",
+ "class ParameterType(initOrder):\n",
+ " \"\"\"Base class for all parameter types.\n",
+ " All available parameter types inherit from this class. The class creates\n",
+ " a parameter object holding the following user defined information.\n",
+ " \"\"\"\n",
+ " def __init__(\n",
+ " self,\n",
+ " name=None,\n",
+ " default=None,\n",
+ " mandatory=False,\n",
+ " max_items=1,\n",
+ " item_separator=\",\",\n",
+ " regex=None,\n",
+ " version=1,\n",
+ " help=\"No help available.\",\n",
+ " print_format=\"%s\",\n",
+ " impact=Parameter.Impact.affects_values,\n",
+ " ):\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# ParameterDictionary Class\n",
+ "```python\n",
+ "class ParameterDictionary(dict):\n",
+ " \"\"\"Directory holding all plugin parameters for a ``Freva`` plugin.\n",
+ " This class behaves like a built-in ``dict`` with additional features.\n",
+ " The most prominent feature is that the order of added items is preserved,\n",
+ " as opposed to a normal build-in ``dict``.\n",
+ " \"\"\"\n",
+ " def __init__(self, *parameters: ParameterType) -> None:\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "# _ConfigWrapper class\n",
+ "```python\n",
+ "class _ConfigWrapper:\n",
+ " \"\"\"Convenience class that helps to dynamically set the location of the\n",
+ " evaluation system config file.\"\"\"\n",
+ " _env: str = \"EVALUATION_SYSTEM_CONFIG_FILE\"\n",
+ " def __init__(\n",
+ " self,\n",
+ " default_file: str,\n",
+ " ):\n",
+ " self.default_file = default_file\n",
+ " def __fspath__(self) -> str:\n",
+ " return os.environ.get(self._env, self.default_file)\n",
+ " def __repr__(self) -> str:\n",
+ " return self.__fspath__()\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Plug a tool into Freva\n",
+ "\n",
+ "In this step we need to clone a plugin in the user directory of freva-dev and replace `` with your `k-number` or `b-number` username:\n",
+ "\n",
+ "```sh\n",
+ "ssh @levante.dkrz.de\n",
+ "#After login to Levante\n",
+ "cd work/ch1187/clint/freva-dev/share/work//freva-dev/\n",
+ "git clone https://gitlab.dkrz.de/freva/plugins4freva/plugin_workshop/example_plugin.git\n",
+ "```\n",
+ "In this step you need to install the dependencies. Have a look at the previous section (How To Install All Needed Libraries Of A Plugin)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Plug a tool into Freva\n",
+ "\n",
+ "\n",
+ "
+
+- Other projects have also their own, e.g.:
+ - climxtreme: https://gitlab.dkrz.de/bm1159/plugins4freva
+ - regiklim-nukleus: https://gitlab.dkrz.de/ch1187/plugins4freva
+- some plugins can be found in different projects (see [forks](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html)).
+
+
To download a repository there are 2 ways: via https and via ssh:
+
+- `https`: straightforward, requires to enter (dkrz) username and password (usually).
+- `ssh`: convenient, requires to provide a ssh key previously.
+
$cdNEW_PLUGIN# for example
+$gitremoterenameoriginold-origin
+$gitremoteaddorigingit@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git
+$gitpush-uorigin--all
+$gitpush-uorigin--tags
+
+
+
If we did not already have an existing Git repository:
+
+
$cdNEW_PLUGIN# for example
+$gitinit
+$gitremoteaddorigingit@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git
+# create your new files and folders
+$gitadd.
+$gitcommit-m"Initial commit of my new project"
+$gitpush-uoriginmaster
+
We encourage open-source software to foster collaboration within and outside your projects.
+
That said, common courtesy should be expected, e.g.:
+
reference to the first manuscript where the developer used the plugin (the plugin documentation is useful for this!).
+
contacting him/her in case there is no such manuscript.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Connecting your IDE / Code Editor to a remote session using SSH¶
+
+
+
+
+
+
+
+
+
+
+
1. VS-Code
+
Open a new window in VS Code. There are several ways to connect to a remote session, as indicated in the picture below:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Clicking either of the buttons opens a prompt at the top of the window, giving you several options to start a remote session:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Click "Connect Current Window to Host..." ("Connect to Host..." opens a new window for the connection).
+
You can type in the address of the remote host that you want to connect to or, in case the host is specified in your ssh config file (.ssh/config or ssh_config), select them from the list of hosts. To connect to levante, enter [k-number@]levante.dkrz.de :
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
The first time connecting to a host, you will be prompted to specify the OS of the remote host (i.e. Linux / Windows / macOS).
+
Depending on the authentication set up for the remote host, you might be prompted to type in a password. Once the connection has been established, your can open a folder on the remote host, like you would do on your local machine (by pressing Ctrl + K, Ctrl + O or by pressing "Open Folder"):
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
You may be prompted again to type in the password at this point. The window should refresh and give you access to the remote folder that you selected in the previous step, similar to what can be seen in the following image:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
From here you can use the editor to work with remote files basically as you would with files on your local machine. Any terminals opened in this window will also connect to the remote host.
+
For detailed documentation on remote development using VS Code, including development on dev containers or WSL, click here.
+
+
+
+
+
+
+
+
+
+
+
2. PyCharm
+
+
😱
+Remote Development is currently in beta and only supported for the Professional Version of PyCharm!
+
+
Open your Plugins (Ctrl+Alt+S) and confirm that "Remote Development Gateway" is installed and that it is enabled:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
To open a remote session in PyCharm, go to the welcome screen and click on the "Remote Development" Tab in the left side bar. This should give you a number of options, including the option to connect to a remote session using SSH:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
To establish a new SSH Connection, click "New Connection" below the SSH Connection option. This should open the following dialogue:
+
+
+
⚠
+Note that PyCharm currently only supports connecting to remote hosts running Linux.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Again, depending on the type of authentication set up by the remote host, you may be prompted to enter a password to establish the remote connection.
+
In the next step (pending the successful connection to the remote host), you will be asked to set the IDE version (i.e. the version of the IDE that will be installed on the remote host) and the project directory (i.e. the directory on the remote host that contains the project files that you will be working with):
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Click "Download IDE and Connect". The installation of the selected IDE on the remote host might take a while. Once the installation is complete, a window containing the remote IDE should open.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
The remote IDE gives you the ability to work with project files, much like you would with local project files. Any terminal sessions started in this IDE will also be connected to the remote host.
+
For detailed documentation on remote development using PyCharm, click here.
+
+
+
+
+
+
+
+
+
+
+
3. R Studio
+
There is currently only one method of using R Studio on a remote host that is officially supported by R, namely by connecting to an instance of R Studio Server running on the remote host. The previously available alternative, by creating a remote connection in R Studio Desktop Pro, has been removed since version 2023.06.2.
+
However, R Studio Server is not available on Levante, so creating a remote development environment instance this way is not possible. However, it is possible to remotely develop and execute R-code by using the VS Code R Extension.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
First, connect to Levante as outlined in the VS Code section of this guide. Once the connection is established, you will want to install the R (REditorSupport.r) and R Debugger (RDebugger.r-debugger) Extensions, as shown below:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Make sure that you have installed the extensions to the remote host, and not your local machine, by going to Extensions (Ctrl+Shift+X):
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
In a terminal that is connected to levante via ssh (e.g. in the VS Code window connected to levante), type in the following command:
+
$moduleloadr
+
+
The result should be something along the lines of:
Install the httpgd package to embed live R graphics into the VS Code environment. You can do this by running the following commands in a terminal on the remote host:
+
# Start and R command line session
+$R
+
+
The following command is executed in the R command line session
+
install.packages("httpgd")
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Open the settings (Ctrl + ,) and type r.rterm.linux in the search bar as shown in the image below. Make sure that you paste the previously copied path into the field, as depicted:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Once that is done, open any R file on the remote host. Once an R terminal is ready, you could either select the code or put the cursor at the beginning or ending of the code you want to run, press (Ctrl+Enter), and then code will be sent to the active R terminal.
+
If you want to run an entire R file, open the file in the editor, and press Ctrl+Shift+S and the file will be sourced in the active R terminal:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
For more information on working with R in VS Code, click here.
classPluginAbstract(abc.ABC):
+"""Base class that is used as a template for all Freva plugins.
+ Any api wrapper class defining Freva plugins must inherit from this class."""
+
+ @deprecated_method("PluginAbstract","class_basedir")
+ defgetClassBaseDir(self)->Optional[str]:
+"""Deprecated method for class_basedir."""
+
+ defclass_basedir(self)->str:
+"""Get absolute path to the module defining the plugin class."""
+
classPluginAbstract(abc.ABC):
+"""Base class that is used as a template for all Freva plugins.
+ Any api wrapper class defining Freva plugins must inherit from this class."""
+
+ defcall(
+ self,
+ cmd_string:Union[str,list[str]],
+ check:bool=True,
+ **kwargs,
+ )->sub.Popen[Any]:
+"""Run command with arguments and return a CompletedProcess instance."""
+
classPluginAbstract(abc.ABC):
+"""Base class that is used as a template for all Freva plugins.
+ Any api wrapper class defining Freva plugins must inherit from this class."""
+
+ @deprecated_method("PluginAbstract","prepare_output")
+ defprepareOutput(self,*args)->dict[str,dict[str,str]]:
+"""Deprecated method for :class:`prepare_output`. """
+
+ defprepare_output(
+ self,output_files:Union[str,list[str],dict[str,dict[str,str]]]
+ )->dict[str,dict[str,str]]:
+"""Prepare output for files supposedly created."""
+
classParameterType(initOrder):
+"""Base class for all parameter types.
+ All available parameter types inherit from this class. The class creates
+ a parameter object holding the following user defined information.
+ """
+ def__init__(
+ self,
+ name=None,
+ default=None,
+ mandatory=False,
+ max_items=1,
+ item_separator=",",
+ regex=None,
+ version=1,
+ help="No help available.",
+ print_format="%s",
+ impact=Parameter.Impact.affects_values,
+ ):
+
classParameterDictionary(dict):
+"""Directory holding all plugin parameters for a ``Freva`` plugin.
+ This class behaves like a built-in ``dict`` with additional features.
+ The most prominent feature is that the order of added items is preserved,
+ as opposed to a normal build-in ``dict``.
+ """
+ def__init__(self,*parameters:ParameterType)->None:
+
class_ConfigWrapper:
+"""Convenience class that helps to dynamically set the location of the
+ evaluation system config file."""
+ _env:str="EVALUATION_SYSTEM_CONFIG_FILE"
+ def__init__(
+ self,
+ default_file:str,
+ ):
+ self.default_file=default_file
+ def__fspath__(self)->str:
+ returnos.environ.get(self._env,self.default_file)
+ def__repr__(self)->str:
+ returnself.__fspath__()
+
We can define a Makefile as below and then after definition of environment.yml, install all the dependencies listed above via make commands. But first and foremost, we need to ensure, we have make installed on our machine.
+
env:
+ @# Installs a conda environment with R and the IDF package
+@if[-d"./plugin_env"];then\
+rm-rf./plugin_env;\
+fi
+condaenvcreate--prefix./plugin_env-fenvironment.yml--force
+Rscript-e'install.packages("IDF", repos="https://CRAN.R-project.org/")'
+cartopy:
+ @# Download various cartopy shape-files
+@if[-d"./cartopy"];then\
+rm-rf./cartopy;\
+fi
+./plugin_env/bin/python3assets/cartopy_download.pyphysicalcultural
+.PHONY:envcartopy
+
+
+
Using mamba instead of using conda can make the way of installation faster.
+
+- Other projects have also their own, e.g.:
+ - climxtreme: https://gitlab.dkrz.de/bm1159/plugins4freva
+ - regiklim-nukleus: https://gitlab.dkrz.de/ch1187/plugins4freva
+- some plugins can be found in different projects (see [forks](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html)).
+
+
To download a repository there are 2 ways: via https and via ssh:
+
+- `https`: straightforward, requires to enter (dkrz) username and password (usually).
+- `ssh`: convenient, requires to provide a ssh key previously.
+
$cdNEW_PLUGIN# for example
+$gitremoterenameoriginold-origin
+$gitremoteaddorigingit@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git
+$gitpush-uorigin--all
+$gitpush-uorigin--tags
+
+
+
If we did not already have an existing Git repository:
+
+
$cdNEW_PLUGIN# for example
+$gitinit
+$gitremoteaddorigingit@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git
+# create your new files and folders
+$gitadd.
+$gitcommit-m"Initial commit of my new project"
+$gitpush-uoriginmaster
+
We encourage open-source software to foster collaboration within and outside your projects.
+
That said, common courtesy should be expected, e.g.:
+
reference to the first manuscript where the developer used the plugin (the plugin documentation is useful for this!).
+
contacting him/her in case there is no such manuscript.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Connecting your IDE / Code Editor to a remote session using SSH¶
+
+
+
+
+
+
+
+
+
+
+
1. VS-Code
+
Open a new window in VS Code. There are several ways to connect to a remote session, as indicated in the picture below:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Clicking either of the buttons opens a prompt at the top of the window, giving you several options to start a remote session:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Click "Connect Current Window to Host..." ("Connect to Host..." opens a new window for the connection).
+
You can type in the address of the remote host that you want to connect to or, in case the host is specified in your ssh config file (.ssh/config or ssh_config), select them from the list of hosts. To connect to levante, enter [k-number@]levante.dkrz.de :
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
The first time connecting to a host, you will be prompted to specify the OS of the remote host (i.e. Linux / Windows / macOS).
+
Depending on the authentication set up for the remote host, you might be prompted to type in a password. Once the connection has been established, your can open a folder on the remote host, like you would do on your local machine (by pressing Ctrl + K, Ctrl + O or by pressing "Open Folder"):
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
You may be prompted again to type in the password at this point. The window should refresh and give you access to the remote folder that you selected in the previous step, similar to what can be seen in the following image:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
From here you can use the editor to work with remote files basically as you would with files on your local machine. Any terminals opened in this window will also connect to the remote host.
+
For detailed documentation on remote development using VS Code, including development on dev containers or WSL, click here.
+
+
+
+
+
+
+
+
+
+
+
2. PyCharm
+
+
😱
+Remote Development is currently in beta and only supported for the Professional Version of PyCharm!
+
+
Open your Plugins (Ctrl+Alt+S) and confirm that "Remote Development Gateway" is installed and that it is enabled:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
To open a remote session in PyCharm, go to the welcome screen and click on the "Remote Development" Tab in the left side bar. This should give you a number of options, including the option to connect to a remote session using SSH:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
To establish a new SSH Connection, click "New Connection" below the SSH Connection option. This should open the following dialogue:
+
+
+
⚠
+Note that PyCharm currently only supports connecting to remote hosts running Linux.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Again, depending on the type of authentication set up by the remote host, you may be prompted to enter a password to establish the remote connection.
+
In the next step (pending the successful connection to the remote host), you will be asked to set the IDE version (i.e. the version of the IDE that will be installed on the remote host) and the project directory (i.e. the directory on the remote host that contains the project files that you will be working with):
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Click "Download IDE and Connect". The installation of the selected IDE on the remote host might take a while. Once the installation is complete, a window containing the remote IDE should open.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
The remote IDE gives you the ability to work with project files, much like you would with local project files. Any terminal sessions started in this IDE will also be connected to the remote host.
+
For detailed documentation on remote development using PyCharm, click here.
+
+
+
+
+
+
+
+
+
+
+
3. R Studio
+
There is currently only one method of using R Studio on a remote host that is officially supported by R, namely by connecting to an instance of R Studio Server running on the remote host. The previously available alternative, by creating a remote connection in R Studio Desktop Pro, has been removed since version 2023.06.2.
+
However, R Studio Server is not available on Levante, so creating a remote development environment instance this way is not possible. However, it is possible to remotely develop and execute R-code by using the VS Code R Extension.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
First, connect to Levante as outlined in the VS Code section of this guide. Once the connection is established, you will want to install the R (REditorSupport.r) and R Debugger (RDebugger.r-debugger) Extensions, as shown below:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Make sure that you have installed the extensions to the remote host, and not your local machine, by going to Extensions (Ctrl+Shift+X):
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
In a terminal that is connected to levante via ssh (e.g. in the VS Code window connected to levante), type in the following command:
+
$moduleloadr
+
+
The result should be something along the lines of:
Install the httpgd package to embed live R graphics into the VS Code environment. You can do this by running the following commands in a terminal on the remote host:
+
# Start and R command line session
+$R
+
+
The following command is executed in the R command line session
+
install.packages("httpgd")
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Open the settings (Ctrl + ,) and type r.rterm.linux in the search bar as shown in the image below. Make sure that you paste the previously copied path into the field, as depicted:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Once that is done, open any R file on the remote host. Once an R terminal is ready, you could either select the code or put the cursor at the beginning or ending of the code you want to run, press (Ctrl+Enter), and then code will be sent to the active R terminal.
+
If you want to run an entire R file, open the file in the editor, and press Ctrl+Shift+S and the file will be sourced in the active R terminal:
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
For more information on working with R in VS Code, click here.
classPluginAbstract(abc.ABC):
+"""Base class that is used as a template for all Freva plugins.
+ Any api wrapper class defining Freva plugins must inherit from this class."""
+
+ @deprecated_method("PluginAbstract","class_basedir")
+ defgetClassBaseDir(self)->Optional[str]:
+"""Deprecated method for class_basedir."""
+
+ defclass_basedir(self)->str:
+"""Get absolute path to the module defining the plugin class."""
+
classPluginAbstract(abc.ABC):
+"""Base class that is used as a template for all Freva plugins.
+ Any api wrapper class defining Freva plugins must inherit from this class."""
+
+ defcall(
+ self,
+ cmd_string:Union[str,list[str]],
+ check:bool=True,
+ **kwargs,
+ )->sub.Popen[Any]:
+"""Run command with arguments and return a CompletedProcess instance."""
+
classPluginAbstract(abc.ABC):
+"""Base class that is used as a template for all Freva plugins.
+ Any api wrapper class defining Freva plugins must inherit from this class."""
+
+ @deprecated_method("PluginAbstract","prepare_output")
+ defprepareOutput(self,*args)->dict[str,dict[str,str]]:
+"""Deprecated method for :class:`prepare_output`. """
+
+ defprepare_output(
+ self,output_files:Union[str,list[str],dict[str,dict[str,str]]]
+ )->dict[str,dict[str,str]]:
+"""Prepare output for files supposedly created."""
+
classParameterType(initOrder):
+"""Base class for all parameter types.
+ All available parameter types inherit from this class. The class creates
+ a parameter object holding the following user defined information.
+ """
+ def__init__(
+ self,
+ name=None,
+ default=None,
+ mandatory=False,
+ max_items=1,
+ item_separator=",",
+ regex=None,
+ version=1,
+ help="No help available.",
+ print_format="%s",
+ impact=Parameter.Impact.affects_values,
+ ):
+
classParameterDictionary(dict):
+"""Directory holding all plugin parameters for a ``Freva`` plugin.
+ This class behaves like a built-in ``dict`` with additional features.
+ The most prominent feature is that the order of added items is preserved,
+ as opposed to a normal build-in ``dict``.
+ """
+ def__init__(self,*parameters:ParameterType)->None:
+
class_ConfigWrapper:
+"""Convenience class that helps to dynamically set the location of the
+ evaluation system config file."""
+ _env:str="EVALUATION_SYSTEM_CONFIG_FILE"
+ def__init__(
+ self,
+ default_file:str,
+ ):
+ self.default_file=default_file
+ def__fspath__(self)->str:
+ returnos.environ.get(self._env,self.default_file)
+ def__repr__(self)->str:
+ returnself.__fspath__()
+
We can define a Makefile as below and then after definition of environment.yml, install all the dependencies listed above via make commands. But first and foremost, we need to ensure, we have make installed on our machine.
+
env:
+ @# Installs a conda environment with R and the IDF package
+@if[-d"./plugin_env"];then\
+rm-rf./plugin_env;\
+fi
+condaenvcreate--prefix./plugin_env-fenvironment.yml--force
+Rscript-e'install.packages("IDF", repos="https://CRAN.R-project.org/")'
+cartopy:
+ @# Download various cartopy shape-files
+@if[-d"./cartopy"];then\
+rm-rf./cartopy;\
+fi
+./plugin_env/bin/python3assets/cartopy_download.pyphysicalcultural
+.PHONY:envcartopy
+
+
+
Using mamba instead of using conda can make the way of installation faster.
![\"freva](\"media/freva_overview/freva-people.jpg\")
\n",
+ "\n",
+ "# Introduction to Freva ![\"freva](\"media/freva_overview/freva-logo.png\"){kind=logo}
\n",
+ "\n",
+ "\n",
+ "**Martin Bergemann, Bijan Fallah, Andrej Fast, Mostafa Hadizadeh, \n",
+ "Christopher Kadow, Etor E. Lucio-Eceiza, Felix Oertel, Manuel Reis, and many others…**\n",
+ "\n",
+ "Deutsches Klimarechenzentrum @ CLINT in Data Analysis Dpt. (+ Data Management Dpt.*)\n",
+ "\n",
+ "![\"freva](\"media/freva_overview/logo-fona-dkrz.jpg\"){kind=logo}
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "## Common Problem I: Finding Data\n",
+ "![\"data\"](\"media/freva_overview/problem-data.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Common Problem II: Using code of others\n",
+ "![\"code\"](\"media/freva_overview/problem-usability.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Common Problem III: Reproducing your results\n",
+ "![\"reproduce\"](\"media/freva_overview/problem-reproducibility.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "* How can we search and access various datasets efficiently?\n",
+ "* How can we streamline user data analysis tools (reusable and reproducible)?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "fragment"
+ }
+ },
+ "source": [
+ "## Yet another solution: The Freva framework ![\"freva](\"media/freva_overview/freva-flowchart_basic.png\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Flexible access ![\"freva](\"media/freva_overview/freva-accessibility.png\")
\n",
+ "![\"parts\"](\"https://swift.dkrz.de/v1/dkrz_d0859f7c-0da8-41c7-824d-58b3765f5ec4/freva_workshop_202406/animations/freva_jupyter_databrowser.gif?temp_url_sig=5724e51d563103eb80efaf5197d34b95a1d83f5e&temp_url_expires=2029-06-25T10:06:38Z\")
\n",
+ " ![\"freva](\"media/freva_overview/freva-data.png\")
\n",
+ "![\"parts\"](\"https://swift.dkrz.de/v1/dkrz_d0859f7c-0da8-41c7-824d-58b3765f5ec4/freva_workshop_202406/animations/freva_databrowser_CLI.gif?temp_url_sig=e8a9b30f892c1cd0adb9f3adb6243a20468f6f2d&temp_url_expires=2029-06-25T10:03:38Z\")
\n",
+ " ![\"parts\"](\"https://swift.dkrz.de/v1/dkrz_d0859f7c-0da8-41c7-824d-58b3765f5ec4/freva_workshop_202406/animations/freva_web_databrowser.gif?temp_url_sig=0efad95be935d96c0e8f3490af4d1902014effd8&temp_url_expires=2029-06-25T10:09:09Z\")
\n",
+ " ![\"parts\"](\"media/freva_overview/blank-plot.jpg\")
\n",
+ " ![\"freva](\"media/freva_overview/freva-plugins.png\")
\n",
+ "\n",
+ "* Flexible programming language: NO specific language (![\"parts\"](\"media/freva_overview/plugin-examples.jpg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Transparency & reproducibility ![\"freva](\"media/freva_overview/freva-transparency.png\")
\n",
+ "![\"parts\"](\"https://swift.dkrz.de/v1/dkrz_d0859f7c-0da8-41c7-824d-58b3765f5ec4/freva_workshop_202406/animations/freva_web_history_and_result-browser.gif?temp_url_sig=88807433f3db77ced775c5f63793fd7539853104&temp_url_expires=2029-06-25T10:10:30Z\")
\n",
+ " ![\"freva](\"media/freva_overview/freva-flowchart_plus.png\")
\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## A development environment ![\"freva](\"media/freva_overview/freva-how-to-plug.png\")
\n",
+ "\n",
+ "![\"parts\"](\"https://swift.dkrz.de/v1/dkrz_d0859f7c-0da8-41c7-824d-58b3765f5ec4/freva_workshop_202406/animations/freva_CLI_plugins.gif?temp_url_sig=8b3d115fdf054ea0c483a0a0ad6546ccfabb17e7&temp_url_expires=2029-06-25T09:57:21Z\")
\n",
+ " ![\"parts\"](\"https://swift.dkrz.de/v1/dkrz_d0859f7c-0da8-41c7-824d-58b3765f5ec4/freva_workshop_202406/animations/freva_web_plugins.gif?temp_url_sig=1bb6567d4a988953c05768d9299a9dbf9705a725&temp_url_expires=2029-06-25T10:12:01Z\")
\n",
+ " ![\"freva](\"media/freva_overview/freva-link-my-data.png\")
![\"freva](\"media/freva_overview/link2database.jpg\")
![\"freva](\"media/freva_overview/user-data.jpg\")
![\"freva](\"media/freva_overview/user-indata-plugin.jpg\")
![\"freva](\"media/freva_overview/freva-docu.png\")
![\"parts\"](\"media/freva_overview/frevacode-docu.jpg\")
\n",
+ " ![\"parts\"](\"media/freva_overview/plugin-docu.jpg\")
\n",
+ " ![\"freva](\"media/freva_overview/freva-next.png\")
![\"parts\"](\"media/freva_overview/freva-whatisnext.jpg\")
\n",
+ " 
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+![\"freva](\"media/plugin_intro/freva-plugindev.png\")
\n",
+ "\n",
+ "**Martin Bergemann, Bijan Fallah, Andrej Fast, Mostafa Hadizadeh, \n",
+ "Christopher Kadow, Etor E. Lucio-Eceiza, Felix Örtel, Manuel Reis, and many others…**\n",
+ "\n",
+ "Deutsches Klimarechenzentrum @ CLINT in Data Analysis Dpt. (+ Data Management Dpt.*)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# Overview of a Freva plugin\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Why should I build a Freva plugin?\n",
+ "It allows you to be part of \"The Freva Cycle\"\n",
+ "\n",
+ "![\"parts\"](\"media/plugin_intro/plugin-livecycle.png\")
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Structure of a Plugin\n",
+ "![\"parts\"](\"media/plugin_intro/plugin-parts.png\")
\n",
+ " ![\"parts\"](\"media/plugin_intro/wrapper.jpg\")
\n",
+ " ![\"parts\"](\"media/plugin_intro/code.jpg\")
\n",
+ " ![\"parts\"](\"media/plugin_intro/make-yaml.jpg\")
\n",
+ " ![\"freva](\"media/gitlab/git-logo.jpg\"){kind=logo}
\n",
+ "Is a free and open source *Distributed* **Version Control System**:\n",
+ "- **Version Control System**: a system that tracks and provides control over changes to source code, documentation and configuration files.\n",
+ "- *Distributed*: a form of version control in which the complete codebase, including its full history, is mirrored on every developer's computer\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What is gitlab? ![\"freva](\"media/gitlab/gitlab-logo.jpg\"){kind=logo}
\n",
+ "\n",
+ "![\"freva](\"media/gitlab/gitlab-flow.jpg\")
\n",
+ " ![\"freva](\"media/gitlab/freva-gitlab-flow.jpg\")
\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: location of the plugins\n",
+ "\n",
+ "- Under [https://gitlab.dkrz.de/freva/plugins4freva](https://gitlab.dkrz.de/freva/plugins4freva) project:\n",
+ "![\"freva](\"media/gitlab/freva-gitlab.jpg\")
\n",
+ "- Other projects have also their own, e.g.: \n",
+ " - climxtreme: https://gitlab.dkrz.de/bm1159/plugins4freva\n",
+ " - regiklim-nukleus: https://gitlab.dkrz.de/ch1187/plugins4freva\n",
+ "- some plugins can be found in different projects (see [forks](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html)).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: cloning a repository\n",
+ "To download a repository there are 2 ways: via `https` and via `ssh`:\n",
+ "![\"freva](\"media/gitlab/clone-repo.jpg\")
\n",
+ "- `https`: straightforward, requires to enter (dkrz) username and password (usually). \n",
+ "- `ssh`: convenient, requires to provide a ssh key previously.\n",
+ "\n",
+ "In both cases:\n",
+ "\n",
+ "```\n",
+ "$ ssh ![\"freva](\"media/gitlab/ssh-keyadd.jpg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## Gitlab and Freva: creating a new project\n",
+ "- If we already had an existing Git repository:\n",
+ "```shell\n",
+ "$ cd NEW_PLUGIN # for example\n",
+ "$ git remote rename origin old-origin\n",
+ "$ git remote add origin git@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git\n",
+ "$ git push -u origin --all\n",
+ "$ git push -u origin --tags\n",
+ "```\n",
+ "\n",
+ "- If we did not already have an existing Git repository:\n",
+ "```shell\n",
+ "$ cd NEW_PLUGIN # for example\n",
+ "$ git init\n",
+ "$ git remote add origin git@gitlab.dkrz.de:freva/plugins4freva/new_plugin.git\n",
+ "# create your new files and folders\n",
+ "$ git add .\n",
+ "$ git commit -m \"Initial commit of my new project\"\n",
+ "$ git push -u origin master\n",
+ "```\n",
+ "\n",
+ "Now you could (hypothetically) go to https://gitlab.dkrz.de/freva/plugins4freva/new_plugin"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# License of Plugins"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What kind of License should I follow?\n",
+ "\n",
+ "[The Freva code](https://github.com/FREVA-CLINT/freva/blob/main/LICENSE.md) follows a [BSD3 license](https://opensource.org/license/bsd-3-clause), a permissive open-source license with minimal restrictions:\n",
+ "- It allows for wide usage, including in proprietary software.\n",
+ "- it imposes very few restrictions on how the software can be redistributed.\n",
+ "\n",
+ "In principle, any derived product from Freva (i.e. plugins), should follow this license. *Unless* stated otherwise by the developer.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## What kind of License should I follow?\n",
+ "\n",
+ "- We encourage open-source software to foster collaboration within and outside your projects.\n",
+ "- That said, common courtesy should be expected, e.g.:\n",
+ " - reference to the first manuscript where the developer used the plugin (the plugin documentation is useful for this!).\n",
+ " - contacting him/her in case there is no such manuscript."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "slide"
+ }
+ },
+ "source": [
+ "# Connecting your IDE / Code Editor to a remote session using SSH"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "## 1. VS-Code ![\"freva](\"media/vscode/vscode_logo.svg\"){kind=logo}
\n",
+ "\n",
+ "Open a new window in VS Code. There are several ways to connect to a remote session, as indicated in the picture below:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "\n",
+ "![\"freva](\"media/vscode/05_remote_folder_success.jpg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "![\"freva](\"media/pycharm/pycharm_logo.svg\"){kind=logo}
\n",
+ "\n",
+ "\n",
+ "> 😱\n",
+ ">Remote Development is currently in beta and only supported for the Professional Version of PyCharm!\n",
+ "\n",
+ "Open your Plugins (Ctrl+Alt+S) and confirm that \"Remote Development Gateway\" is installed and that it is enabled:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "![\"freva](\"media/rstudio/RStudio_logo.svg\"){kind=logo}
\n",
+ "\n",
+ "There is currently only one method of using R Studio on a remote host that is officially supported by R, namely by connecting to an instance of R Studio Server running on the remote host. The previously available alternative, by creating a remote connection in [R Studio Desktop Pro](https://docs.posit.co/ide/desktop-pro/2022.02.1+461.pro1/remote-desktop.html), has been removed since version [2023.06.2](https://docs.posit.co/ide/desktop-pro/remote_desktop/remote_desktop.html).\n",
+ "\n",
+ "However, R Studio Server is not available on Levante, so creating a remote development environment instance this way is not possible. However, it is possible to remotely develop and execute R-code by using the [VS Code R Extension](https://code.visualstudio.com/docs/languages/r). "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "subslide"
+ }
+ },
+ "source": [
+ "![](\"media/plugin_general/python-logo.png\"){kind=logo}
![\"What](\"media/plugin_general/wrapper.gif\")
\n",
+ " ![\"wrapper's](\"media/plugin_general/structure.gif\")
\n",
+ " ![](\"media/plugin_general/yaml.png\")
![](\"media/plugin_general/gnu-make.jpg\")
![\"install](\"media/plugin_general/env.gif\")
\n",
+ " ![](\"media/plugin_general/freva.png\")
![\"plug](\"media/plugin_general/plugin_general.png\")
\n",
+ " ![\"plug](\"media/plugin_general/module_load_code.png\")
\n",
+ " ![\"plug](\"media/plugin_general/module_load.png\")
\n",
+ " ![\"plug](\"media/plugin_general/plug_in_code.png\")
\n",
+ " ![\"plug](\"media/plugin_general/plug_in_web.png\")
\n",
+ " ![\"plug](\"media/plugin_general/plugin_list_code.png\")
\n",
+ " ![\"plug](\"media/plugin_general/plugin_list_web.png\")
\n",
+ " ![\"freva](\"media/freva-people.jpg\")
\n",
+ "\n",
+ "
+
+
+
+
+
+
+
+
+- Other projects have also their own, e.g.:
+ - climxtreme: https://gitlab.dkrz.de/bm1159/plugins4freva
+ - regiklim-nukleus: https://gitlab.dkrz.de/ch1187/plugins4freva
+- some plugins can be found in different projects (see [forks](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html)).
+
+
+- `https`: straightforward, requires to enter (dkrz) username and password (usually).
+- `ssh`: convenient, requires to provide a ssh key previously.
+
+
+
+
+
+
+
+
+
+
+
+
+