Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 0.0.119 #59

Merged
merged 11 commits into from
Dec 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/faq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# FAQ

- [Why Python](why-python.md)
- [Does Zrb has A Scheduler](does-zrb-has-a-scheduler.md)
- [Does Zrb have a scheduler](does-zrb-have-a-scheduler.md)
- [How to get data from other tasks](how-to-get-data-from-other-tasks.md)

🔖 [Table of Contents](../README.md)
20 changes: 0 additions & 20 deletions docs/faq/does-zrb-has-a-scheduler.md

This file was deleted.

20 changes: 20 additions & 0 deletions docs/faq/does-zrb-have-a-scheduler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
🔖 [Table of Contents](../README.md) / [FAQ](README.md)


# Does Zrb Have a Scheduler?

No, but you can use `RecurringTask` and `TimeWatcher`


# How To Make A Scheduled Task?

There are some [tricks](../tutorials/running-task-by-schedule.md) you can use.

For example, you can use:

- RecurringTask
- Infinite loop
- Cronjob
- Orchestrator like Airflow.

🔖 [Table of Contents](../README.md) / [FAQ](README.md)
86 changes: 30 additions & 56 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Welcome to Zrb's getting started guide. We will cover everything you need to kno
- [Task Definition](#task-definition)
- [Creating a Task Using Task Classes](#creating-a-task-using-task-classes)
- [Creating a Task Using Python Decorator](#creating-a-task-using-python-decorator)
- [Task Parameters](#task-parameters)
- [Task Parameters](#task-parameters)
- [Task Inputs](#task-inputs)
- [Task Environments](#task-environments)
- [Switching Environment](#switching-environment)
Expand Down Expand Up @@ -282,17 +282,21 @@ Commands:

## Activating Virtual Environment

Working in a virtual environment is recommended in most cases. This encapsulates your project pip packages, ensuring better independence and reproducibility.
> __📝 NOTE:__ Using virtual environment is not required, but highly recommended.

A virtual environment encapsulates your project pip packages from other projects, ensuring better independence and reproducibility.

### Activating Virtual Environment On A Generated Project

If you generate the project by invoking `zrb project create`, then you need to run the following command every time you start working on the project:
If you generate the project by invoking `zrb project create`, you can create/activate the virtual environment by running the following command:

```bash
source project.sh
```

The command will activate the project's virtual environment and install necessary pip packages.
The command will activate the project's virtual environment and install the necessary pip packages.

You should run the command every time you start working on the project.

### Activating Virtual Environment On A Manually Created Project

Expand All @@ -310,10 +314,7 @@ Once you make the virtual environment, you can activate it by invoking the follo
source .venv/bin/activate
```

You need to run the command every time you start working on the project.


> __💡 HINT:__ Working with virtual environment is recommended whenever you work with any Python project, including Zrb project.
You should activate the virtual environment whenever you start working on the project.


# Creating a Task
Expand Down Expand Up @@ -530,14 +531,16 @@ from zrb import runner, TaskClass
# Define a task, along with it's parameters
task_name = TaskClass(
name='task-name',
parameter=value,
other_parameter=other_value
description='the description'
# ... other task parameters
)

# regiter the task to zrb runner
runner.register(task_name)
```

> __💡 HINT:__ Check out [task-parameter section](#task-parameters) to see the commonly used parameters

There are several built-in task classes. Each with its specific use case:

- __CmdTask__: Run a CLI command/shell script.
Expand All @@ -562,8 +565,8 @@ from zrb import runner, python_task
# Decorate a function named `task_name`
@python_task(
name='task-name',
parameter=value,
other_parameter=other_value,
description='the description'
# ... other task parameters
runner=runner # register the task to zrb runner
)
def task_name(*args, **kwargs):
Expand All @@ -572,12 +575,14 @@ def task_name(*args, **kwargs):
# Note that python_task decorator turn your function into a task. So `task_name` is now a task, not a function.
```

> __💡 HINT:__ Check out [task-parameter section](#task-parameters) to see the commonly used parameters

Using `@python_task` decorator is your best choice if you need to write complex logic in Python.


## Task Parameters
### Task Parameters

Each task has its specific parameter. However, the following tasks are typically available:
Each task has its specific parameter. However, the following parameters are typically available:

- __name__: The name of the task. When you invoke the task using the CLI, you need to use this name. By convention, the name should-be written in `kebab-case` (i.e., separated by `-`)
- __description__: The description of the task.
Expand Down Expand Up @@ -873,7 +878,7 @@ Host: stalchmst.com

# Creating a long-running task

Commonly, you can determine whether a task is successful/failed after it is completed. However, some tasks might run forever, and you can only see whether the task is completed or failed by checking other behaviors. For example, a web server is successfully running if you can get the expected HTTP response from the server.
Commonly, you can determine whether a task is successful/failed after the task is finished. However, some tasks might run forever, and you can only see whether the task is completed or failed by checking other behaviors. For example, a web server is successfully running if you can get the expected HTTP response from the server.

Zrb has some checking mechanisms to handle this use case.

Expand All @@ -883,54 +888,18 @@ Let's start by scaffolding a CmdTask named `run-jupyterlab`.
zrb project add cmd-task --project-dir "." --task-name "run-jupyterlab"
```

Once you do so, you can start modifying `_automate/`

In some cases, your task has to run forever (i.e., web server).

Arasaka is a data-driven (and family-driven) company. They need their data scientists to experiment a lot to present the most valuable information/knowledge.

For this, they need to be able to create a lot of notebooks for experimentation.

To make sure things work, you need to:
- Install jupyterlab.
- Add Jupyterlab to your `requirements.txt`.
- Create a `notebooks` directory under `src`.
- Create a `start-jupyter` task.

Let's start by installing jupyterlab

```bash
pip install jupyterlab
```

Once jupyterlab has been installed, you need to add it into requirements.txt. You can do so by typing `pip freeze | grep jupyterlab` and add the output to your `requirements.txt`. Or you can do it with a single command:
You will notice that Zrb automatically creates a file named `_automate/run_jupyterlab.py`

```bash
pip freeze | grep jupyterlab >> requirements.txt
```

Now let's make a `notebooks` directory under `src`.

```bash
mkdir -p src/notebooks
touch src/notebooks/.gitkeep
```
We will need to modify the file.

You need an empty `.gitkeep` file, to tell git to not ignore the directory.

## Adding start-jupyterlab

We have a few requirements for `start-jupyterlab` task

- You should show Arasaka banner before starting jupyterlab.
- `start-jupyterlab` is considered completed only if the port is accessible.
- Arasaka employee can choose the port to serve jupyterlab in their computer.

Let's start by adding the task to your project.

```bash
zrb project add cmd-task --project-dir . --task-name start-jupyterlab
```
- Before starting Jupyterlab, you need to make sure that Jupyterlab is already installed.
- Jupyterlab is considered completed once the port is accessible.
- Jupyterlab HTTP port should be `8080` by default, but users should be able to override the Jupyterlab HTTP port.

Now, let's modify `_automate/start_jupyterlab.py` into the following:

Expand Down Expand Up @@ -978,6 +947,9 @@ Let's run the task:
zrb project run-jupyterlab
```

<details>
<summary>Show output</summary>

```
Jupyterlab port [8080]:
🤖 ○ ◷ 2023-11-12 10:26:32.759 ❁ 58728 → 1/3 🐨 zrb project install-jupyterlab • Run script: pip install jupyterlab
Expand Down Expand Up @@ -1011,6 +983,8 @@ Support zrb growth and development!
🤖 ○ ◷ 2023-11-12 10:26:36.807 ❁ 58920 → 1/3 🐹 zrb project run-jupyterlab • Completed in 4.050489664077759 seconds
```

</details>

Open up your browser on [http://localhost:8080](http://localhost:8080) to start working with the notebook.

# Now you are ready
Expand Down
46 changes: 44 additions & 2 deletions docs/tutorials/running-task-by-schedule.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,57 @@
🔖 [Table of Contents](../README.md) / [Tutorials](README.md)

# Run Task by Schedule
# Running Task by Schedule

Zrb doesn't have any built-in scheduler. However, there are some workarounds you can use:

- Using RecurringTask
- Creating an infinite loop
- Using Cronjob
- Using Airflow or other orchestrator

# Using RecurringTask

The best approach is by turning your task into a recurring task.

```python
from zrb import CmdTask, TimeWatcher, RecurringTask, runner

# Your original task
hello = CmdTask(
name='hello',
cmd='echo "hello world"'
)
runner.register(hello)


# Your recurring task
scheduled_hello = RecurringTask(
name='scheduled-hello',
inputs=[
StrInput(name='schedule', default='* * * * *')
],
triggers=[
TimeWatcher(schedule='{{input.schedule}}')
],
task=hello
)
runner.register(scheduled_hello)
```

Notice that `TimeWatcher`'s `schedule` is a [cron schedule expression](https://crontab.guru/).

The expression `* * * * *` means that the task will be executed every minute.

To execute `scheduled-hello`, you can invoke:

```bash
zrb scheduled-hello
```


# Creating an Infinite Loop

The simplest approach is by turning your task into a function, and call your function after some interval.
Another simple approach is by turning your task into a function, and call your function after some interval.

For example, you want to run `hello` task every 5 seconds. Then you can create the following Python script and run it.

Expand Down Expand Up @@ -38,6 +79,7 @@ while True:
python scheduled_hello.py
```


# Using CronJob


Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ build-backend = "flit_core.buildapi"

[project]
name = "zrb"
version = "0.0.118"
version = "0.0.119"
authors = [
{ name="Go Frendi Gunawan", email="gofrendiasgard@gmail.com" },
]
description = "Super framework for your super app"
description = "A Framework to Enhance Your Workflow"
readme = "README.md"
requires-python = ">=3.10.0"
classifiers = [
Expand Down
4 changes: 2 additions & 2 deletions src/zrb/builtin/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from zrb.builtin import generator
from zrb.builtin import process
from zrb.builtin import say
from zrb.builtin import watch
from zrb.builtin import watch_changes
from zrb.builtin import schedule

assert base64
Expand All @@ -29,5 +29,5 @@
assert generator
assert process
assert say
assert watch
assert watch_changes
assert schedule
Empty file.
46 changes: 46 additions & 0 deletions src/zrb/builtin/helper/reccuring_action.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
from zrb.task.task import Task
from zrb.task.cmd_task import CmdTask
from zrb.task.notifier import Notifier
from zrb.task_input.str_input import StrInput


def create_recurring_action(
title: str,
default_message: str = '👋'
) -> Task:
# define inputs
message_input = StrInput(
name='message',
default=default_message,
prompt='Message to be shown',
)
command_input = StrInput(
name='command',
default='',
prompt='Command to be executed',
)
# define tasks
run_command = CmdTask(
name='run-command',
icon='⚙️',
color='blue',
inputs=[command_input],
should_execute='{{ input.command != "" }}',
cmd='{{ input.command }}'
)
notify = Notifier(
name='notify',
icon='📢',
color='green',
inputs=[message_input],
title=title,
message='{{ input.message }}',
should_execute='{{ input.message != "" }}',
)
# return aggregator task
return Task(
name='recurring-action',
inputs=[message_input, command_input],
upstreams=[run_command, notify],
retry=0
)
Loading