Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug/#1226 added doc for polars #1259

Merged
merged 6 commits into from
Jan 21, 2025
Merged

bug/#1226 added doc for polars #1259

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1be7c60
added doc for polars
Jan 14, 2025
0f8ece0
re-added flak8
Jan 14, 2025
9b4d526
replace polars representation with actual polars code
Jan 20, 2025
708707a
reset minor changes
Jan 20, 2025
b88b124
align the pandas constructor Python
Jan 21, 2025
30e13cc
return flake8
Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 149 additions & 34 deletions docs/userman/scenario_features/data-integration/data-node-usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,12 +263,11 @@ The following examples represent the results when reading from a CSV data node w
=== "exposed_type = "pandas""

```python
pandas.DataFrame
(
date nb_sales
0 12/24/2018 1550
1 12/25/2018 2315
2 12/26/2018 1832
pandas.DataFrame(
{
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

Expand All @@ -285,6 +284,7 @@ The following examples represent the results when reading from a CSV data node w
```

=== "exposed_type = SaleRow"

```python
[
SaleRow("12/24/2018", 1550),
Expand All @@ -293,6 +293,23 @@ The following examples represent the results when reading from a CSV data node w
]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }


When writing data to a CSV data node, the `CSVDataNode.write()^` method can take several datatype as the input:

- list, numpy array
Expand Down Expand Up @@ -409,12 +426,11 @@ The following examples represent the results when reading from an Excel data nod
=== "exposed_type = "pandas""

```python
pandas.DataFrame
(
date nb_sales
0 12/24/2018 1550
1 12/25/2018 2315
2 12/26/2018 1832
pandas.DataFrame(
{
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

Expand All @@ -439,6 +455,22 @@ The following examples represent the results when reading from an Excel data nod
]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }

When writing data to an Excel data node, the `ExcelDataNode.write()^` method can take several datatype as the input:

- list, numpy array
Expand Down Expand Up @@ -557,12 +589,12 @@ node with different _exposed_type_:
=== "exposed_type = "pandas""

```python
pandas.DataFrame
(
ID date nb_sales
0 1 12/24/2018 1550
1 2 12/25/2018 2315
2 3 12/26/2018 1832
pandas.DataFrame(
{
"ID": [1, 2, 3],
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

Expand All @@ -587,6 +619,23 @@ node with different _exposed_type_:
]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"ID": [1, 2, 3],
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }

When writing data to a SQL Table data node, the `SQLTableDataNode.write()^`
method can take several datatype as the input:

Expand Down Expand Up @@ -927,12 +976,11 @@ The following examples represent the results when read from Parquet data node wi
=== "exposed_type = "pandas""

```python
pandas.DataFrame
(
date nb_sales
0 12/24/2018 1550
1 12/25/2018 2315
2 12/26/2018 1832
pandas.DataFrame(
{
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

Expand All @@ -957,12 +1005,29 @@ The following examples represent the results when read from Parquet data node wi
]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"date": ["12/24/2018", "12/25/2018", "12/26/2018"],
"nb_sales": [1550, 2315, 1832]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }

When writing data to a Parquet data node, the `ParquetDataNode.write()^` method can take several
datatype as the input depending on the _exposed type_:

- pandas dataframes
- numpy arrays
- any object, which will be passed to the `pd.DataFrame` constructor (e.g., list of dictionaries)
- polars dataframes (Available in Taipy Enterprise edition only)

The following examples will write to the path of the Parquet data node:

Expand Down Expand Up @@ -1556,11 +1621,7 @@ filtered_data = data_node.filter(("nb_sales", 1550, Operator.EQUAL))
=== "exposed_type = "pandas""

```python
pandas.DataFrame
(
date nb_sales
0 12/24/2018 1550
)
pandas.DataFrame({"date": ["12/24/2018"], "nb_sales": [1550]})
```

=== "exposed_type = "numpy""
Expand All @@ -1576,6 +1637,22 @@ filtered_data = data_node.filter(("nb_sales", 1550, Operator.EQUAL))
[SaleRow("12/24/2018", 1550)]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"date": ["12/24/2018"],
"nb_sales": [1550]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }

If a list of operators is provided, it is necessary to provide a join operator that will be
used to combine the filtered results from the operators. The default join operator is `JoinOperator.AND`.

Expand All @@ -1600,6 +1677,12 @@ filtered_data = data_node.filter(
0 12/24/2018 1550
1 12/26/2018 1832
)
pandas.DataFrame(
{
"date": ["12/24/2018", "12/26/2018"],
"nb_sales": [1550, 1832]
}
)
```

=== "exposed_type = "numpy""
Expand All @@ -1621,6 +1704,22 @@ filtered_data = data_node.filter(
]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"date": ["12/24/2018", "12/26/2018"],
"nb_sales": [1550, 1832]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }

In another example, the `DataNode.filter()^` method will return all the records from the data node
where the value of the "nb_sales" field is equal to 1550 or greater than 2000.
The following examples represent the results when read from a data node with different _exposed_type_:
Expand All @@ -1637,11 +1736,11 @@ filtered_data = data_node.filter(
=== "exposed_type = "pandas""

```python
pandas.DataFrame
(
date nb_sales
0 12/24/2018 1550
1 12/25/2018 2315
pandas.DataFrame(
{
"date": ["12/24/2018", "12/25/2018"],
"nb_sales": [1550, 2315]
}
)
```

Expand All @@ -1664,6 +1763,22 @@ filtered_data = data_node.filter(
]
```

=== "exposed_type = polars"

```python
polars.DataFrame(
{
"date": ["12/24/2018", "12/25/2018"],
"nb_sales": [1550, 2315]
}
)
```

!!! warning "Available in Taipy Enterprise edition"

The Polars exposed type is only available in the Enterprise edition of Taipy.
[Contact us](https://taipy.io/book-a-call){: .tp-btn .tp-btn--accent target='blank' }

With Pandas data frame as the exposed type, it is also possible to use pandas indexing
and filtering style:

Expand Down
Loading