Skip to content

Commit

Permalink
Revert "Document yaml pipeline options (#30490)"
Browse files Browse the repository at this point in the history
This reverts commit 3c46415.
  • Loading branch information
Abacn authored Mar 6, 2024
1 parent 3f47453 commit b82603e
Showing 1 changed file with 0 additions and 40 deletions.
40 changes: 0 additions & 40 deletions website/www/site/content/en/documentation/sdks/yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -366,8 +366,6 @@ pipeline:
config:
topic: anotherPubSubTopic
format: json
options:
streaming: true
```

Rather than using an explicit `WindowInto` operation, one may instead tag a
Expand All @@ -394,8 +392,6 @@ pipeline:
config:
topic: anotherPubSubTopic
format: json
options:
streaming: true
```

Note that the `Sql` operation itself is often a from of aggregation, and
Expand All @@ -421,8 +417,6 @@ pipeline:
config:
topic: anotherPubSubTopic
format: json
options:
streaming: true
```

The specified windowing is applied to all inputs, in this case resulting in
Expand Down Expand Up @@ -454,8 +448,6 @@ pipeline:
windowing:
type: fixed
size: 60s
options:
streaming: true
```

For a transform with no inputs, the specified windowing is instead applied to
Expand All @@ -481,8 +473,6 @@ pipeline:
config:
topic: anotherPubSubTopic
format: json
options:
streaming: true
```

One can also specify windowing at the top level of a pipeline (or composite),
Expand All @@ -509,8 +499,6 @@ pipeline:
windowing:
type: fixed
size: 60
options:
streaming: true
```

Note that all these windowing specifications are compatible with the `source`
Expand Down Expand Up @@ -542,9 +530,6 @@ pipeline:
windowing:
type: fixed
size: 5m
options:
streaming: true
```


Expand Down Expand Up @@ -599,31 +584,6 @@ providers:
MyCustomTransform: "pkg.subpkg.PTransformClassOrCallable"
```

## Pipeline Options

[Pipeline options](https://beam.apache.org/documentation/programming-guide/#configuring-pipeline-options)
are used to configure different aspects of your pipeline, such as the pipeline runner that will execute
your pipeline and any runner-specific configuration required by the chosen runner. To set pipeline options,
append an options block at the end of your yaml file. For example:

```
pipeline:
type: chain
transforms:
- type: ReadFromPubSub
config:
topic: myPubSubTopic
format: ...
schema: ...
...
- type: WriteToPubSub
config:
topic: anotherPubSubTopic
format: json
options:
streaming: true
```

## Other Resources

* [Example pipelines](https://gist.github.com/robertwb/2cb26973f1b1203e8f5f8f88c5764da0)
Expand Down

0 comments on commit b82603e

Please sign in to comment.