Skip to content

Commit

Permalink
Getting value from environment variables for maxBufferingDurationMs. …
Browse files Browse the repository at this point in the history
…Information about it added to README (#14)

Getting value from environment variables for maxBufferingDurationMs
  • Loading branch information
Nuzhdina-Elena authored Mar 23, 2021
1 parent 906e6e7 commit 9ad7e12
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,9 @@ The template allows for the user to supply the following optional parameter:

- **nonTokenizedDeadLetterPath**: Folder where failed to tokenize data will be stored

The template also allows user to override the environment variable:

- **MAX_BUFFERING_DURATION_MS**: Max duration of buffering rows in milliseconds. Default value: 100ms.

in the following format:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,8 @@ public class DataProtectors {
private static final Logger LOG = LoggerFactory.getLogger(DataProtectors.class);

public static final String ID_FIELD_NAME = "ID";
private static final Integer MAX_BUFFERING = 100;
private static final Long MAX_BUFFERING_DURATION_MS =
Long.valueOf(System.getenv().getOrDefault("MAX_BUFFERING_DURATION_MS", "100"));

/**
* The {@link RowToTokenizedRow} transform converts {@link Row} to {@link TableRow} objects. The
Expand Down Expand Up @@ -107,7 +108,7 @@ public PCollectionTuple expand(PCollection<KV<Integer, Row>> inputRows) {
FailsafeElementCoder<Row, Row> coder =
FailsafeElementCoder.of(RowCoder.of(schema()), RowCoder.of(schema()));

Duration maxBuffering = Duration.millis(MAX_BUFFERING);
Duration maxBuffering = Duration.millis(MAX_BUFFERING_DURATION_MS);
PCollectionTuple pCollectionTuple =
inputRows
.apply(
Expand Down

0 comments on commit 9ad7e12

Please sign in to comment.