From 2302aee627e38169ae6ba0a37b139a0b56d563a3 Mon Sep 17 00:00:00 2001 From: Davor Bonaci Date: Tue, 25 Oct 2016 21:27:22 -0700 Subject: [PATCH] Rebuild the site following merge of PR #50 --- .../2016/04/03/presentation-materials.html | 2 +- content/blog/2016/10/20/test-stream.html | 180 ++++++++--------- .../contribute/contribution-guide/index.html | 17 +- content/css/main.css | 6 +- content/feed.xml | 186 +++++++++--------- .../learn/presentation-materials/index.html | 8 +- content/learn/programming-guide/index.html | 2 +- .../runners/capability-matrix/index.html | 2 +- content/project/public-meetings/index.html | 8 +- content/styles/site.css | 140 +++++++++++++ 10 files changed, 340 insertions(+), 211 deletions(-) diff --git a/content/beam/capability/2016/04/03/presentation-materials.html b/content/beam/capability/2016/04/03/presentation-materials.html index c16c8d1070d45..3b9e357f3460a 100644 --- a/content/beam/capability/2016/04/03/presentation-materials.html +++ b/content/beam/capability/2016/04/03/presentation-materials.html @@ -175,7 +175,7 @@

Apache Beam Presentation Materia -

As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the Apache Beam presentation materials page or email the user@beam.incubator.apache.org mailing list with your ideas or questions.

+

As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the Apache Beam presentation materials page or email the user@beam.incubator.apache.org mailing list with your ideas or questions.

diff --git a/content/blog/2016/10/20/test-stream.html b/content/blog/2016/10/20/test-stream.html index 2fc411eb378c4..9e06e10a84cc4 100644 --- a/content/blog/2016/10/20/test-stream.html +++ b/content/blog/2016/10/20/test-stream.html @@ -258,17 +258,17 @@

Everything arrives on-time

watermark and provide the result PCollection as input to the CalculateTeamScores PTransform:

-
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
-    .addElements(new GameActionInfo("sky", "blue", 12, new Instant(0L)),
-                 new GameActionInfo("navy", "blue", 3, new Instant(0L)),
-                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3))))
-    // Move the watermark past the end the end of the window
-    .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION)
-                                       .plus(Duration.standardMinutes(1)))
-    .advanceWatermarkToInfinity();
-
-PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
-    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
+
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
+    .addElements(new GameActionInfo("sky", "blue", 12, new Instant(0L)),
+                 new GameActionInfo("navy", "blue", 3, new Instant(0L)),
+                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3))))
+    // Move the watermark past the end the end of the window
+    .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION)
+                                       .plus(Duration.standardMinutes(1)))
+    .advanceWatermarkToInfinity();
+
+PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
+    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
 
@@ -276,11 +276,11 @@

Everything arrives on-time

Elements all arrive before the watermark, and are produced in the on-time pane

-
// Only one value is emitted for the blue team
-PAssert.that(teamScores)
-       .inWindow(window)
-       .containsInAnyOrder(KV.of("blue", 18));
-p.run();
+
// Only one value is emitted for the blue team
+PAssert.that(teamScores)
+       .inWindow(window)
+       .containsInAnyOrder(KV.of("blue", 18));
+p.run();
 
@@ -292,27 +292,27 @@

Some ele the system to be on time, as it arrives before the watermark passes the end of the window

-
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
-    .addElements(new GameActionInfo("sky", "blue", 3, new Instant(0L)),
-                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3))))
-    // Move the watermark up to "near" the end of the window
-    .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION)
-                                       .minus(Duration.standardMinutes(1)))
-    .addElements(new GameActionInfo("sky", "blue", 12, Duration.ZERO))
-    .advanceWatermarkToInfinity();
-
-PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
-    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
+
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
+    .addElements(new GameActionInfo("sky", "blue", 3, new Instant(0L)),
+                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3))))
+    // Move the watermark up to "near" the end of the window
+    .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION)
+                                       .minus(Duration.standardMinutes(1)))
+    .addElements(new GameActionInfo("sky", "blue", 12, Duration.ZERO))
+    .advanceWatermarkToInfinity();
+
+PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
+    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
 

An element arrives late, but before the watermark passes the end of the window, and is produced in the on-time pane

-
// Only one value is emitted for the blue team
-PAssert.that(teamScores)
-       .inWindow(window)
-       .containsInAnyOrder(KV.of("blue", 18));
-p.run();
+
// Only one value is emitted for the blue team
+PAssert.that(teamScores)
+       .inWindow(window)
+       .containsInAnyOrder(KV.of("blue", 18));
+p.run();
 
@@ -322,31 +322,31 @@

Elements are l demonstrate the triggering behavior that causes the system to emit an on-time pane, and then after the late data arrives, a pane that refines the result.

-
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
-    .addElements(new GameActionInfo("sky", "blue", 3, new Instant(0L)),
-                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3))))
-    // Move the watermark up to "near" the end of the window
-    .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION)
-                                       .minus(Duration.standardMinutes(1)))
-    .addElements(new GameActionInfo("sky", "blue", 12, Duration.ZERO))
-    .advanceWatermarkToInfinity();
-
-PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
-    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
+
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
+    .addElements(new GameActionInfo("sky", "blue", 3, new Instant(0L)),
+                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3))))
+    // Move the watermark up to "near" the end of the window
+    .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION)
+                                       .minus(Duration.standardMinutes(1)))
+    .addElements(new GameActionInfo("sky", "blue", 12, Duration.ZERO))
+    .advanceWatermarkToInfinity();
+
+PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
+    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
 

Elements all arrive before the watermark, and are produced in the on-time pane

-
// An on-time pane is emitted with the events that arrived before the window closed
-PAssert.that(teamScores)
-       .inOnTimePane(window)
-       .containsInAnyOrder(KV.of("blue", 6));
-// The final pane contains the late refinement
-PAssert.that(teamScores)
-       .inFinalPane(window)
-       .containsInAnyOrder(KV.of("blue", 18));
-p.run();
+
// An on-time pane is emitted with the events that arrived before the window closed
+PAssert.that(teamScores)
+       .inOnTimePane(window)
+       .containsInAnyOrder(KV.of("blue", 6));
+// The final pane contains the late refinement
+PAssert.that(teamScores)
+       .inFinalPane(window)
+       .containsInAnyOrder(KV.of("blue", 18));
+p.run();
 
@@ -356,33 +356,33 @@

TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
-    .addElements(new GameActionInfo("sky", "blue", 3, Duration.ZERO),
-                 new GameActionInfo("navy", "blue", 3, Duration.standardMinutes(3)))
-    // Move the watermark up to "near" the end of the window
-    .advanceWatermarkTo(new Instant(0).plus(TEAM_WINDOW_DURATION)
-                                         .plus(ALLOWED_LATENESS)
-                                         .plus(Duration.standardMinutes(1)))
-    .addElements(new GameActionInfo(
-                     "sky",
-                     "blue",
-                     12,
-                     new Instant(0).plus(TEAM_WINDOW_DURATION).minus(Duration.standardMinutes(1))))
-    .advanceWatermarkToInfinity();
-
-PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
-    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
+
TestStream<GameActionInfo> infos = TestStream.create(AvroCoder.of(GameActionInfo.class))
+    .addElements(new GameActionInfo("sky", "blue", 3, Duration.ZERO),
+                 new GameActionInfo("navy", "blue", 3, Duration.standardMinutes(3)))
+    // Move the watermark up to "near" the end of the window
+    .advanceWatermarkTo(new Instant(0).plus(TEAM_WINDOW_DURATION)
+                                         .plus(ALLOWED_LATENESS)
+                                         .plus(Duration.standardMinutes(1)))
+    .addElements(new GameActionInfo(
+                     "sky",
+                     "blue",
+                     12,
+                     new Instant(0).plus(TEAM_WINDOW_DURATION).minus(Duration.standardMinutes(1))))
+    .advanceWatermarkToInfinity();
+
+PCollection<KV<String, Integer>> teamScores = p.apply(createEvents)
+    .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS));
 

Elements all arrive before the watermark, and are produced in the on-time pane

-
// An on-time pane is emitted with the events that arrived before the window closed
-PAssert.that(teamScores)
-       .inWindow(window)
-       .containsInAnyOrder(KV.of("blue", 6));
+
// An on-time pane is emitted with the events that arrived before the window closed
+PAssert.that(teamScores)
+       .inWindow(window)
+       .containsInAnyOrder(KV.of("blue", 6));
 
-p.run();
+p.run();
 
@@ -392,29 +392,29 @@

CalculateUserScores

-
TestStream.create(AvroCoder.of(GameActionInfo.class))
-    .addElements(new GameActionInfo("scarlet", "red", 3, new Instant(0L)),
-                 new GameActionInfo("scarlet", "red", 2, new Instant(0L).plus(Duration.standardMinutes(1))))
-    .advanceProcessingTime(Duration.standardMinutes(12))
-    .addElements(new GameActionInfo("oxblood", "red", 2, new Instant(0L)).plus(Duration.standardSeconds(22)),
-                 new GameActionInfo("scarlet", "red", 4, new Instant(0L).plus(Duration.standardMinutes(2))))
-    .advanceProcessingTime(Duration.standardMinutes(15))
-    .advanceWatermarkToInfinity();
-
-PCollection<KV<String, Integer>> userScores =
-    p.apply(infos).apply(new CalculateUserScores(ALLOWED_LATENESS));
+
TestStream.create(AvroCoder.of(GameActionInfo.class))
+    .addElements(new GameActionInfo("scarlet", "red", 3, new Instant(0L)),
+                 new GameActionInfo("scarlet", "red", 2, new Instant(0L).plus(Duration.standardMinutes(1))))
+    .advanceProcessingTime(Duration.standardMinutes(12))
+    .addElements(new GameActionInfo("oxblood", "red", 2, new Instant(0L)).plus(Duration.standardSeconds(22)),
+                 new GameActionInfo("scarlet", "red", 4, new Instant(0L).plus(Duration.standardMinutes(2))))
+    .advanceProcessingTime(Duration.standardMinutes(15))
+    .advanceWatermarkToInfinity();
+
+PCollection<KV<String, Integer>> userScores =
+    p.apply(infos).apply(new CalculateUserScores(ALLOWED_LATENESS));
 

Elements all arrive before the watermark, and are produced in the on-time pane

-
PAssert.that(userScores)
-       .inEarlyGlobalWindowPanes()
-       .containsInAnyOrder(KV.of("scarlet", 5),
-                           KV.of("scarlet", 9),
-                           KV.of("oxblood", 2));
+
PAssert.that(userScores)
+       .inEarlyGlobalWindowPanes()
+       .containsInAnyOrder(KV.of("scarlet", 5),
+                           KV.of("scarlet", 9),
+                           KV.of("oxblood", 2));
 
-p.run();
+p.run();
 
diff --git a/content/contribute/contribution-guide/index.html b/content/contribute/contribution-guide/index.html index 317748cc9a997..95d88d08947c6 100644 --- a/content/contribute/contribution-guide/index.html +++ b/content/contribute/contribution-guide/index.html @@ -224,7 +224,7 @@

Apache Beam Contribution Guide

Engage

Mailing list(s)

-

We discuss design and implementation issues on dev@beam.incubator.apache.org mailing list, which is archived here. Join by emailing dev-subscribe@beam.incubator.apache.org.

+

We discuss design and implementation issues on dev@beam.incubator.apache.org mailing list, which is archived here. Join by emailing dev-subscribe@beam.incubator.apache.org.

If interested, you can also join the other mailing lists too.

@@ -517,20 +517,11 @@

Website

The Beam website is in the Beam Site GitHub mirror repository in the asf-site branch (not master).

-

The repository contains:

- -
    -
  1. _<directory>: the infrastructure for building, testing, and styling the website
  2. -
  3. _posts: blog post entries
  4. -
  5. use/, learn/, and contribute/: markdown files containing the bulk of the content
  6. -
  7. content/: html generated from the markdown by jekyll (which is what is actually hosted on the website)
  8. -
-

Issues are tracked in the website component in JIRA.

One-time Setup

-

The website uses Jekyll, a ruby-based development framework. The README file in the website repository has more information on how to set up the required dependencies for your development environment.

+

The README file in the website repository has more information on how to set up the required dependencies for your development environment.

The general guidelines for cloning a repository can be adjusted to use the asf-site branch of incubator-beam-site:

@@ -546,7 +537,7 @@

Working on your change

While you are working on your pull request, you can test and develop live by running the following command in the root folder of the website:

-
$ jekyll serve
+
$ bundle exec jekyll serve
 
@@ -554,7 +545,7 @@

Working on your change

In addition, you can run the tests to valid your links using:

-
$ rake test
+
$ bundle exec rake test
 
diff --git a/content/css/main.css b/content/css/main.css index 1fb1692b6443c..f189274f6002c 100644 --- a/content/css/main.css +++ b/content/css/main.css @@ -26,8 +26,7 @@ body { */ h1, h2, h3, h4, h5, h6, p, blockquote, pre, -ul, ol, dl, figure, -.highlight { +ul, ol, dl, figure { margin-bottom: 15px; } /** @@ -326,8 +325,7 @@ pre { color: #998; font-style: italic; } .highlight .err { - color: #a61717; - background-color: #e3d2d2; } + color: #a61717; } .highlight .k { font-weight: bold; } .highlight .o { diff --git a/content/feed.xml b/content/feed.xml index b36be599608da..9d5de98f241d8 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://beam.incubator.apache.org/ - Thu, 20 Oct 2016 17:09:19 -0700 - Thu, 20 Oct 2016 17:09:19 -0700 + Tue, 25 Oct 2016 21:24:51 -0700 + Tue, 25 Oct 2016 21:24:51 -0700 Jekyll v3.2.0 @@ -115,17 +115,17 @@ associated pane.</p> watermark and provide the result PCollection as input to the CalculateTeamScores PTransform:</p> -<div class="highlighter-rouge"><pre class="highlight"><code>TestStream&lt;GameActionInfo&gt; infos = TestStream.create(AvroCoder.of(GameActionInfo.class)) - .addElements(new GameActionInfo("sky", "blue", 12, new Instant(0L)), -                 new GameActionInfo("navy", "blue", 3, new Instant(0L)), -                 new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3)))) -    // Move the watermark past the end the end of the window - .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION) -                                 .plus(Duration.standardMinutes(1))) - .advanceWatermarkToInfinity(); - -PCollection&lt;KV&lt;String, Integer&gt;&gt; teamScores = p.apply(createEvents) - .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">TestStream</span><span class="o">&lt;</span><span class="n">GameActionInfo</span><span class="o">&gt;</span> <span class="n">infos</span> <span class="o">=</span> <span class="n">TestStream</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">AvroCoder</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">GameActionInfo</span><span class="o">.</span><span class="na">class</span><span class="o">))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"sky"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">12</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">)),</span> + <span class="err">                </span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"navy"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">)),</span> + <span class="err">                </span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"navy"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">3</span><span class="o">))))</span> + <span class="err">   </span><span class="c1">// Move the watermark past the end the end of the window</span> + <span class="o">.</span><span class="na">advanceWatermarkTo</span><span class="o">(</span><span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">)</span> + <span class="err">                               </span> <span class="o">.</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">1</span><span class="o">)))</span> + <span class="o">.</span><span class="na">advanceWatermarkToInfinity</span><span class="o">();</span> + +<span class="n">PCollection</span><span class="o">&lt;</span><span class="n">KV</span><span class="o">&lt;</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">&gt;&gt;</span> <span class="n">teamScores</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">createEvents</span><span class="o">)</span> + <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="k">new</span> <span class="n">CalculateTeamScores</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">,</span> <span class="n">ALLOWED_LATENESS</span><span class="o">));</span> </code></pre> </div> @@ -133,11 +133,11 @@ PCollection&lt;KV&lt;String, Integer&gt;&gt; teamScores = p.appl <p><img class="center-block" src="/images/blog/test-stream/elements-all-on-time.png" alt="Elements all arrive before the watermark, and are produced in the on-time pane" width="442" /></p> -<div class="highlighter-rouge"><pre class="highlight"><code>// Only one value is emitted for the blue team -PAssert.that(teamScores) - .inWindow(window) - .containsInAnyOrder(KV.of("blue", 18)); -p.run(); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="c1">// Only one value is emitted for the blue team</span> +<span class="n">PAssert</span><span class="o">.</span><span class="na">that</span><span class="o">(</span><span class="n">teamScores</span><span class="o">)</span> + <span class="o">.</span><span class="na">inWindow</span><span class="o">(</span><span class="n">window</span><span class="o">)</span> + <span class="o">.</span><span class="na">containsInAnyOrder</span><span class="o">(</span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"blue"</span><span class="o">,</span> <span class="mi">18</span><span class="o">));</span> +<span class="n">p</span><span class="o">.</span><span class="na">run</span><span class="o">();</span> </code></pre> </div> @@ -149,27 +149,27 @@ of the window (shown below to the left of the red watermark), which demonstrates the system to be on time, as it arrives before the watermark passes the end of the window</p> -<div class="highlighter-rouge"><pre class="highlight"><code>TestStream&lt;GameActionInfo&gt; infos = TestStream.create(AvroCoder.of(GameActionInfo.class)) - .addElements(new GameActionInfo("sky", "blue", 3, new Instant(0L)), -         new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3)))) -    // Move the watermark up to "near" the end of the window - .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION) -                                 .minus(Duration.standardMinutes(1))) - .addElements(new GameActionInfo("sky", "blue", 12, Duration.ZERO)) - .advanceWatermarkToInfinity(); - -PCollection&lt;KV&lt;String, Integer&gt;&gt; teamScores = p.apply(createEvents) - .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">TestStream</span><span class="o">&lt;</span><span class="n">GameActionInfo</span><span class="o">&gt;</span> <span class="n">infos</span> <span class="o">=</span> <span class="n">TestStream</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">AvroCoder</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">GameActionInfo</span><span class="o">.</span><span class="na">class</span><span class="o">))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"sky"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">)),</span> + <span class="err">       </span> <span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"navy"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">3</span><span class="o">))))</span> + <span class="err">   </span><span class="c1">// Move the watermark up to "near" the end of the window</span> + <span class="o">.</span><span class="na">advanceWatermarkTo</span><span class="o">(</span><span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">)</span> + <span class="err">               </span> <span class="err">                </span><span class="o">.</span><span class="na">minus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">1</span><span class="o">)))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"sky"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">12</span><span class="o">,</span> <span class="n">Duration</span><span class="o">.</span><span class="na">ZERO</span><span class="o">))</span> + <span class="o">.</span><span class="na">advanceWatermarkToInfinity</span><span class="o">();</span> + +<span class="n">PCollection</span><span class="o">&lt;</span><span class="n">KV</span><span class="o">&lt;</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">&gt;&gt;</span> <span class="n">teamScores</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">createEvents</span><span class="o">)</span> + <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="k">new</span> <span class="n">CalculateTeamScores</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">,</span> <span class="n">ALLOWED_LATENESS</span><span class="o">));</span> </code></pre> </div> <p><img class="center-block" src="/images/blog/test-stream/elements-unobservably-late.png" alt="An element arrives late, but before the watermark passes the end of the window, and is produced in the on-time pane" width="442" /></p> -<div class="highlighter-rouge"><pre class="highlight"><code>// Only one value is emitted for the blue team -PAssert.that(teamScores) - .inWindow(window) - .containsInAnyOrder(KV.of("blue", 18)); -p.run(); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="c1">// Only one value is emitted for the blue team</span> +<span class="n">PAssert</span><span class="o">.</span><span class="na">that</span><span class="o">(</span><span class="n">teamScores</span><span class="o">)</span> + <span class="o">.</span><span class="na">inWindow</span><span class="o">(</span><span class="n">window</span><span class="o">)</span> + <span class="o">.</span><span class="na">containsInAnyOrder</span><span class="o">(</span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"blue"</span><span class="o">,</span> <span class="mi">18</span><span class="o">));</span> +<span class="n">p</span><span class="o">.</span><span class="na">run</span><span class="o">();</span> </code></pre> </div> @@ -179,31 +179,31 @@ p.run(); demonstrate the triggering behavior that causes the system to emit an on-time pane, and then after the late data arrives, a pane that refines the result.</p> -<div class="highlighter-rouge"><pre class="highlight"><code>TestStream&lt;GameActionInfo&gt; infos = TestStream.create(AvroCoder.of(GameActionInfo.class)) - .addElements(new GameActionInfo("sky", "blue", 3, new Instant(0L)), -           new GameActionInfo("navy", "blue", 3, new Instant(0L).plus(Duration.standardMinutes(3)))) -    // Move the watermark up to "near" the end of the window - .advanceWatermarkTo(new Instant(0L).plus(TEAM_WINDOW_DURATION) -                                 .minus(Duration.standardMinutes(1))) - .addElements(new GameActionInfo("sky", "blue", 12, Duration.ZERO)) - .advanceWatermarkToInfinity(); - -PCollection&lt;KV&lt;String, Integer&gt;&gt; teamScores = p.apply(createEvents) - .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">TestStream</span><span class="o">&lt;</span><span class="n">GameActionInfo</span><span class="o">&gt;</span> <span class="n">infos</span> <span class="o">=</span> <span class="n">TestStream</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">AvroCoder</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">GameActionInfo</span><span class="o">.</span><span class="na">class</span><span class="o">))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"sky"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">)),</span> + <span class="err">          </span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"navy"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">3</span><span class="o">))))</span> +<span class="err"> </span> <span class="err">  </span><span class="c1">// Move the watermark up to "near" the end of the window</span> + <span class="o">.</span><span class="na">advanceWatermarkTo</span><span class="o">(</span><span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">)</span> +<span class="err">                                </span> <span class="o">.</span><span class="na">minus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">1</span><span class="o">)))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"sky"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">12</span><span class="o">,</span> <span class="n">Duration</span><span class="o">.</span><span class="na">ZERO</span><span class="o">))</span> + <span class="o">.</span><span class="na">advanceWatermarkToInfinity</span><span class="o">();</span> + +<span class="n">PCollection</span><span class="o">&lt;</span><span class="n">KV</span><span class="o">&lt;</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">&gt;&gt;</span> <span class="n">teamScores</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">createEvents</span><span class="o">)</span> + <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="k">new</span> <span class="n">CalculateTeamScores</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">,</span> <span class="n">ALLOWED_LATENESS</span><span class="o">));</span> </code></pre> </div> <p><img class="center-block" src="/images/blog/test-stream/elements-observably-late.png" alt="Elements all arrive before the watermark, and are produced in the on-time pane" width="442" /></p> -<div class="highlighter-rouge"><pre class="highlight"><code>// An on-time pane is emitted with the events that arrived before the window closed -PAssert.that(teamScores) - .inOnTimePane(window) - .containsInAnyOrder(KV.of("blue", 6)); -// The final pane contains the late refinement -PAssert.that(teamScores) - .inFinalPane(window) - .containsInAnyOrder(KV.of("blue", 18)); -p.run(); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="c1">// An on-time pane is emitted with the events that arrived before the window closed</span> +<span class="n">PAssert</span><span class="o">.</span><span class="na">that</span><span class="o">(</span><span class="n">teamScores</span><span class="o">)</span> + <span class="o">.</span><span class="na">inOnTimePane</span><span class="o">(</span><span class="n">window</span><span class="o">)</span> + <span class="o">.</span><span class="na">containsInAnyOrder</span><span class="o">(</span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"blue"</span><span class="o">,</span> <span class="mi">6</span><span class="o">));</span> +<span class="c1">// The final pane contains the late refinement</span> +<span class="n">PAssert</span><span class="o">.</span><span class="na">that</span><span class="o">(</span><span class="n">teamScores</span><span class="o">)</span> + <span class="o">.</span><span class="na">inFinalPane</span><span class="o">(</span><span class="n">window</span><span class="o">)</span> + <span class="o">.</span><span class="na">containsInAnyOrder</span><span class="o">(</span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"blue"</span><span class="o">,</span> <span class="mi">18</span><span class="o">));</span> +<span class="n">p</span><span class="o">.</span><span class="na">run</span><span class="o">();</span> </code></pre> </div> @@ -213,33 +213,33 @@ p.run(); configured allowed lateness, we can demonstrate that the late element is dropped by the system.</p> -<div class="highlighter-rouge"><pre class="highlight"><code>TestStream&lt;GameActionInfo&gt; infos = TestStream.create(AvroCoder.of(GameActionInfo.class)) - .addElements(new GameActionInfo("sky", "blue", 3, Duration.ZERO), -          new GameActionInfo("navy", "blue", 3, Duration.standardMinutes(3))) -    // Move the watermark up to "near" the end of the window - .advanceWatermarkTo(new Instant(0).plus(TEAM_WINDOW_DURATION) -                                         .plus(ALLOWED_LATENESS) -                                         .plus(Duration.standardMinutes(1))) - .addElements(new GameActionInfo( -                     "sky", -                     "blue", -                     12, -                     new Instant(0).plus(TEAM_WINDOW_DURATION).minus(Duration.standardMinutes(1)))) - .advanceWatermarkToInfinity(); - -PCollection&lt;KV&lt;String, Integer&gt;&gt; teamScores = p.apply(createEvents) - .apply(new CalculateTeamScores(TEAM_WINDOW_DURATION, ALLOWED_LATENESS)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">TestStream</span><span class="o">&lt;</span><span class="n">GameActionInfo</span><span class="o">&gt;</span> <span class="n">infos</span> <span class="o">=</span> <span class="n">TestStream</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">AvroCoder</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">GameActionInfo</span><span class="o">.</span><span class="na">class</span><span class="o">))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"sky"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="n">Duration</span><span class="o">.</span><span class="na">ZERO</span><span class="o">),</span> + <span class="err">         </span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"navy"</span><span class="o">,</span> <span class="s">"blue"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">3</span><span class="o">)))</span> +<span class="err">    </span><span class="c1">// Move the watermark up to "near" the end of the window</span> + <span class="o">.</span><span class="na">advanceWatermarkTo</span><span class="o">(</span><span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">)</span> + <span class="err">                                        </span><span class="o">.</span><span class="na">plus</span><span class="o">(</span><span class="n">ALLOWED_LATENESS</span><span class="o">)</span> + <span class="err">                                        </span><span class="o">.</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">1</span><span class="o">)))</span> + <span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span> +<span class="err">                     </span><span class="s">"sky"</span><span class="o">,</span> +<span class="err">                     </span><span class="s">"blue"</span><span class="o">,</span> +<span class="err">                     </span><span class="mi">12</span><span class="o">,</span> +<span class="err">                     </span><span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">).</span><span class="na">minus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">1</span><span class="o">))))</span> + <span class="o">.</span><span class="na">advanceWatermarkToInfinity</span><span class="o">();</span> + +<span class="n">PCollection</span><span class="o">&lt;</span><span class="n">KV</span><span class="o">&lt;</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">&gt;&gt;</span> <span class="n">teamScores</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">createEvents</span><span class="o">)</span> + <span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="k">new</span> <span class="n">CalculateTeamScores</span><span class="o">(</span><span class="n">TEAM_WINDOW_DURATION</span><span class="o">,</span> <span class="n">ALLOWED_LATENESS</span><span class="o">));</span> </code></pre> </div> <p><img class="center-block" src="/images/blog/test-stream/elements-droppably-late.png" alt="Elements all arrive before the watermark, and are produced in the on-time pane" width="442" /></p> -<div class="highlighter-rouge"><pre class="highlight"><code>// An on-time pane is emitted with the events that arrived before the window closed -PAssert.that(teamScores) - .inWindow(window) - .containsInAnyOrder(KV.of("blue", 6)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="c1">// An on-time pane is emitted with the events that arrived before the window closed</span> +<span class="n">PAssert</span><span class="o">.</span><span class="na">that</span><span class="o">(</span><span class="n">teamScores</span><span class="o">)</span> + <span class="o">.</span><span class="na">inWindow</span><span class="o">(</span><span class="n">window</span><span class="o">)</span> + <span class="o">.</span><span class="na">containsInAnyOrder</span><span class="o">(</span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"blue"</span><span class="o">,</span> <span class="mi">6</span><span class="o">));</span> -p.run(); +<span class="n">p</span><span class="o">.</span><span class="na">run</span><span class="o">();</span> </code></pre> </div> @@ -249,29 +249,29 @@ triggers by advancing the processing time of the TestStream. If we add elements to an input PCollection, occasionally advancing the processing time clock, and apply <code class="highlighter-rouge">CalculateUserScores</code></p> -<div class="highlighter-rouge"><pre class="highlight"><code>TestStream.create(AvroCoder.of(GameActionInfo.class)) -    .addElements(new GameActionInfo("scarlet", "red", 3, new Instant(0L)), -                new GameActionInfo("scarlet", "red", 2, new Instant(0L).plus(Duration.standardMinutes(1)))) - .advanceProcessingTime(Duration.standardMinutes(12)) -    .addElements(new GameActionInfo("oxblood", "red", 2, new Instant(0L)).plus(Duration.standardSeconds(22)), -                new GameActionInfo("scarlet", "red", 4, new Instant(0L).plus(Duration.standardMinutes(2)))) - .advanceProcessingTime(Duration.standardMinutes(15)) - .advanceWatermarkToInfinity(); - -PCollection&lt;KV&lt;String, Integer&gt;&gt; userScores = -    p.apply(infos).apply(new CalculateUserScores(ALLOWED_LATENESS)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">TestStream</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">AvroCoder</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">GameActionInfo</span><span class="o">.</span><span class="na">class</span><span class="o">))</span> + <span class="err">   </span><span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"scarlet"</span><span class="o">,</span> <span class="s">"red"</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">)),</span> + <span class="err">              </span> <span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"scarlet"</span><span class="o">,</span> <span class="s">"red"</span><span class="o">,</span> <span class="mi">2</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">1</span><span class="o">))))</span> + <span class="o">.</span><span class="na">advanceProcessingTime</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">12</span><span class="o">))</span> + <span class="err">   </span><span class="o">.</span><span class="na">addElements</span><span class="o">(</span><span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"oxblood"</span><span class="o">,</span> <span class="s">"red"</span><span class="o">,</span> <span class="mi">2</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">)).</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardSeconds</span><span class="o">(</span><span class="mi">22</span><span class="o">)),</span> + <span class="err">              </span> <span class="k">new</span> <span class="n">GameActionInfo</span><span class="o">(</span><span class="s">"scarlet"</span><span class="o">,</span> <span class="s">"red"</span><span class="o">,</span> <span class="mi">4</span><span class="o">,</span> <span class="k">new</span> <span class="n">Instant</span><span class="o">(</span><span class="mi">0L</span><span class="o">).</span><span class="na">plus</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">2</span><span class="o">))))</span> + <span class="o">.</span><span class="na">advanceProcessingTime</span><span class="o">(</span><span class="n">Duration</span><span class="o">.</span><span class="na">standardMinutes</span><span class="o">(</span><span class="mi">15</span><span class="o">))</span> + <span class="o">.</span><span class="na">advanceWatermarkToInfinity</span><span class="o">();</span> + +<span class="n">PCollection</span><span class="o">&lt;</span><span class="n">KV</span><span class="o">&lt;</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">&gt;&gt;</span> <span class="n">userScores</span> <span class="o">=</span> + <span class="err">   </span><span class="n">p</span><span class="o">.</span><span class="na">apply</span><span class="o">(</span><span class="n">infos</span><span class="o">).</span><span class="na">apply</span><span class="o">(</span><span class="k">new</span> <span class="n">CalculateUserScores</span><span class="o">(</span><span class="n">ALLOWED_LATENESS</span><span class="o">));</span> </code></pre> </div> <p><img class="center-block" src="/images/blog/test-stream/elements-processing-speculative.png" alt="Elements all arrive before the watermark, and are produced in the on-time pane" width="442" /></p> -<div class="highlighter-rouge"><pre class="highlight"><code>PAssert.that(userScores) - .inEarlyGlobalWindowPanes() - .containsInAnyOrder(KV.of("scarlet", 5), -                         KV.of("scarlet", 9), - KV.of("oxblood", 2)); +<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">PAssert</span><span class="o">.</span><span class="na">that</span><span class="o">(</span><span class="n">userScores</span><span class="o">)</span> + <span class="o">.</span><span class="na">inEarlyGlobalWindowPanes</span><span class="o">()</span> + <span class="o">.</span><span class="na">containsInAnyOrder</span><span class="o">(</span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"scarlet"</span><span class="o">,</span> <span class="mi">5</span><span class="o">),</span> + <span class="err">                        </span><span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"scarlet"</span><span class="o">,</span> <span class="mi">9</span><span class="o">),</span> + <span class="n">KV</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"oxblood"</span><span class="o">,</span> <span class="mi">2</span><span class="o">));</span> -p.run(); +<span class="n">p</span><span class="o">.</span><span class="na">run</span><span class="o">();</span> </code></pre> </div> @@ -621,7 +621,7 @@ PCollection&lt;O&gt; output = input </li> </ul> -<p>As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the <a href="/learn/presentation-materials/">Apache Beam presentation materials</a> page or email the <a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#117;&#115;&#101;&#114;&#064;&#098;&#101;&#097;&#109;&#046;&#105;&#110;&#099;&#117;&#098;&#097;&#116;&#111;&#114;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;"><code class="highlighter-rouge">user@beam.incubator.apache.org</code></a> mailing list with your ideas or questions.</p> +<p>As Apache Beam grows, so will this repository of presentation materials. We are excited to add new materials as the Apache Beam ecosystem grows with new runners, SDKs, and so on. If you are interested in contributing content or have a request, please see the <a href="/learn/presentation-materials/">Apache Beam presentation materials</a> page or email the <a href="mailto:user@beam.incubator.apache.org"><code class="highlighter-rouge">user@beam.incubator.apache.org</code></a> mailing list with your ideas or questions.</p> Sun, 03 Apr 2016 11:00:00 -0700 http://beam.incubator.apache.org/beam/capability/2016/04/03/presentation-materials.html diff --git a/content/learn/presentation-materials/index.html b/content/learn/presentation-materials/index.html index 85d5bbb8707e6..2f629611c0cbe 100644 --- a/content/learn/presentation-materials/index.html +++ b/content/learn/presentation-materials/index.html @@ -158,17 +158,17 @@

Apache Beam Presentation MaterialsUsing these materials

These materials can be used by anyone. If you do use these materials, please remember to give credit to the authors cited in each presentation. These materials include speaker notes and you can view them by clicking View and selecting Show speaker notes. Some presentations have [Long] or [Short] in their name, such as the Apache Beam Model, which means there are two different presentations (a long and short version) depending on your needs and time constraints.

-

If you have any questions about the these materials, please ask on the user@beam.incubator.apache.org mailing list.

+

If you have any questions about the these materials, please ask on the user@beam.incubator.apache.org mailing list.

-

The Apache Beam community is looking for feedback on these materials to improve them over time. If you deliver a presentation based on these materials, we’d love to hear from you on the user@beam.incubator.apache.org mailing list. Please share your thoughts, questions you were asked, and (if available and appropriate) a link to the recording of your talk so the rest of the Apache Beam community can benefit from it.

+

The Apache Beam community is looking for feedback on these materials to improve them over time. If you deliver a presentation based on these materials, we’d love to hear from you on the user@beam.incubator.apache.org mailing list. Please share your thoughts, questions you were asked, and (if available and appropriate) a link to the recording of your talk so the rest of the Apache Beam community can benefit from it.

Contributing new content

The Apache Beam community is always looking for new core content and contributions!

-

If you are interested in contributing new content, we recommend you first search the Apache Beam JIRA to see if your idea currently has an open JIRA item. If it does not, send an email to the user@beam.incubator.apache.org mailing list or open a new issue in the Apache Beam JIRA using the website component. Based on community submissions or contributions, the Beam community collaboratively reviews or creates new content and then adds it to this shared space for everyone to use.

+

If you are interested in contributing new content, we recommend you first search the Apache Beam JIRA to see if your idea currently has an open JIRA item. If it does not, send an email to the user@beam.incubator.apache.org mailing list or open a new issue in the Apache Beam JIRA using the website component. Based on community submissions or contributions, the Beam community collaboratively reviews or creates new content and then adds it to this shared space for everyone to use.

Corrections and updates

-

If you encounter missing, incorrect, or out of date material, please make a comment in the presentation or send an email to the user@beam.incubator.apache.org mailing list.

+

If you encounter missing, incorrect, or out of date material, please make a comment in the presentation or send an email to the user@beam.incubator.apache.org mailing list.

diff --git a/content/learn/programming-guide/index.html b/content/learn/programming-guide/index.html index 8d2f5c1d70dcf..18e1fa61d3f35 100644 --- a/content/learn/programming-guide/index.html +++ b/content/learn/programming-guide/index.html @@ -296,7 +296,7 @@

Creating a PCollection from }

-

### PCollection Characteristics

+

PCollection Characteristics

A PCollection is owned by the specific Pipeline object for which it is created; multiple pipelines cannot share a PCollection. In some respects, a PCollection functions like a collection class. However, a PCollection can differ in a few key ways:

diff --git a/content/learn/runners/capability-matrix/index.html b/content/learn/runners/capability-matrix/index.html index 2c7a606a0b812..debddb166ec2b 100644 --- a/content/learn/runners/capability-matrix/index.html +++ b/content/learn/runners/capability-matrix/index.html @@ -146,7 +146,7 @@

Beam Capability Matrix

-

Last updated: 2016-10-20 17:09 PDT

+

Last updated: 2016-10-25 21:24 PDT

Apache Beam (incubating) provides a portable API layer for building sophisticated data-parallel processing engines that may be executed across a diversity of exeuction engines, or runners. The core concepts of this layer are based upon the Beam Model (formerly referred to as the Dataflow Model), and implemented to varying degrees in each Beam runner. To help clarify the capabilities of individual runners, we’ve created the capability matrix below.

diff --git a/content/project/public-meetings/index.html b/content/project/public-meetings/index.html index 78417f6976f9f..345ae3a8ae615 100644 --- a/content/project/public-meetings/index.html +++ b/content/project/public-meetings/index.html @@ -201,20 +201,20 @@

Apache Beam Public Meetings

I want to give a public talk about Apache Beam

To get started, we recommend you review the Apache Beam presentation materials page to review the content the Apache Beam community has already created. These materials will possibly save you time and energy as you create content for your event.

-

Once you have scheduled your event, we welcome you to announce it on the user@beam.incubator.apache.org mailing list. Additionally, please open a JIRA item using the website component with details so we can update this page.

+

Once you have scheduled your event, we welcome you to announce it on the user@beam.incubator.apache.org mailing list. Additionally, please open a JIRA item using the website component with details so we can update this page.

-

If you have any questions as you prepare for your event, we recommend you reach out to the Apache Beam community through the dev@beam.incubator.apache.org mailing list. The Beam community can help provide feedback on your materials and promote your event.

+

If you have any questions as you prepare for your event, we recommend you reach out to the Apache Beam community through the dev@beam.incubator.apache.org mailing list. The Beam community can help provide feedback on your materials and promote your event.

Frequently asked questions about public meetings

How can I remotely attend these meetings?

-

We are investigating methods, such as video conferencing and IRC chat, to allow anyone to attend remotely. At present we do not have an estimated date. If you are interested in this option, please send an email to the user@beam.incubator.apache.org with your ideas and recommendations.

+

We are investigating methods, such as video conferencing and IRC chat, to allow anyone to attend remotely. At present we do not have an estimated date. If you are interested in this option, please send an email to the user@beam.incubator.apache.org with your ideas and recommendations.

What is a public meeting?

Public meetings include scheduled Apache Beam Dev/PPMC meetings, Meetup events, conference talks, and other events where the public meets to discuss Beam.

How do I learn about new meetings?

-

The Apache Beam community announces upcoming public meetings on the dev@beam.incubator.apache.org mailing list. If you want to learn about new events, we recommend you subscribe to that list. If you are holding a public event, please send an email to the dev@ list.

+

The Apache Beam community announces upcoming public meetings on the dev@beam.incubator.apache.org mailing list. If you want to learn about new events, we recommend you subscribe to that list. If you are holding a public event, please send an email to the dev@ list.

diff --git a/content/styles/site.css b/content/styles/site.css index afba989386d55..0aa93eccfab20 100644 --- a/content/styles/site.css +++ b/content/styles/site.css @@ -5809,6 +5809,146 @@ button.close { @media print { .hidden-print { display: none !important; } } +/** + * Syntax highlighting styles + */ +.highlight { + background: #fff; } + .highlighter-rouge .highlight { + background: #eef; } + .highlight .c { + color: #998; + font-style: italic; } + .highlight .err { + color: #a61717; } + .highlight .k { + font-weight: bold; } + .highlight .o { + font-weight: bold; } + .highlight .cm { + color: #998; + font-style: italic; } + .highlight .cp { + color: #999; + font-weight: bold; } + .highlight .c1 { + color: #998; + font-style: italic; } + .highlight .cs { + color: #999; + font-weight: bold; + font-style: italic; } + .highlight .gd { + color: #000; + background-color: #fdd; } + .highlight .gd .x { + color: #000; + background-color: #faa; } + .highlight .ge { + font-style: italic; } + .highlight .gr { + color: #a00; } + .highlight .gh { + color: #999; } + .highlight .gi { + color: #000; + background-color: #dfd; } + .highlight .gi .x { + color: #000; + background-color: #afa; } + .highlight .go { + color: #888; } + .highlight .gp { + color: #555; } + .highlight .gs { + font-weight: bold; } + .highlight .gu { + color: #aaa; } + .highlight .gt { + color: #a00; } + .highlight .kc { + font-weight: bold; } + .highlight .kd { + font-weight: bold; } + .highlight .kp { + font-weight: bold; } + .highlight .kr { + font-weight: bold; } + .highlight .kt { + color: #458; + font-weight: bold; } + .highlight .m { + color: #099; } + .highlight .s { + color: #d14; } + .highlight .na { + color: #008080; } + .highlight .nb { + color: #0086B3; } + .highlight .nc { + color: #458; + font-weight: bold; } + .highlight .no { + color: #008080; } + .highlight .ni { + color: #800080; } + .highlight .ne { + color: #900; + font-weight: bold; } + .highlight .nf { + color: #900; + font-weight: bold; } + .highlight .nn { + color: #555; } + .highlight .nt { + color: #000080; } + .highlight .nv { + color: #008080; } + .highlight .ow { + font-weight: bold; } + .highlight .w { + color: #bbb; } + .highlight .mf { + color: #099; } + .highlight .mh { + color: #099; } + .highlight .mi { + color: #099; } + .highlight .mo { + color: #099; } + .highlight .sb { + color: #d14; } + .highlight .sc { + color: #d14; } + .highlight .sd { + color: #d14; } + .highlight .s2 { + color: #d14; } + .highlight .se { + color: #d14; } + .highlight .sh { + color: #d14; } + .highlight .si { + color: #d14; } + .highlight .sx { + color: #d14; } + .highlight .sr { + color: #009926; } + .highlight .s1 { + color: #d14; } + .highlight .ss { + color: #990073; } + .highlight .bp { + color: #999; } + .highlight .vc { + color: #008080; } + .highlight .vg { + color: #008080; } + .highlight .vi { + color: #008080; } + .highlight .il { + color: #099; } + /* What/Where/When/How colors. */ .wwwh-what-dark { color: #ca1;