feat: frame animations with time encoding and timer param #8921

jonathanzong · 2023-05-25T02:10:59Z

This change implements basic features of Animated Vega-Lite. With this change, users can create frame animations using a time encoding, a timer point selection, and a filter transform.

This change does not include more complex features e.g.: interpolation, custom predicates, rescale, interactive sliders, or data-driven pausing.

Adds a time encoding channel
Adds isTimerSelection function to check if a selection is an animation selections
Builds _curr animation dataset for timer selections to store the current animation frame
Adds animation signals to track the elapsed time (anim_clock), current animation value (anim_value), current position in the animation field's domain (t_index), etc.
When time encoding is present, updates associated marks' from.data to use the animation dataset (current frame).

Relevant issue: #4060

Coauthor: @joshpoll

Example specs

Hop example:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "url": "data/seattle-weather.csv"
  },
  "mark": "tick",
  "config": {
    "tick": {
      "thickness": 3
    }
  },
  "params": [
    {
      "name": "date",
      "select": {
        "type": "point",
        "fields": [
          "date"
        ],
        "on": "timer"
      }
    }
  ],
  "transform": [
    {
      "filter": {
        "param": "date"
      }
    }
  ],
  "encoding": {
    "y": {
      "field": "precipitation",
      "type": "quantitative"
    },
    "time": {
      "field": "date"
    }
  }
}

Gapminder:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "url": "data/gapminder.json"
  },
  "mark": "point",
  "params": [
    {
      "name": "avl",
      "select": {
        "type": "point",
        "fields": [
          "year"
        ],
        "on": "timer"
      }
    }
  ],
  "transform": [
    {
      "filter": {
        "param": "avl"
      }
    }
  ],
  "encoding": {
    "color": {
      "field": "country"
    },
    "x": {
      "field": "fertility",
      "type": "quantitative"
    },
    "y": {
      "field": "life_expect",
      "type": "quantitative"
    },
    "time": {
      "field": "year"
    }
  }
}

jonathanzong · 2023-05-25T22:33:15Z

@domoritz @arvind this is ready for review. we were thinking hold off on docs and examples until a later PR, but let us know if you'd rather us include it here

domoritz · 2023-05-28T09:29:52Z

Thanks. Can we see whether you can run the formatting action as well so we know the formatting doesn't break.

jonathanzong · 2023-05-29T16:26:31Z

i was able to run yarn format with no problems, we've been using the eslint config

domoritz · 2023-05-30T09:17:07Z

Great. I was hoping we could get the GitHub action for checking formatting to work somehow.

joshpoll · 2023-05-30T14:49:30Z

I think it's running in the Runtime, Linting, and Coverage test.

src/compile/scale/range.ts

src/compile/selection/parse.ts

src/compile/selection/point.ts

arvind · 2023-08-07T16:43:50Z

src/compile/selection/point.ts

+export const CURR = '_curr';
+
+const animationSignals = (selectionName: string, scaleName: string): Signal[] => {
+  return [


Are these selection-specific signals, or global signals (i.e., one set of signals that are used by all selections)? I think it's the former based on some of the references within. In which case, I think they'll all need to be prefixed by the selection name. Otherwise, if you have multiple timer selections within the same unit spec, you're going to get a duplicate signal error.

i think deciding what's selection-specific vs global matters for two situations:

multiple timer selections in the same unit spec

i don't think this should be allowed, because what does it mean to animate a single set of marks two different ways? what's the best way to disallow this?

multiple timer selections across multiple unit specs in a multi-view

we ideally want the animation clock ANIM_CLOCK to be global, so we can share a common timer across multiple units. but the problem is that the animation clock needs to know the MAX_RANGE_EXTENT to know when to wrap back around to 0, but this is calculated from the scale for each unit. imo for now this is fine since we didn't consider multi-view in the scope of the paper, but we'll have to figure this out when we think about multi-view / scale resolution. the extent of the clock needs to be determined from the unioned scale or the max of the independent scale extents

Both of these make sense to me. But, can we add some error checking and logging (see src/log) to guard against the compiler throwing a really hairy error because of duplicate signals (either at the top level or nested within a unit). Eg, during parsing, we can check if more than one timer selection is defined and ignore them with a warning message.

Re: (2) a global ANIM_CLOCK, I agree that we didn't consider multi-view in the paper, but I feel a little strange to introduce this constrain in production since, from an end user perspective, it might seem like an arbitrary limitation? Could you humor me and help me think through how complex this would be to implement? If the issue is primarily about how to resolve MAX_RANGE_EXTENT for wrapping around back to 0, are there more than these two cases we would need to handle:

No global ANIM_CLOCK: so each unit can have only one timer selection, and they are independent and loop independently.

A shared ANIM_CLOCK: they would wrap around at the same time; that means, you essentially would need to max over the constituent MAX_RANGE_EXTENT and then use that to determine the wrap around. The effect would be that one timer would "pause" while the other(s) finish, and then they wrap.

These two cases feel like they map well to VL's existing notion of resolve (which is used for scales/axes) and which we overload for selection predicates in terms of union/intersect. So I feel like there's conceptual machinery already in place that we might be able to piggy back off?

logging and unit tests added

https://github.com/vega/vega-lite/pull/8921/files/25d1b1c02b4aac8c2cbc72758eb36dda7cedb2ef..c2b4b59bad9f2e6b6b3fc180c5009e56dd61fd92

option 2 sounds like what we want eventually. i looked back at what we did for the paper and we did implement layer and vconcat in the prototypes. but in our prototypes these were two separate implementations with not a lot of overlap. i also spent most of the day trying to get facet to work, and it almost works modulo the range resolution stuff, but i am not really sure how to do it. we knew when we were doing the paper that conceptually there's overlap, but i suspect there isn't overlap in the implementation (at least right now) since we're using signals and not stores, and can't make use of vlSelectionResolve directly

all in all, i think it'd be a pretty big lift to get multi-view working in this PR. when we set out to do this PR, we thought of it as the minimal set of features to get people making basic animations that would cover the most common use case (we also aren't scoping in things like interpolation which i would prioritize over multi-view). i'm hesitant to increase the scope at this point because of 1) how hard it is for me to build velocity in this codebase due to complexity 2) the accessibility libraries being more of a personal priority

src/compile/selection/point.ts

src/compile/selection/toggle.ts

src/compile/unit.ts

arvind

Sorry for the delay, y'all! A great start. I've left line-level comments throughout and will do a more holistic review next. In the meantime, some higher-level thoughts:

Thanks for including example specs in your PR OP. Could you include them as json specs under examples as well please? That'll slurp them into our CI process (and, I think?, make them easier to expose in the documentation)
It might behoove us to also include some runtime tests since they've saved our (namely, my) behind a number of times when the compile-time specs appeared to be correctly constructed? (I'm happy to walk through the runtime test infrastructure post-CHI since it's a little complicated).

jonathanzong · 2023-09-25T19:35:48Z

first round of comments has been addressed. pending todos:

question here feat: frame animations with time encoding and timer param #8921 (comment)
find some time to talk about how runtime testing works

mattijn · 2024-04-06T09:03:22Z

c72cb7d was an attempt to trigger the deployment preview of #9294..

mattijn · 2024-04-07T10:35:46Z

Hi @jonathanzong and @joshpoll!👋 Would you mind updating this PR so the latest changes from the main repository are also included in this branch?

Im not brave enough to do it myself🙈, but I'm sensing that this is the reason why the new deployment preview is not yet triggering.

mattijn · 2024-04-07T11:08:18Z

Just thinking about making a map with the trajectory of the upcoming eclipse🥳

joelostblom · 2024-04-07T14:16:47Z

Just thinking about making a map with the trajectory of the upcoming eclipse🥳

Such a cool idea! Please share if you create it, I would love to see an animated VL chart for this.

Btw, @jonathanzong are you waiting for a review on this branch or are you planning to add more commits (I saw you were still adding more since you last requested a review).

joshpoll · 2024-04-08T22:28:33Z

I rebased

jonathanzong · 2024-04-08T23:11:37Z

We are indeed just waiting for a review

arvind · 2024-04-08T23:40:16Z

Yes, apologies, it's been on my docket for a while but I've been underwater with various other deadlines. My plan is to wrap this up this month 🤞

jonathanzong · 2024-10-29T21:22:32Z

i have rebased, fixed checks, and verified that the animation works as expected in local editor

edit: i'm going to wait to fix the codecov until after we get a review because otherwise we'll just have to do it again

arvind · 2024-10-30T14:42:03Z

examples/specs/animated_gapminder.vl.json

+  "mark": "point",
+  "params": [
+    {
+      "name": "avl",


Minor point but it'd be great to name this in a more semantically-meaningful way vs (what I assume) is an abbreviation of "Animated Vega-Lite".

arvind · 2024-10-30T15:36:54Z

src/compile/selection/point.ts

+export const CURR = '_curr';
+
+const animationSignals = (selectionName: string, scaleName: string): Signal[] => {
+  return [


Both of these make sense to me. But, can we add some error checking and logging (see src/log) to guard against the compiler throwing a really hairy error because of duplicate signals (either at the top level or nested within a unit). Eg, during parsing, we can check if more than one timer selection is defined and ignore them with a warning message.

arvind · 2024-10-30T16:12:37Z

src/compile/selection/point.ts

+export const CURR = '_curr';
+
+const animationSignals = (selectionName: string, scaleName: string): Signal[] => {
+  return [


Re: (2) a global ANIM_CLOCK, I agree that we didn't consider multi-view in the paper, but I feel a little strange to introduce this constrain in production since, from an end user perspective, it might seem like an arbitrary limitation? Could you humor me and help me think through how complex this would be to implement? If the issue is primarily about how to resolve MAX_RANGE_EXTENT for wrapping around back to 0, are there more than these two cases we would need to handle:

No global ANIM_CLOCK: so each unit can have only one timer selection, and they are independent and loop independently.

A shared ANIM_CLOCK: they would wrap around at the same time; that means, you essentially would need to max over the constituent MAX_RANGE_EXTENT and then use that to determine the wrap around. The effect would be that one timer would "pause" while the other(s) finish, and then they wrap.

These two cases feel like they map well to VL's existing notion of resolve (which is used for scales/axes) and which we overload for selection predicates in terms of union/intersect. So I feel like there's conceptual machinery already in place that we might be able to piggy back off?

jonathanzong · 2024-10-31T22:23:26Z

i have added an error explaining that facet, layer, and concat animations are unsupported when a user tries to create them (and unit tests for this error).

how serious are we about codecov? if you look at what's left, it's codecov asking for:

unit test for consts that i added
unit tests for one-line util functions for the runtime tests
one-line cleanup changes i made to existing logs that previously apparently also did not have coverage

can we just bypass this?

jonathanzong marked this pull request as ready for review May 25, 2023 22:17