Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI: Fix bug where task logs would suddenly switch to another task's logs #8833

Merged
merged 1 commit into from
Sep 8, 2020

Conversation

DingoEatingFuzz
Copy link
Contributor

Fixes #8545

Issue
When streaming logs for a task, the logs will suddenly flip to the logs of a different task for the allocation. When this happens, the entire log output is flipped.

Reproduction

  1. Run a job with multiple tasks (job file at the bottom)
  2. Open the logs for one task in a tab
  3. In another tab, open the task detail page of the other tab.
  4. Restart the task.
  5. Sometimes this will cause the logs of the task being streamed to change to the other tasks

Root cause
A few things are important to know to understand the root cause.

  1. Tasks are modeled as Fragments.
  2. The task logs page watches the allocation, which means whenever it changes server-side it is reloaded client-side.
  3. TaskStates in the API response are a map[string]interface{} that get converted to an array of objects in the UI

So what happens is when the allocation is reloaded, the API response is normalized and pushed into the Ember Data store. When the TaskStates are converted to an array, the order of tasks aren't stable. So when tasks switch places in the array, their respective records in the store are updated as if they are the other. This causes Task2 to become Task1 and vice versa.

Fixing it is simple, just sort the keys when converting objects to arrays to ensure stable model fragments.

Job for Reproduction

job "multi" {
  datacenters = ["dc1"]

  group "cache" {
    task "one" {
      driver = "raw_exec"

      config {
        command = "/bin/bash"
        args = [ "-c", "while true; do echo 'One Fish'; sleep 1; done" ]
      }
    }

    task "two" {
      driver = "raw_exec"

      config {
        command = "/bin/bash"
        args = [ "-c", "while true; do echo 'Two Fish'; sleep 1; done" ]
      }
    }
  }
}

@github-actions
Copy link

github-actions bot commented Sep 5, 2020

Ember Asset Size action

As of 3147d5c

Files that got Bigger 🚨:

File raw gzip
nomad-ui.js +28 B +5 B

Files that stayed the same size 🤷‍:

File raw gzip
vendor.js 0 B 0 B
nomad-ui.css 0 B 0 B
vendor.css 0 B 0 B

@github-actions
Copy link

github-actions bot commented Sep 5, 2020

Ember Test Audit comparison

master 3147d5c change
passes 1385 1385 0
failures 0 0 0
flaky 0 1 +1
duration 7m 33s 611ms 6m 57s 909ms -35s 702ms

Copy link
Contributor

@backspace backspace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow, nice sleuthing!! 😯

@DingoEatingFuzz DingoEatingFuzz merged commit 3020111 into master Sep 8, 2020
@DingoEatingFuzz DingoEatingFuzz deleted the b-ui/wrong-task-logs branch September 8, 2020 21:05
@notnoop notnoop added this to the 0.12.5 milestone Sep 14, 2020
notnoop pushed a commit that referenced this pull request Sep 16, 2020
UI: Fix bug where task logs would suddenly switch to another task's logs
backspace added a commit that referenced this pull request Sep 17, 2020
I believe this test became flaky after #8833, you can see
an example failure here:
https://app.circleci.com/pipelines/github/hashicorp/nomad/11809/workflows/0dc3c8f4-187c-44af-8bdb-6f010d653081/jobs/100848

My understanding of the goal here is not to assert that the
first task group in the server database matches the first
row displayed, but that before #8833 it could be assumed
that they did match. This changes to find the row corresponding
to the first server task group instead of assuming it’ll
be first.
backspace added a commit that referenced this pull request Sep 17, 2020
I believe this test became flaky after #8833, you can see
an example failure here:
https://app.circleci.com/pipelines/github/hashicorp/nomad/11809/workflows/0dc3c8f4-187c-44af-8bdb-6f010d653081/jobs/100848

The goal here is not to assert that the first task group in the
server database matches the first row displayed, but before
#8833 it could be assumed that they did match. This
changes to find the row corresponding to the first server
task group instead of assuming it’ll be first.
fredrikhgrelland pushed a commit to fredrikhgrelland/nomad that referenced this pull request Sep 28, 2020
I believe this test became flaky after hashicorp#8833, you can see
an example failure here:
https://app.circleci.com/pipelines/github/hashicorp/nomad/11809/workflows/0dc3c8f4-187c-44af-8bdb-6f010d653081/jobs/100848

The goal here is not to assert that the first task group in the
server database matches the first row displayed, but before
hashicorp#8833 it could be assumed that they did match. This
changes to find the row corresponding to the first server
task group instead of assuming it’ll be first.
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ui] Logs from other tasks in same group interleaved in logs view
3 participants