Refactor Report for better per-entity metadata #123

peterbourgon · 2015-05-27T13:43:25Z

Addresses #120.

Refactor Report and Topology structures
Make various ID generation consistent and understandable
Endpoint topology (was Process)
Address topology (was Network)
Process topology
Host topology (was OriginHost struct)

Follow-up tickets #114, #171, #172

peterbourgon · 2015-06-03T15:14:21Z

@tomwilkie it's very possible I've forgotten some stuff in this PR, because I'm doing about six simultaneous things right now :) but I think this is good to go. Please give it a read, I suggest starting package report. And also run it as a test, to make sure I didn't overlook any behaviors.

- Report is composed exclusively of topologies - Certain topologies contain only node metadata - Refactor package report for better comprehensibility and test coverage Closes #120

probe/tag/docker_tagger.go

@@ -0,0 +1,172 @@
+package tag


tomwilkie · 2015-06-03T18:45:11Z

I've had a ~quick look (still took a few hours). This change is un-reviewable. At 7.6k lines, it would take me weeks to go through this code throughly. It has to be broken down into smaller changes; I'm not going to rubber stamp such a big change.

Given that you're busy, I'll have a stab a teasing out smaller changes we can upstream quickly. But I would fully expect this process to take the next week or so.

peterbourgon · 2015-06-03T21:28:05Z

I can't really break it down. It's not a diff, it's a complete refactoring. Look at it on a per-package basis, not as a change but as a complete entity. It should be comparatively easy to reason through vs. the previous state.

edit: But! Thank you, feedback received. I'll take things like this more granularly in the future, even if it means doing a multi-stage refactor.

Another baby step towards #123, this change follows from #192 and merges the two concepts of Origin in a renderable node. We also cut out a layer of abstraction, and add an OriginTable method to Report, which directly generates a table of info for the detail pane given any origin node ID.

Another baby step towards #123, this change follows from #192 and merges the two concepts of Origin in a renderable node. We also cut out a layer of abstraction, and add an OriginTable method to Report, which directly generates a table of info for the detail pane given any origin node ID. Other changes from feedback: - Assume origin IDs are unique and don't reflect.Dedupe - Improve origin ID lookup - Move OriginTable to detailed_node.go, as a free function - rm app/detail_pane.go (empty)

peterbourgon · 2015-06-16T14:17:47Z

We're past the point of this being salvageable. Closing it out.

2bbc9a08 Merge pull request #140 from weaveworks/sched-http-retry c3726dea Add retries to sched util http calls 2cc7b5ac Merge pull request #139 from meghalidhoble/master fd9b0a72 Change : Modified the lint tools to skip the shfmt check if not installed. Why the change : For ppc64le the specific version of shfmt is not available, hence skipped completely the installation of shfmt tool. Thus this change made. bc645c70 Merge pull request #138 from dholbach/add-license-file a642e022 license: add Apache 2.0 license text 9bf59568 Merge pull request #109 from hallum/master d971d821 Merge pull request #134 from weaveworks/2018-07-03-gcloud-regepx 32e7aa2d Merge pull request #137 from weaveworks/gcp-fw-allow-kube-apiserver bbb6735e Allow CI to access k8s API server on GCP instances 764d46ca Merge pull request #135 from weaveworks/2018-07-04-docker-ansible-playbook ecc2a4e3 Merge pull request #136 from weaveworks/2018-07-05-gcp-private-ips 209b7fb6 tools: Add private_ips to the terraform output 369a655f tools: Add an ansible playbook that just installs docker a643e270 tools: Use --filter instead of --regexp with gcloud b8eca887 Merge pull request #128 from weaveworks/actually-say-whats-wrong 379ce2bb Merge pull request #133 from weaveworks/fix-decrypt 3b906b54 Fix incompatibility with recent versions of OpenSSL f091ab43 Merge pull request #132 from weaveworks/add-opencontainers-labels-to-dockerfiles 248def1b Inject git revision in Dockerfiles 64f2c280 Add org.opencontainers.image.* labels to Dockerfiles ea96d8ed add information about how to get help (#129) f066ccdd Make yapf diff failure look like an error 34d81d70 Merge pull request #127 from weaveworks/golang-1.10.0-stretch 89a0b4f8 Use golang:1.10.0-stretch image. ca69607f Merge pull request #126 from weaveworks/disable-apt-daily-test f5dc5d54 Create "setup-apt" role 7fab4413 Rename bazel to bazel-rules (#125) ccc83168 Revert "Gocyclo should return error code if issues detected" (#124) 1fe184f1 Bazel rules for building gogo protobufs (#123) b917bb89 Merge pull request #122 from weaveworks/fix-scope-gc c029ce01 Add regex to match scope VMs 0d4824b1 Merge pull request #121 from weaveworks/provisioning-readme-terraform 5a82d64c Move terraform instructions to tf section d285d78d Merge pull request #120 from weaveworks/gocyclo-return-value 76b94a47 Do not spawn subshell when reading cyclo output 93b3c0d5 Use golang:1.9.2-stretch image d40728f6 Gocyclo should return error code if issues detected c4ac1c3f Merge pull request #114 from weaveworks/tune-spell-check 89806560 Only check files 12ebc73a Don't spell-check pki files 578904ab Special-case spell-check the same way we do code checks e772ed59 Special-case on mime type and extension using just patterns ae82b50c Merge pull request #117 from weaveworks/test-verbose 89434738 Propagate verbose flag to 'go test'. 7c79b43c Merge pull request #113 from weaveworks/update-shfmt-instructions 258ef015 Merge pull request #115 from weaveworks/extra-linting e690202b Use tools in built image to lint itself 126eb561 Add shellcheck to bring linting in line with scope 63ad68f0 Don't run lint on files under .git 51d908a3 Update shfmt instructions e91cb0d3 Merge pull request #112 from weaveworks/add-python-lint-tools 0c87554d Add yapf and flake8 to golang build image 35679ee5 Merge pull request #110 from weaveworks/parallel-push-errors 3ae41b6f Remove unneeded if block 51ff31a5 Exit on first error 0faad9f7 Check for errors when pushing images in parallel d87cd026 Add arg flag override for destination socks host:port in pacfile. git-subtree-dir: tools git-subtree-split: 2bbc9a08a7f672eae62cfe110d7f536c6cc53ce3

peterbourgon force-pushed the new-report branch from 9709eb1 to 03fca99 Compare June 1, 2015 23:26

peterbourgon mentioned this pull request Jun 2, 2015

Don't show bytes ingress/egress until we actually have some info to show #155

Closed

tomwilkie mentioned this pull request Jun 3, 2015

Create Weave topology with info from the Weave router (or equivalent in another topology) #100

Closed

peterbourgon force-pushed the new-report branch from 71523fb to a33cccd Compare June 3, 2015 15:09

This was referenced Jun 3, 2015

Add Container topology #171

Closed

Restore per-source pseudonode behavior #172

Closed

Pull data from docker stats #114

Closed

peterbourgon changed the title ~~[WIP] Refactor Report for better per-entity metadata~~ Refactor Report for better per-entity metadata Jun 3, 2015

peterbourgon assigned tomwilkie Jun 3, 2015

Refactor Report and Topology structures

f72abb8

- Report is composed exclusively of topologies - Certain topologies contain only node metadata - Refactor package report for better comprehensibility and test coverage Closes #120

peterbourgon force-pushed the new-report branch from a33cccd to f72abb8 Compare June 3, 2015 15:15

peterbourgon mentioned this pull request Jun 3, 2015

Refactor out the docker process mapper for use in tracer. #174

Merged

tomwilkie reviewed Jun 3, 2015
View reviewed changes

probe/tag/docker_tagger.go

@@ -0,0 +1,172 @@

package tag

This comment was marked as abuse.

Sign in to view

This comment was marked as abuse.

Sign in to view

This was referenced Jun 4, 2015

Rename Resolver to staticResolver #175

Merged

Add interface to xfer/collector.go, and some more tests #176

Merged

Fix docker_[container_]{name,id} mismatches

1794830

This was referenced Jun 4, 2015

Refactor process mappers into taggers. #178

Merged

Host topology #192

Merged

peterbourgon mentioned this pull request Jun 9, 2015

OriginHosts and OriginNodes become Origins #194

Merged

tomwilkie removed their assignment Jun 15, 2015

peterbourgon closed this Jun 16, 2015

tomwilkie deleted the new-report branch July 13, 2015 16:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor Report for better per-entity metadata #123

Refactor Report for better per-entity metadata #123

peterbourgon commented May 27, 2015

peterbourgon commented Jun 3, 2015

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Jun 3, 2015

peterbourgon commented Jun 3, 2015

peterbourgon commented Jun 16, 2015

Refactor Report for better per-entity metadata #123

Refactor Report for better per-entity metadata #123

Conversation

peterbourgon commented May 27, 2015

peterbourgon commented Jun 3, 2015

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Jun 3, 2015

peterbourgon commented Jun 3, 2015

peterbourgon commented Jun 16, 2015