Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Try to get the pthreads idle example running under docker #401

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

jlfwong
Copy link
Owner

@jlfwong jlfwong commented Jul 31, 2022

Working towards fixing #393

@jlfwong
Copy link
Owner Author

jlfwong commented Jul 31, 2022

I'm trying to build the perf output for the test file provided in #393.

To do so, in the /sample/programs/cpp directory, I run make linux-perf, which spins up a docker container, installs linux perf tools, and then executes a script which runs perf on the given program.

Here's what the output looks like at the moment:

docker container prune -f
Deleted Containers:
8da1b9e8d59c057cc5697e10a54a1c1a272becdb731f357c7a2b6b91557c4633

Total reclaimed space: 125.8kB
docker image rm speedscope-ubuntu || true
Untagged: speedscope-ubuntu:latest
Deleted: sha256:75ccfbd75d99caa7d5f5074926fd252c1be59938e9ee3bc53b7219d31dcc5e9c
docker build -t speedscope-ubuntu .
[+] Building 0.6s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.0s
 => => transferring dockerfile: 347B                                                                                                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/ubuntu:bionic                                                                                                                                                                                                           0.5s
 => [internal] load build context                                                                                                                                                                                                                                          0.0s
 => => transferring context: 5.09kB                                                                                                                                                                                                                                        0.0s
 => [1/7] FROM docker.io/library/ubuntu:bionic@sha256:478caf1bec1afd54a58435ec681c8755883b7eb843a8630091890130b15a79af                                                                                                                                                     0.0s
 => CACHED [2/7] RUN apt-get update                                                                                                                                                                                                                                        0.0s
 => CACHED [3/7] RUN apt-get install -y build-essential linux-tools-common linux-tools-4.15.0-30-generic libpthread-stubs0-dev                                                                                                                                             0.0s
 => CACHED [4/7] RUN ln -sf /usr/lib/linux-tools/4.15.0-30-generic/perf /usr/bin/perf                                                                                                                                                                                      0.0s
 => CACHED [5/7] WORKDIR /workdir                                                                                                                                                                                                                                          0.0s
 => CACHED [6/7] ADD . /workdir                                                                                                                                                                                                                                            0.0s
 => CACHED [7/7] RUN make clean simple simple-terminates forks idle                                                                                                                                                                                                        0.0s
 => exporting to image                                                                                                                                                                                                                                                     0.0s
 => => exporting layers                                                                                                                                                                                                                                                    0.0s
 => => writing image sha256:75ccfbd75d99caa7d5f5074926fd252c1be59938e9ee3bc53b7219d31dcc5e9c                                                                                                                                                                               0.0s
 => => naming to docker.io/library/speedscope-ubuntu                                                                                                                                                                                                                       0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
docker run --privileged speedscope-ubuntu /bin/bash perf-script.sh idle > ../../profiles/linux-perf/idle.linux-perf.txt
+ [[ idle == \s\y\s\t\e\m\-\w\i\d\e ]]
+ [[ idle == \f\o\r\k\s ]]
+ [[ idle == \i\d\l\e ]]
+ perf record -a -F 999 -g ./idle
Couldn't record kernel reference relocation symbol
Symbol resolution may be skewed if relocation was used (e.g. kexec).
Check /proc/kallsyms permission or run as root.
[ perf record: Woken up 1 times to write data ]
0xe8 [0x6c73]: failed to process type: 1768383842
[ perf record: Captured and wrote 0.116 MB perf.data ]
+ echo 'Done recording'
+ ls -alh perf.data
+ perf script -i perf.data
0xe8 [0x6c73]: failed to process type: 1768383842
make: *** [linux-perf] Error 234

The problem line seems to be the following:

0xe8 [0x6c73]: failed to process type: 1768383842

I have no idea what's causing this, and I'm not sure how to diagnose further.

@jlfwong jlfwong mentioned this pull request Jul 31, 2022
@coveralls
Copy link

coveralls commented Jul 31, 2022

Coverage Status

Coverage remained the same at 41.652% when pulling e0ceccb on jlfwong/perf-idle-fix into 6493c5f on main.

@javierhonduco
Copy link

javierhonduco commented Aug 3, 2022

Hi @jlfwong! Found this issue by chance while checking out speed scope and I think this might be happening due to the way the perf.data file is being created. As the perf record output is saved to perf.data by default, we aren't only storing the actual binary data produced by perf, but the intermixed test program's output as well, corrupting the binary file! 🙅

You can see this by running:

$ docker run -it --privileged speedscope-ubuntu /bin/bash
root@448b544d0dfd:/workdir# # And now, inside of the container
root@448b544d0dfd:/workdir# perf record -a -F 999 -g ./idle > perf.data
root@448b544d0dfd:/workdir# strings perf.data | grep "beg"
sleep begin
sleep begin
sleep begin
sleep begin

which is the output of ./idle :)

Added a possible way of sorting this out inline

@@ -14,6 +14,14 @@ if [[ "$1" == "forks" ]]; then
exit 0
fi

if [[ "$1" == "idle" ]]; then
perf record -a -F 999 -g ./idle > perf.data

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
perf record -a -F 999 -g ./idle > perf.data
perf record -a -F 999 -g --output=perf.data ./idle

The same applies below and above. These commands aren't printing anything yet, but perhaps it would be worth adding this flag instead as if in the future they print something, perf script and related tooling will break and it will be a bit confusing 😄

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I needed to do something very slightly fancier here, because the stdout of perf-script.sh is used directly to generate the output files in the Makefile, but this --output switch was still a helpful key

@jlfwong
Copy link
Owner Author

jlfwong commented Aug 4, 2022

@javierhonduco Ahhh thanks for the catch! I now have it generating a valid profile. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants