-
Notifications
You must be signed in to change notification settings - Fork 15.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"Official" Wireshark dissector #3303
Comments
In Google, to send protobuf over the wire, you will be using the rpc library (the internal version of http://grpc.io/) instead of writing your own socket code, and a set of tools/libraries are provided to inspect and diagnose the rpc traffic. For example, with a command line tool you can easily write up a proto message in text format and send it to an arbitrary rpc server and get the response in text format. Hardly anyone needs to inspect the network traffic to understand the proto data being sent and it's also very hard to do because the rpc library will compress and encrypt the proto data. People who work on the rpc library probably need to use wireshark or tcpdump to inspect the network traffic, but again they don't need to inspect the proto data (I know they send payload of 1 byte in their tests). So pretty much nobody in Google will need this protobuf dessector for Wireshark and I doubt it will ever be officially supported... |
Actually after reading the OP I tried using the Length Prefixed Protocol Buffer Dissector together with the protobuf_dissector (forked). And it was pretty easy to setup and get it to work in my context which is; Wireshark on Windows, TCP connection, length prefix (little endian, for this I made a small change on the lppb), proto2 message format. |
@ruudsieb Did you get it to work with the proto3 language? |
Sorry, forgot to mention that, no, I'm using proto2 message format. |
@FlorianWolters I am also trying to write a wireshark plugin in c to parse google protocol buffers over TCP, but not finding proper source to know how. Have you found any solution for it? |
@jfjffilk No, as mentioned in the OP there is currently no out-of-the-box solution working for the proto3 language. For general information how-to implement a Wireshark dissector refer to Chapter 9. Packet dissection of the official Wireshark documentation. But I strongly recommend to contact the author(s) of the Wireshark Protobuf Dissector and to analyse/fork/extend that project before beginning from scratch and implementing your own dissector. Please read the OP again for problems related to the existing Protobuf dissectors. |
I would also be interested in an official Wireshark dissector for protobuf and perhaps gRPC. I agree that it may not be of much use in Google's environment, however I think that many developers and testers would benefit from it. |
Sorry for double-posting, but I tried a few things after posting above. I found out, that in the official Wireshark repo on GitHub, there is a gRPC dissector - namely the wireshark/epan/dissectors/packet-grpc.c located here: https://github.com/wireshark/wireshark/blob/master/epan/dissectors/packet-grpc.c It seems that it was officially added in the end of September this year: Note that the original Wireshark repo is located on their website. Please check https://www.wireshark.org/develop.html. I opted to see how this dissector is working and cloned the Wireshark repo:
My system is a 64-bit Arch Linux, so I built Wireshark with the following commands:
(using this as a reference: https://www.wireshark.org/docs/wsdg_html_chunked/ChSrcBuildFirstTime.html) It took around 20-30 minutes to build. The Wireshark version is: I then ran it as I cloned, built and started the official gRPC The console outputs are, as expected: The captured packets in Wireshark are: You can download the .pcapng file from my repo: Here you can see that they are decoded as TCP packets. I clicked with the right mouse button on a packet and selected And configured the packets as HTTP2: And the packets are now correctly recognized as either HTTP2 or gRPC: Let's take a look at the packet which carries the actual gRPC I had also opened the The actual value of field 1 from the The gRPC The Wireshark interpretation of fields and their values could definitely be extended. I did not come across any option to specify and input Nevertheless, I think this is a significant improvement in the right direction. Please share your opinion on this dissector. Has anyone else tried it and published some results/improvements? |
@ndandanov Great findings! Could you also post this to grpc github repo? I think lots of grpc users will find this useful. |
Hi @xfxyjwf, thank you for the kind words! I also recorded two videos which show the process in depth. They are somewhat lengthy and unfortunately I could not replicate the same successful packet dissection - probably due to the slightly newer development version of Wireshark. Nevertheless, they could be of help to somebody: |
@ndandanov This is really great! Do you happen to have any packet captures (PCAP files)? |
@ndandanov Found it in your previous post :) https://github.com/ndandanov/wireshark-grpc/blob/master/grpc-hello-world.pcapng |
Thank you for all the posts here. Posting them at an issue doesn't seem like the best place though. I'd recommend creating a page somewhere, and add a link to that page at https://github.com/protocolbuffers/protobuf/blob/master/docs/third_party.md. You would probably want to reach out to gRPC and figure out where to best post it there too. Thanks! |
Hope following information might be useful: New features about Protobuf and gRPC dissectors have been added into Wireshark since version 3.2.0:
Another two new features will be released in 3.3.0 or 3.4.0:
Preferences userguide: |
@huangqiangxiong would you be interested in writing a guest blogpost for https://grpc.io/blog/ about how to use wireshark with grpc and what are some of the new features? |
OK. I like to do that. But I may spend a little time to study how to get the privilege to post article on that website.
|
Btw a blogpost is created by opening a PR (e.g. grpc/grpc.io#121). |
Add this blogpost as recommended on protocolbuffers/protobuf#3303
Add this blogpost as recommended on protocolbuffers/protobuf#3303
Add this blogpost as recommended on protocolbuffers/protobuf#3303
Add this blogpost as recommended on protocolbuffers/protobuf#3303
…#467) * Blogpost about how to use wireshark with protobuf and grpc dissectors Add this blogpost as recommended on protocolbuffers/protobuf#3303 * Update content according to last comments 1. change title to "How to analyze gRPC with Wireshark" 2. add link to github id (after the name attribute). 3. add "Protocol Buffers" for "Protobuf" in first sentence. 4. add summary info about the intent and scope of this blog. 5. move the history to a nearly end section. 6. add a short section about how to capture gRPC traffic. 7. change "official *.proto library of Protobuf" to "Protocol Buffers Well-Known Types" 8. remove the section about JSON-serialized capture. 9. remove the sections about build-in UDP and custom dissectors, instead leaving a link to wireshark protobuf wiki page. 10. remove sample link of Protobuf only capture from referrence section. * Update how-to-analyze-grpc-with-wireshark.md Updated according to comments. 1. Update date. 2. Fixed some link errors. * PR #467 copyedits, spelling and text wrap 2020-11-27 * PR #467 correct clerical and grammar errors 2020-11-28 also update date * Mark as draft Co-authored-by: Patrice Chalin <chalin@users.noreply.github.com>
Note: I know this is not the best place to post this kind of request, but I do not know where to post this elsewhere. Hopefully other people are also interested in this.
I do wonder what tools the folks at Google use internally to debug and analyze the Protobuf messages transferred over the wire (if using network communication).
The defacto standard to do network analyzes is the network tool Wireshark. The functionality of Wireshark can be extended with a Protocol Dissector (i.e. a plug-in) to support a (custom) protocol.
I've researched a bit and found the following dissectors for Protobuf:
Dissector written in Lua. Does support
proto2
and UDP (without any prefixing) only. Pretty clumsy usage and buggy visualization.Seems to add length prefix support to the Lua dissector in order to support TCP. Haven't tested it.
Undocumented C dissector. Haven't tested it.
The first C dissector around. Project seems dead. Haven't tested it.
See above.
There are plenty of requests for a working and user-friendly dissector on the WWW, e.g.:
I would be very interested in some kind of official Wireshark protocol dissector with the following features:
proto3
andproto2
.That would allow to support TCP as well as custom protocols that define additional fields.
.proto
files).[numeric field id:value]
pairs. This does not require the.proto
file(s).[field name:value]
pairs. This requires the.proto
file(s) for the specific protocol. The addition of.proto
file(s) should be possible via the GUI of Wireshark and the directory path to the official Protobuf.proto
files should be configurable as well (e.g.$PROTOBUF_ROOT/include/google/protobuf
).At the end of the day it would be very nice to have some kind of official protocol dissector contributed to Wireshark.
Are there any efforts towards an official Protobuf dissector for Wireshark? What does the community think? Is this considered useful? Or did I overlook an already existing solution that satisfies the requirements stated above?
The text was updated successfully, but these errors were encountered: