Looking for good examples of decompiler PRs

[thixotropist]

What are some good examples of community pull requests extending the C++ decompiler code? I'd like to start some experimental extensions of userop.* and constseq.*. The current ghidra decompiler has code to transform low level pcode loads and stores into builtin_memcpy and builtin_strncpy CALL_OTHER nodes. Adding rules to recognize sequences of user pcode as builtin_memcpy and builtin_strncpy looks attractive.

The decompiler code looks to be very nicely documented but without a lot of integration tests or contributor guidelines on what we should do before expecting someone like @caheckman to review and comment.

This work would be a proof of concept PR aimed more at probing complexity/utility tradeoffs than a quick merge to master.

[ryanmkurtz]

https://github.com/NationalSecurityAgency/ghidra/pulls?q=is%3Apr+label%3A%22Feature%3A+Decompiler%22+is%3Amerged

[thixotropist]

Thanks. That's a good example of a decompiler bug fix PR. Are there any good examples of a reviewed feature extension PR?

[msm-code]

@thixotropist I believe the link directs to a list of all merged PRs (labeled decompiler). Most of them are actually fixes, but not all of them - one recent example is #6578.

Though I can't find many more merged decompiler features. While there are dozens examples of unmerged ones, like PRs by @kkots (link) and @LukeSerne (link).

There's nothing wrong with avoiding community contributions in critical components of the project (which decompiler is without doubt). But if that's the case, working on decompiler PRs may not be a good investment of your time, so I think your question is a very valuable one.

[thixotropist]

Thanks to all for the comments - these are very useful. My core issue here is whether the US government is providing the Ghidra core development team enough resources and guidance to evolve components like the decompiler as fast as processors and compilers are evolving. I suspect not, and that the broader Ghidra community might be able to help that evolution without too much fragmentation. For example, if I want to explore user pcode transformations, then a good place for me to start might be simple PRs to extend unit tests and data tests for coverage of existing user pcode and transformation decompiler features.

[gemesa]

It depends on how you define a "good investment." One can always custom-build their own version of Ghidra and include open PRs as patches. I have some unmerged PRs as well but I dont consider them a waste of time. I implemented some fixes and features to suit my needs and then upstreamed them. If the Ghidra maintainers decide they are not worth merging the PRs are still available and anyone can use them in custom builds.

[kkots]

I didn't really have a choice to not do the PRs I did. Ghidra was just not decompiling the stuff properly and to get any kind of sensible output I had to make the changes I made in PRs and will probably have to make more as some stuff is still completely garbled. Sure I could sit with pen and paper all day trying to decipher broken decompiler output in each case but I'm not that type of person. I like when it does it automatically. So I don't really care if the Ghidra team merges the PRs or not, they're byproduct of my personal use.

[msm-code]

Thanks both,

It depends on how you define a "good investment."

Maybe the word I used was too strong. I recently spend my free time learning Ghidra internals for fun, and to solve the problems I encounter when scripting. I don't consider that a "waste of time". I think I was surprised how few decompiler PRs were merged (also it was 4AM for me, and I didn't think too much before posting). On the other hand, I plan to eventually contribute something upstream, and if PRs improving decompiler output have a very low chance of being merged, I'll probably try to PR other things instead.

One can always custom-build their own version of Ghidra and include open PRs as patches.

That's a great idea! Now I want to try building my own "super-decompiler" with pending PRs applied, for my personal use. If the result turns out to be unstable, at least I'll be able to give feedback in the PRs.

I didn't really have a choice to not do the PRs I did.

I see where you are coming from, but you always have a choice of not doing a PR - just don't push your changes. I know many people who fix issues for themselves but don't spend time trying to upstream them, so give yourself some credit :).

So I don't really care if the Ghidra team merges the PRs or not, they're byproduct of my personal use.

If that's really true then good for you (and even bigger respect for upstreaming them and being active in review). For me it makes a difference - there are some pain points where I can't justify fixing them just for myself (or I already have clunky workaround), but fixing them "for everyone" sounds more valuable.