protobuf support

[ngrigoriev]

https://docs.pact.io/faq#what-is-pact-good-for does mentions Protobuf and talks about the questionable need for contract testing when a firm contract exists (protobuf definition) and the compatibility is ensured by the framework. I am wondering if there are any plans for Pact. Personally I believe that the nature of Protobuf (read: gRPC) solves only part of the problem - the backward/forward compatibility of the parser. I believe this is not enough. Consider this: the API goes through incremental changes and the implementations (on both side) may end up being wrong with their assumptions - while the contract is still followed! In Protobuf v3 (again, talking about gRPC) everything is optional. So, introducing a new field unknown to either party is transparent, yes, they won't touch what they do not know. However, fields will get obsolete and the consumer may decide no longer use field A because there is a field A1 that seems to be more appropriate. And the server internally assumes that both values A and A1 are mandatory. Or the server may decide that if the client sends A1, then it would be reading the response field B1 and won't look at B. In other words, protobuf "schema" is not strong and expressive enough to carry the contract enforcement deep enough (which is good, in fact) - so I do belive this scenario can and should be covered by contract tests.

I am not aware of any existing contract testing framework that supports gPRC. I am thinking about using Pact but I do not see a way other than introducing transparent transcoding between gRPC/HTTP2 and JSON/HTTP so Pact would operate in a traditional environment.

[mefellows]

Thanks @ngrigoriev. In short, the core Pact team just hasn't had the opportunity yet to fully appreciate the situation and how Pact might work with Protobufs. Almost all of the bits of pact have come from real-world use cases, where one of the core contributors has had to solve a genuinely difficult problem that couldn't be solved in a better way. This might be one of those.

You might want to join our slack group (slack.pact.io) and jump into the #protobufs channel. We are in the process of working with a contributor who is keen to see this in Pact - and we are excited for the proposition.

[mcon]

Hi @ngrigoriev,

I'm looking into adding protobuf over HTTP support for Pact at the moment - I completely agree with you in that simply using protobuf / grpc does not give the same level of confidence in the integration of an application as a contract test.

I was wondering whether you'd had any thoughts as to how to go about solving the problem of facilitating grpc Pacts?

There are a few things about grpc which makes this problem difficult/interesting, but I think the primary difficulty revolves around the fact that grpc supports synchronous requests, asynchronous requests and also streaming. As far as async requests go, it should be possible to support these using the "Message" pacts which are bring rolled out, however when it comes to the other two types of interactions, the difficulty is as follows:

• grpc server/client code is generated by protoc, and there's no way to define a 'dynamic' service, which could be given instruction as to how to behave as a mock at runtime.

Given the difficulties above, I think in order for this support to be implemented you'd have to go down one of the following routes:
either

• Write some code which has knowledge of how the grpc protocol operates, in order that you can have a server that can accept instructions from test code as to how it should act as a mock. This information would then have to be encoded into the pact contract format (the former of these tasks probably being the hardest).
  alternatively
• Have an abstraction layer between your tests and the current Pact core which, at runtime, invokes protoc in order to generate the code required to run a particular service, which would then allow it to act as a mock (obviously this would require a dynamic language, and is also pretty gross, though would be the quickest to get working probably).

I'm not an expert on grpc, so I could be missing something here - as @mefellows suggests, do get in touch over Slack if you have any thoughts you'd like input on (or clarification of the above).

[PaithanQ]

Can I ask what's the status of this? I would be really interested in such a feature for PACT

[mcon]

Hi @PaithanQ, sorry for the late response, have just returned from vacation.

I've not had as much time to work on this as I thought I would have (changing priorities etc), and so not much is happening on this from my end.

There's an almost-working-example of testing protobuf over HTTP in my fork of the pact-net repo, but that's not quite grpc.

If someone is interested in picking up were I left off, I'd be happy to do some work to tidy things up, and put things in a state where I could hand it over - otherwise, I'm doubtful this will make it to the top of my list any time soon.
https://github.com/mcon/pact-net/blob/protobuf-serialization/TODO.md
https://github.com/mcon/pact-serialization-proxy/blob/master/README.md

[Nadrendion]

Vi,
I just wanted to announce my interest in this feature. I am more or less working with proto and grpc every day and I can see the benefits of having a tool such as this for a deeper contract testing.

I will try to join you on slack tomorrow at work if I am allowed by work policy to install slack on my computer.

Br. Johnny

[PaithanQ]

Hey,

Still super interested on this, please let me know how it's going!

[Piste]

We're also very interested!

[martinschneider]

FYI, Spring Cloud Contract supports contract testing for protobuf. It seems like they're simply hard-coding binary requests and responses though:

request {
	method 'POST'
	url '/check'
	body(fileAsBytes("PersonToCheck_old_enough.bin"))
	headers {
		contentType("application/x-protobuf")
	}
}
response {
	status 200
	body(fileAsBytes("Response_old_enough.bin"))
	headers {
		contentType("application/x-protobuf")
	}
}

[martinschneider]

There's also some progress on the Slack channel :-)

https://gist.github.com/mcon/2e3ae3eff0b9712d02e06ac1275b7f65

[mefellows]

Hmm interesting on the SCC implementation. In that regard then we probably also support protobufs. Although presumably it's only supported over HTTP not TCP

[noel-yap]

There are a few things about grpc which makes this problem difficult/interesting, but I think the primary difficulty revolves around the fact that grpc supports synchronous requests, asynchronous requests and also streaming. As far as async requests go, it should be possible to support these using the "Message" pacts which are bring rolled out, however when it comes to the other two types of interactions, the difficulty is as follows:

• grpc server/client code is generated by protoc, and there's no way to define a 'dynamic' service, which could be given instruction as to how to behave as a mock at runtime.

One can create a protoc add-on which would generate the code that defines a 'dynamic' service which could be given instructions as to how to behave as a mock at runtime.

Having said this, I'm still learning about Pact myself and am still unsure exactly what you mean by "a 'dynamic' service which could be given instructions". In what way would that be different from a simple mock? Mocking gRPC services is quite straightforward OOTB.

[mefellows]

Yes, you can mock gRPC/protobuf (Golang and JS seem to have pretty darn good support for this now). Where it gets challenging is that we don't want to have to implement all of the transports, protocols, encoding etc. for every language we support. At the least, we want to make that as simple as possible. We have to be able to do this in a way that ensures consistent functionality, across platform etc.

The other thing that makes it messy is that many languages require a compilation step to convert .proto into files, and then dynamically attaching those files to a gRPC endpoint. We've been using Rust behind the scenes for our engine, and it's not really a fan of this level of dynamism ;)

Also, we don't want to have to rely on external tools to be available at test time (although you would expect in most cases the protoc binary to exist - or we could package it up).

None of it is existential, it just adds a level of complexity when you wrap it into a framework.

[ivangsa]

Hi,

we are successfully implementing contract testing with pact and gRPC with java and spring-boot

https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c

it's true "there's no way to define a 'dynamic' service, which could be given instruction as to how to behave as a mock at runtime"

but you can collect all information at runtime to do the mocking

• on consumer side tests, you already need to write a grpc Channel that connects to pact mocking server: inside that channel you will receive all information you need to do the un/marshalling
• on provider verification tests, you don't have the information on a "per request" basis but you can collect all MethodDescriptors at service startup and store them, mapping them to urls (url <-> method.fullname)

with spring-boot we collect that information on a start up event, but even if in your platform you don't have that kind of event, you can still register grpc services "by hand"

[YOU54F]

Plugins, Protobufs and gRPC (oh my!)
Back in September 2021 Matt introduced us to The case for contract testing Protobufs, gRPC and Avro.

We are pleased to announced initial support for testing gRPC interactions via plugins has been added to Pact-JVM (for Junit5) and Pact-Rust including the shared core, enabling distribution to other client libraries.

We have released an official Pactflow Protobuf / gRPC plugin for Pact.

Join the Developer Preview Program for updates, or chat to us in #protobufs.

[YOU54F]

Read more about the pact plugin framework on the docs site

https://docs.pact.io/plugins/quick_start

Closing this as complete now :)