azurerm_data_factory_pipeline: Support for activities

[lawrencegripper]

This allows you to define the activities in the data factory pipeline. Would be great to get some thoughts on this approach and feedback so I can polish up and move out of draft. Currently also has the changes from #6195 which I'll rip out once ready for merge.

There is some complexity to this due to the complex setup of objects used to define activities. To work around this I've kept the activity definition as a JSON block.

REST: https://docs.microsoft.com/en-us/rest/api/datafactory/pipelines/createorupdate
SDK Issue: Azure/azure-sdk-for-go#7934 (explains some of the weird serialization stuff I do)

Integration test:

[lawrencegripper]

I've updated to remove the devcontainer commit from #6195

[lawrencegripper]

If we think this is useful to include let me know and I'll fixup the formatting and update the docs to match the change.

[mbfrahry]

Hey @lawrencegripper. Thanks for getting this feature written up. It's one I've been wrestling with for awhile but struggled with because of how nested some of these activities could be. I think you've nailed the implementation here though.

To get it over the finish line, let's increase the complexity with some more tests by adding/removing/updating activities and then also updating the docs like you mentioned. I think we'll want an example on the docs page of how to use this since writing out the json could be a tad tricky. It should look similar to this where we have our basic example and then another example underneath to show how activities could look.

[lawrencegripper]

@mbfrahry I'm off on leave for a bit as we just had a baby. I can look to pick this up again when back in around 4 weeks or happy for you to finish up while I'm away.

[mbfrahry]

Heyo! Congrats @lawrencegripper! I'm happy to take a look when things clear up off my plate. Hope you enjoy your time with your baby

[mbfrahry]

LGTM!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!