Source Jira: Add pull requests stream #7378
Conversation
|
Ping @marcosmarxm |
| class PullRequests(IncrementalJiraStream): | ||
| """ | ||
| Undocumented internal API used by Jira webapp | ||
| """ | ||
|
|
||
| cursor_field = "updated" | ||
| parse_response_root = "detail" | ||
| # raise_on_http_errors = False | ||
|
|
||
| pr_regex = r"(?P<prDetails>PullRequestOverallDetails{openCount=(?P<open>[0-9]+), mergedCount=(?P<merged>[0-9]+), declinedCount=(?P<declined>[0-9]+)})|(?P<pr>pullrequest={dataType=pullrequest, state=(?P<state>[a-zA-Z]+), stateCount=(?P<count>[0-9]+)})" |
There was a problem hiding this comment.
I don't feel comfortable adding this endpoint. If this starts failing we cann't solve because its not possible to know where to find information to solve, backoff or error handling. @cjwooo can you explain your use case to use this endpoint? Maybe let to add this after Jira allow in official API.
There was a problem hiding this comment.
We use this to associate closed Jira issues with the Github pull requests that resolved the issue.
There was a problem hiding this comment.
Would it help if I mark this stream as an experimental one in the comments?
There was a problem hiding this comment.
@sherifnada can you give your opinion here?
|
@cjwooo thanks for opening this PR! I took a look around online and it while this endpoints seems(?) to maybe have held up via Hyrum's law, it's difficult to support adding it to the connector because it's undocumented; if it changes or breaks or gets removed at any point, it'll cause user disruption. We could always say something like "well, blame JIRA because they removed it" but the problem will likely take the shape of undocumented/unannounced changes which cause ongoing maintenance work on this connector. How have you been thinking about this issue in the context of your product? |
|
@sherifnada If this API breaks suddenly, we currently inform the user, via sync logs, that we can't fetch linked pull requests, and continue pulling the rest of our equivalent of Airbyte streams normally. I can implement a similar strategy here, where the Pull Requests stream will log that the stream is broken due to changes in the Jira API, without failing the entire source sync. |
|
@sherifnada Would that be acceptable? |
|
(waiting updates from user adding safeguards) |
github-actions
bot
added
the
area/documentation
|
@marcosmarxm This is ready for review. |
|
Thanks @cjwooo |
|
Bump |
| return f"https://{self._domain}/rest/dev-status/1.0/" | ||
|
|
||
| def path(self, **kwargs) -> str: | ||
| return f"issue/detail" |
There was a problem hiding this comment.
| return f"issue/detail" | |
| return "issue/detail" |
remove f-string is breaking the format
|
The connector was published in #8559 I'm closing this PR and merging the other to relese the new version of the connector. |
What
Adds a Pull Requests stream to fetch data on linked pull requests for Jira issues. Currently only supports GitHub pull requests.
Also fixes the field_ids_by_name() method on the IssueFields stream to handle multiple field ids per field name, a case we've seen with some of our projects.
How
Uses an undocumented Jira API to fetch data on linked pull requests. Only runs on Jira issues that have been determined to have linked pull requests via their "Development" field.
Recommended reading order
x.javay.pythonPre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/SUMMARY.mddocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampledocs/integrations/README.mdairbyte-integrations/builds.mdAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described hereUpdating a connector
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described hereConnector Generator
-scaffoldin their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplatesthen checking in your changes