[gh-action]: Generate a weekly report of activity

[kevinslin]

Description:

This PR creates a gh action that generates a weekly report on repo statistics.
It delivers on the requirements specified in #24672
You can see the sample output here: kevinslin#16

Link to tracking Issue: #24672

Testing:

Manual testing in fork: https://github.com/kevinslin/opentelemetry-collector-contrib/actions
Example output: kevinslin#17

Documentation:

The architecture:

• we use actions/github-script@v6 to make calls to the gh apis
• we require installing js-yaml in order to parse metadata.yaml files in order to get components. this dependency is installed during the github action run and not persisted

Some caveats about the logic:

• when this action runs, it looks back the previous 7 days and gets issues created in that time period, normalizing times to UTC
  • eg. if running this on wednesday (eg. 2023-08-25 17:35:00), it will scan issues from the previous wednesday (2023-08-18 00:00:00Z) to beginning of this wednesday (2023-08-28 0:00:00Z)
• this action writes the json payload of the report inside the issue. the payload is parsed by future reports to calculate deltas
• the report issue has a custom label: report - this is used so we can properly filter previous issues when calculating deltas. the github issues api only does filtering based on labels and since date

This action currently runs every Tuesday at 1AM UTC

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: kevinslin / name: Kevin Lin (8ec5549, ce431ba, 647de3b, 30d4fa4, b45a980, b39c660, c14d4a9, c783994, 2697e16, e76d4e2, 1059258, 8634175, 41080fd, 8932abd, 61fd613, a7a3e9d, fa4b556, 700fc62, 9412174, 9c6b35b, 31cc755, dc820f9, 3110b14, 3c78f74, bce9e73, 0551502, f61c5f1, ffc4095, 7a11c9e, 2037968, 7401fc3, 983165a, 80559bb, f6a8605, 53f77c6, 6133af1, 5918ac7)
• ✅ login: atoulme / name: Antoine Toulme (0551502, f61c5f1)

[kevinslin]

didn't add a changelog entry since this was an internal facing change. happy to add an entry if we feel this warrants that

[kevinslin]

updated issue format with gh style checkboxes: kevinslin#21

[kevinslin]

looks like the gh workflows requires an approval from a maintainer?

[andrzej-stencel]

Why not use the async versions of the fs functions?

[kevinslin]

the logic was easier to follow when sticking with sync. since the number of directories isn't that large, felt like this wouldn't be a bottleneck

[andrzej-stencel]

the logic was easier to follow when sticking with sync.

Not sure I understand. The flow of the logic is the same with async, you just add the await keyword.

    files = await fs.readdir(startPath);

The await keyword is already used elsewhere in the script, e.g.

const response = await octokit.issues.listForRepo(queryParams);

I thought maybe it's an issue in importing the fs/promises version of the fs library?

[andrzej-stencel]

If I understand correctly, by component you mean component class - a receiver, or a processor, etc. Reading the component type from the path should work, but a cleaner way would be to use the status.class field from the metadata file contents.

You could also read the component's name from type field in the metadata file.

[kevinslin]

this is because we currently collect metadata.yaml from everything. this includes internal components that don't have a status.class field (eg. processor/resourcedetectionprocessor/internal/aws/ec2/metadata.yaml). though to be fair. added a check for the status.class to set component and default to the current method if nothing is found

[andrzej-stencel]

All my comments are minor things that can be either ignored or improved upon in separate PRs. This is good to be merged in my view 👍

[kevinslin]

finished addressing pr changes. thanks for the review @astencel-sumo

[kevinslin]

checking if there are any other blockers for this pr? @atoulme do you have authorization to merge this pr?

[TylerHelmuth]

I am a little concerned that this introduces javascript to our repository. @open-telemetry/collector-contrib-approvers is everyone willing to take on this language dependency?

[kevinslin]

I am a little concerned that this introduces javascript to our repository. @open-telemetry/collector-contrib-approvers is everyone willing to take on this language dependency?

its a valid concern. i just want to point out that there is already javascript in the repository (embedded inside of yaml).
see here and here

[andrzej-stencel]

I am a little concerned that this introduces javascript to our repository. @open-telemetry/collector-contrib-approvers is everyone willing to take on this language dependency?

I've worked with some decently large Node.js apps in my past, I'd be willing to take it.

Unless we'd rather have this report generator written in a different language, that more contributors of this repo are comfortable with. What would it be? Perhaps Python?

[atoulme]

I'm fine helping with js code.

[TylerHelmuth]

Gonna give one more day for approvers/maintainers to speak up, then I'll merge.

[evan-bradley]

Speaking as someone who has written a handful of GitHub workflows as Bash scripts that call the GitHub CLI, I think writing workflows in a programming language that use an API client library is a better alternative: it's much easier to reason about and should be more straightforward to test locally. I have pretty extensive JavaScript experience, so I would be happy to help maintain JavaScript workflows.

Looking at our options, between the three language implementations that GitHub provides SDKs for, I think JavaScript is the most likely to be the best understood both within the Collector SIG and across the wider OTel community (I'd like to maintain an eye toward possibly publishing our actions to be used in other repos). I think using Go would also be an obvious choice, and there is a third-party API client for GitHub written in Go, but I can't comment on its quality compared to the official JavaScript SDK.