Minimal reproduction instructions/templates

[HonkingGoose]

Tell us more.

For context read this first:

• fix(npm): replace pnpm overrides in right position #28606 (comment)

We have a renovate-reproductions/base repository. Do we want people to use this repository as a base to start their reproductions?

I guess not, because that repository has a lot of files. For example, sometimes we only need a renovate.json file and a simple package.json file to get a working reproduction.

[viceice]

no, it's simply another test repo.

[karlhorky]

Thanks for opening this discussion!

Do we want people to use this repository as a base to start their reproductions?

I guess not, because that repository has a lot of files.

no, it's simply another test repo.

Ok, I understand that it is perceived as potentially situationally overkill for Renovate maintainers.

Eg. listening to this seems to make sense to me at first:

For example, sometimes we only need a renovate.json file and a simple package.json file to get a working reproduction.

One thing that this leaves out is the experience of community members, often those who don't have visibility into:

1. What is the "current version" of best practices for creating a reproduction
2. What is required (or not required) for reproductions of different complexity

So codifying these details into 1 (or 2 or 3, if there are different levels of complexity of reproduction) maintained reproduction templates - even if the simpler level(s) of complexity are only 2 files in a repo - would in my experience in open source communities lead to:

A) higher community engagement, especially from those who are less involved right now
B) reduced back and forth between Renovate maintainers and the community, when reproductions are either not simple enough or do not check certain boxes

Even having contributed a few things to the Renovate codebases over the years, I would have to put in some effort, thinking and tie into "how to create a nice, simple reproduction", which will be a chilling effect to a lot of potential contributions.

Prior Art

• Next.js CodeSandbox reproduction template https://codesandbox.io/p/sandbox/github/vercel/next.js/tree/canary/examples/reproduction-template
• Testing Library Reproduction Template https://github.com/testing-library/dom-testing-library-template
• Ent Template Repository For Bug Reporting https://github.com/ent/bug
• Reproduction Template of ESLint Stylistic https://github.com/eslint-community/eslint-stylistic-repro-template
• GitHub template for creating a Rspack minimal reproducible example https://github.com/web-infra-dev/rspack-repro
• Renode issue reproduction template https://github.com/renode/renode-issue-reproduction-template
• Vue Test Utils Reproduction Template issue creation template repro template on StackBlitz
• shadcn-svelte Reproduction Template https://github.com/huntabyte/shadcn-repro-template
• Ant Design reproduction template repro template on CodeSandbox Link to docs
• Storybook reproduction generator in the CLI https://storybook.js.org/docs/contribute/how-to-reproduce#initial-setup
• Porsche Design reproduction generator with StackBlitz https://twitter.com/stackblitz/status/1648341661762633729
• Apollo GraphQL template for creating minimal reproductions https://github.com/apollographql/next-apollo-example

Automation of Reproduction Creation

• Bun automatic reproduction generator idea https://twitter.com/jarredsumner/status/1781214396263661985

[karlhorky]

Good idea - I created a repo here:

• https://github.com/karlhorky/awesome-open-source-automation

Just need to wait 30 days to submit to https://github.com/sindresorhus/awesome

[HonkingGoose]

That's good documentation, and good background (although long and detailed - hard to find the information of "how can I create a repro in 30 seconds")

You're right, we should get to the point quicker. I'll rewrite this soon! 😄

• Rewrite minimal reproduction guide to get to the point quicker #28730

Regardless of how easy it is to read, I would argue that such documentation does not replace a one-click reproduction creator (it could supplement it though).

It would be very nice to have a "one click" reproduction system for Renovate. I fear that creating such a thing would be very hard, as there are many configurations and platforms that end-users can have.

[karlhorky]

It would be very nice to have a "one click" reproduction system for Renovate. I fear that creating such a thing would be very hard, as there are many configurations and platforms that end-users can have.

Just to copy my issue comment over to the discussion too:

Also, for some first baby steps in the direction of a minimal reproduction template, I would add an example config link to step 3:

-1. Reduce your Renovate config to a minimum
+1. Reduce your Renovate config to a minimum (eg. [simple base config starter](https://example.com))

Then, if the idea of a minimum reproduction template is amenable to the Renovate team, this could be expanded to multiple GitHub repo templates - one for each for each package manager (eg. including an empty package.json for Node.js projects), to get closer to the "one-click starter" idea that I mentioned.

So this may be some first possible steps to create an MVP of a "one-click" reproduction starter

[HonkingGoose]

About reproduction templates for package managers

Do we want to create templates for each package manager, or even the most common managers? Like:

• npm/yarn/pnpm
• pip
• docker and docker-compose
• github-actions

Creating the templates is a one-off effort, but then we need to update them when the conventions or ecosystem changes.

Also, with more templates we increase the odds that things get out of date/stale. I don't know if we can monorepo the templates? A bit like this:

• Basic template
  • Folder for npm/yarn/pnpm
  • Folder for pip
  • Folder for docker and docker-compose
  • Folder for github-actions

Also having more options might make it harder to find the correct template, or to get overwhelmed with choice.

I'll let a maintainer weigh these points, and decide on more in-depth templates.

Start with a basic template first?

We could start with a basic template repository, and make the basic setup quicker though.

Basic template repo

I can create a "template repository" that:

• Has a readme.md file with:
  • The correct headings, formatting and styling.
  • A placeholder for a link to the upstream Renovate Issue or Discussion.
• A minimal renovate.json config file which:
  • Extends config:recommended.
  • Has the schema info (for VSCode auto-complete).
• Is marked as a template repository.

Steps for user after forking the template

1. Add the files for their specific problem (say a package.json or package-lock.json for npm type problems)
2. Update the renovate.json file with the correct config to trigger the bug/missing behavior
3. Update the link so it points to the Issue/Discussion
4. Explain Current and Expected behavior

Where to put the template repo?

After I have created the template repository, we'll have to place it in a location that we can maintain. I'll let the maintainer decide where to put it:

1. In the main Renovate organization.
2. In the Renovate reproductions organization.

Marking a repository as template

Quote from the GitHub Docs, creating a template repository:

Who can use this feature?
Anyone with admin permissions to a repository can make the repository a template.

Let me know what you think

Tell me if you like the idea, or not. 😄

[karlhorky]

The idea of trying a first version with the most common package managers would make sense to me, to keep initial investment low, track how users interact with these templates, track the maintenance effort of the templates over time.

we need to update them when the conventions or ecosystem changes

To this point, I can speak a bit:

1. The maintenance effort of package.json over time in a template is not huge. I think there have been almost zero actual necessary changes to package.json with the Next.js Reproduction Template, which I've been working on for a few years.
2. Package managers occasionally change lockfiles with new versions, which would potentially need an update - if lockfiles were to be a part of a starter template (could also be argued that lockfiles shouldn't be a part of the repro template, because users need to generate a lockfile anyway when adding dependencies)

I don't know if we can monorepo the templates?

I think having each starter template as a separate repo would be good, to enable easy usage of the GitHub repo-level features "Use template" or "Fork". So something like:

- https://github.com/renovate-reproductions/reproduction-npm
- https://github.com/renovate-reproductions/reproduction-yarn
- https://github.com/renovate-reproductions/reproduction-pnpm
- https://github.com/renovate-reproductions/reproduction-pip
- https://github.com/renovate-reproductions/reproduction-docker
- https://github.com/renovate-reproductions/reproduction-docker-compose
- https://github.com/renovate-reproductions/reproduction-github-actions

(the reason I included the reproduction- prefix is to make the repo name less confusing after forking / using as a template - could also be renovate-repro- prefix to be more explicit)

[rarkins]

I think it would be great to have some "perfect" examples of reproductions to refer to. They're meant to be minimal, so the example is almost self-documenting - everything is there for a reason

[HonkingGoose]

I'm adding a perfect reproduction example with this PR:

• docs(minimal reproductions): add perfect example #29316

[rarkins]

I hope we can have:

1. A minimal template repository for users to import (not fork, as you can only fork once per repo), e.g. includes README with headings, empty renovate.json
2. 1-2 examples of "perfect" reproductions from the past

Hopefully once done the above have very little maintenance. It doesn't matter much if the examples are for bugs which were solved years ago.

[HonkingGoose]

For item 1, see this issue:

• Create minimal reproduction template repository #29319

For item 2, I'm already adding one example with this PR:

• docs(minimal reproductions): add perfect example #29316