lint on recipe parseability

[beckermr]

The bot and many other tools rely on something being able to parse the recipe YAML.

Right now there are a few such parsers for v0 recipes in existence. They include

• conda-recipe-manager: https://github.com/conda-incubator/conda-recipe-manager
• conda-forge-tick (the bot): https://github.com/regro/cf-scripts/blob/main/conda_forge_tick/recipe_parser/_parser.py#L486
• (conda-)souschef (used by grayskull): https://github.com/marcelotrevisani/souschef

I propose that we issue a

• lint that fails for conda-forge recipes if they cannot be parsed by any of these tools
• hint per tool if the recipe cannot be parsed by that tool

The new v1 recipes are always parseable so this would not apply to them.

Thoughts @conda-forge/core?

[jakirkham]

This seems like a good idea. However it may have moved a bit too fast

Looks like the lint doesn't understand fairly simple CUDA recipes. For example: conda-forge/cudnn-feedstock#96 (comment) or conda-forge/cuda-cuobjdump-feedstock#16 (comment)

Also it notes there are logs, but doesn't actually link them. So there is nowhere to look for info

[beckermr]

I can fix the logs. I thought the websevices included a link.

However, those are both hints, not lints. So you can ignore them.

[jakirkham]

Except this is an issue asking for core's feedback that got closed in less than 24hr and included in a release

Not to mention had asked in this comment ( #2142 (comment) ) to hold off on a release until confirming we fixed an installer bug. Note there is a new pixi bug: #2150

Think we may need a bit more process here:

• Can we please open a release planning issue in the future? This could be as simple as an issue titled "Release 3.45.0".
• Could we please give folks a bit more time to respond to asks like this one?

[beckermr]

I don't think we should be too process-driven about smithy releases and waiting for prs. We are in the situation where active work is blocked on smithy releases. Thus we should make as many as we need. I'll make one tonight if you need it.

I'd also be happy cutting a release for every pr merged to main automatically.

This last thing makes sense a lot of sense IMHO. I've been running the bot this way and it really streamlines fixes propagating into the system live. If we did this, we should move to calver.

In any case, if we'd like a well-defined release process, I suggest that a CFEP be drafted and we vote on it.

If you want something backed out or fixed or changed, I'm happy to discuss as usual.

[beckermr]

I'll add that the need for a faster release cycle is really driven by our inability to write tests against external systems that change in the background.

Human code review is great for catching some kinds of errors (style, clarity, etc.), but not as good at catching bugs of various kinds. That doesn't mean we shouldn't do it, but we should recognize its limits.

Given that we can't write complete tests in many cases, we cannot anticipate all of the bugs, and that smithy has a large number of code options for various edge cases in CI builds, we're basically having to rely on running it live to test things out.

We do that by hand in many cases, but that is a huge pain.

So this leaves us with making more releases to push bug fixes as we use live runs as a test suite.

It is not ideal, but is a very natural approach given the constraints.

One idea, which I'm not a huge fan of but could make some people excited, is automate having the webservices rerender feedstocks using smithy versions from smithy PRs. This would reduce the friction around live testing and hopefully help us ship fewer bugs.

That all being said, I think fundamentally smithy dev is in many cases reactive to external things that block folks and pushing a bug fixes quickly definitely makes for a good user experience.