Rewrite it in Rust: Ruff as alternative Linter

[DavidSchischke]

Closes #22.

• Option for choosing linter in cookiecutter setup
• Automatic removal of unneeded files from other linter
• Ruff used as non-local hook with according changes in pyproject.toml
• No disabled options for ruff (tbd). See full list for reference

[FabianScheidt]

I guess you're excluding this, as we now use some templating in the file, correct?

To still make sure that the generated files pass our own quality standards, we should extend the tests with one variation that uses ruff.

[DavidSchischke]

From my understanding, the check-toml pre-commit hook tries to parse the toml within the project directory as-is, which leads to a flag because it still contains template strings. This is not necessarily related to ruff/pylint, but to the way the check itself is executed. Let's discuss how to solve this :)

[FabianScheidt]

Following up our discussion, I have implemented that the tests now execute the pre-commit hooks. On top of that, I've added the option to specify different configurations to run the tests in. With these changes in place, I'd suggest to:

1. Exclude everything on "{{cookiecutter.repo_name}}" from the top-level pre-commit configuration, as these are now validated in the tests (after cookiecutter initialized the repo)
2. Add some configuration with Ruff to our permutations.

[FabianScheidt]

I don't really understand what this is good for. Can you explain? I guess it's somehow using one file or the other, but I don't understand how 🙈

[DavidSchischke]

This change is related to a change in post_gen_project:

data-science-project-template/hooks/post_gen_project.py

Lines 110 to 116 in 25b6814

	def remove_unused_linter_files(self):
	with open(f"{self.temp_files_dir}/.manifest.yaml", "r", encoding="utf-8") as f:
	manifest = yaml.safe_load(f)
	for feature in manifest["features"]:
	if not feature["enabled"]:
	for resource in feature["resources"]:
	os.remove(resource)

Basically, after the project is created, the type of linter is determined from manifest.yaml and the config for the other linter is deleted prior to committing. The procedure is taken from this cookiecutter issue.

[FabianScheidt]

I think the main confusion here is the name of the folder .temp_ci_cd. I'd instead suggest something like .conditional_files and adjust the post generation script accordingly.

Instead of introducing yet another custom configuration file, we should move both configs (Pylint and Ruff) into that folder and adapt our ConditionalFileManager to only use the appropriate file. While we're at it, we might as well add the option to have no linter at all :)

[FabianScheidt]

As discussed, let's do some testing with some existing projects to see if these settings are reasonable.

[FabianScheidt]

I don't think that we need ruff in the top-level environment, as we're only using one linter for the template repository itself. Let's stick with Pylint for now, so I guess you can revert all changes to that file.

[FabianScheidt]

I think it's a good idea to move "nbqa" into that conditional. However, if we do this, we also need to add a condition to the pre-commit config to avoid that the nbqa-tasks are executed when the dependency is not installed.