Jarek Potiuk thought Apache Airflow’s dependency management system was broken—and that he knew how to fix it. Then the principal engineer at Polidea, Potiuk was new to the open source workflow management project and eager to make his first contribution. So, he jumped into Airflow’s Slack and proposed a new way to handle dependencies.
After talking through his idea with several people, Potiuk sent an email to the Airflow developers’ mailing list announcing his plan to rewrite the project’s dependency management system. “I thought I had consensus,” says Potiuk, now a member of Apache Airflow’s Project Management Committee. “But one of the core maintainers sent a firm yet gentle reply explaining that I hadn’t actually gone through the appropriate process.”
Airflow is a large project with several stakeholders, including major companies that employ many of the project’s maintainers. Like all other Apache Software Foundation projects, Airflow requires all major changes to seek consensus through the developer mailing list. And they do mean consensus: A single person can block a proposal from moving forward.
Once that process started in earnest, Potiuk realized he was actually wrong: The existing dependency management system wasn’t completely broken and his proposal wouldn’t necessarily be an improvement over the status quo. “I changed my mind, but other people changed their minds too,” he says. “They didn’t like the direction I proposed, but we agreed that something needed to change.” Together, the Airflow team built something that was better than both the status quo and Potiuk’s original proposal. “That’s why I love the Apache Software Foundation way,” Potiuk says. “It results in better software.”
Decision-making is the essence of open source maintainership, from setting a project roadmap to evaluating pull requests to assigning commit access to contributors and beyond. Whether those decisions are made by an individual or by a large group of stakeholders, as with Airflow, there are a few principles that maintainers can keep in mind as they grow their community.
Casting the net
Gathering feedback is a key aspect of any decision-making process. Today, the Astro team uses an RFC process inspired by projects like Rustand Vue.js. But Astro founder Fred Schott didn’t put a lot of structure in place during the early days when there were fewer than 100 people on the Astro Discord server, and even fewer logged in and participating at any given time. Instead, he let the project grow organically and added structure along the way.
“When we were just starting out, the stakes were lower and we just discussed ideas in GitHub and Discord, without any formal decision-making process,” Schott says. “I’m a big fan of not over structuring things until you know what problem you’re trying to solve.”
Early on, having little formal process meant there were fewer barriers to leaving feedback, which is important given that open source maintainers don’t always know much about how their software is used, or by whom. “You might not know much about the people using an open source project until something breaks and you get a lot of bug reports or complaints,” says Rose Judge, a maintainer of Tern.
It’s also important to fairly weigh the feedback you receive. “We communicate with hundreds of open source users,” says Zhiyuan Ju of the Apache APISIX Project Management Committee. “We find that they have different usage scenarios and needs for Apache APISIX. Therefore, we need to find the balance of the product to meet the needs of other users.”
Caleb Porzio, creator of Livewire, a full-stack framework for Laravel, and the front-end framework Alpine.js, says he doesn’t limit himself to listening to just users of his projects. He’s found value in talking with those who have opted not to use his projects. “Someone from the Laraval community was very public about how they were planning to use Livewire in an app,” Porzio says. “But they ended up using another tool instead.” Porzio reached out to that developer to learn why they ended up choosing Inertia, and that conversation helped shape the roadmap for Livewire. “Listening to criticism and really digging in is so important,” he says.
Gathering feedback is about more than just listening to ideas. SQLAlchemy maintainer Mike Bayer emphasizes the importance of deeply understanding why people want particular changes made. That means not just understanding the stated rationale behind a new feature or pull request, but its problem space. “Users often want to do something that is a subset of a bigger problem space without taking the bigger picture into account,” Bayer explains. “By checking what that bigger problem space is, we avoid the trap of implementing something that breaks this problem space for future use cases." For example, a user might propose a new SQL construct that accepts a column and a table, only for the developers to later realize that a more general form of that construct could use multiple columns and tables, resulting in a more robust solution. “Even for the most trivial API changes, I absolutely require that I fully understand the problem being solved; no API feature is ’obvious’ otherwise,” he says.
Creating a single source of truth
When Astro was new, the project’s lack of strict processes for decision-making made it easier to gather feedback. As the project grew, the opposite became true: More formality made it easier for people to discuss ideas without having their comments disappear into the ether.
"We added more Discord channels to help organize conversations," Schott says. And, of course, they introduced the RFC process for making major decisions, which gives all stakeholders a single source of truth for planned changes. Similarly, all Apache Software Foundation projects use public-facing mailing lists to discuss major decisions. "If it didn't happen on a mailing list, it didn't happen," says Niklas Merz of the Apache Cordova Project Management Committee. Inclusion is a big motivator for this: It doesn't matter what quality internet connection you have or what time zone you're in, Merz explains. If there's a decision to be made, you can be a part of it.
Of course, seeking community-wide consensus for every single change to a codebase would take too long, so Apache projects handle smaller decisions in different ways. The Apache Cordova project, for example, uses discussions on individual pull requests to decide whether to merge smaller changes, such as bug fixes, and saves the larger proposals for issues, like whether to deprecate support for older platforms.
Not every aspect of decision-making can be standardized, but making it clear how community members should share their thoughts, and making the process as transparent as possible, goes a long way toward making projects healthy and scalable.
Goals as a North Star for decision-making
Knowing the goals of your project can help guide your decisions. Whether they’re made by an individual, a committee, or a whole community, the more you align your decisions with clear goals the better—and easier—those decisions will be.
“Our primary goal for Tern is for it to be a useful tool that can help address the needs of the secure container supply chain ecosystem,” says Judge. “It does this by creating a software bill of materials for containers so developers can know what’s in their container images and ultimately make better decisions about their container supply chain. Whether we’re considering adding a feature to our roadmap or just merging a pull request, it’s helpful to keep those goals in mind.”
For example, the Tern team recently migrated from pulling container images using Docker to using Skopeo. It was an important decision because the switch meant breaking compatibility with the way certain functions were implemented in previous versions of Tern. But the change provided better compatibility for Tern to run in Kubernetes, therefore supporting their goal of addressing the needs of the broader container ecosystem.
Porzio’s goal for Livewire is simplicity. That goal guides decisions from features to seemingly small things like naming conventions. An otherwise technically-sound change might be rejected if it isn’t consistent with other parts of the software, because inconsistencies can introduce friction and complexity into an otherwise simple process. “I try to make everything harmonious,” he says. “Deviating from established patterns makes the experience less seamless.”
It’s about people
Even with clear goals and a sound decision-making process, it can still be hard to make certain calls. The beautiful thing about software is that code can always be changed. Bad commits can be rolled back. But, it can be much harder to patch damaged relationships with the community. “My biggest regrets are more social than technical,” Porzio says. That doesn’t mean you should merge every pull request regardless of technical merit, but it does mean being diplomatic when communicating decisions is essential. “I’m really bad at saying no, so when I reject a pull request, I tell people that it’s not me locking them out, it’s me saying ’not right now,’” Porzio says.
It might be tempting to put off breaking the news to someone that their idea or patch was rejected, but Judge emphasizes the need to respond quickly. “A lack of response discourages participation,” she explained in a talk at Maintainer Summit 2021 on the topic of letting contributors down easy.
Schott agrees. “You don’t have to spend all your free time answering questions or fixing bugs, but you need to be responsive,” he says. “If you’re not engaging with people, you can’t expect them to stay interested.”
When in doubt, maintainers should always remember that they’re not in it alone: They have a community to help them make the right call.
