Change the "Blacklist / Whitelist" nomenclature

[tlfeng]

Goal of the enhancement

Coming from #2589

Replace non-inclusive terminologies "blacklist" and "whitelist" throughout this repository with inclusive ones.

Context

What is “inclusive language”?

Inclusive language is designed to avoid excluding people on the basis of gender, sexual preference, age, race, ability, etc. By using language that avoids expressions which imply or express ideas that have prejudice to any particular group of people, we aim to create a more equitable community.

Why using “inclusive language” is important?

Using inclusive language helps build an inclusive environment, which accepts diversity and ensures any individuals and groups feel welcome, respected, and safe. Diversity does contribute to create a fair and strong organization, and also leads to diversity of ideas, which change into creativity and innovation.

How to apply “inclusive language” in a software program?

Replace “objectionable language in code bases and documentation. It involves assessing existing code bases and documentation, identifying potentially problematic language, then replacing terms with more preferable language.

Solution

Overview

To solve the issue, breaking changes will be introduced to OpenSearch, which requires backwards compatibility. The exclusionary terms exist in APIs and configurations, which will impact the compatibility with previous versions of OpenSearch and its third-party clients and tools.
We will support both and new APIs or configurations for a period of time, to let users get used to the breaking changes gradually. Considering OpenSearch follows Semantic Versioning, we are going to include the new API and configuration in current minor versions, while continuing to provide support for the previous APIs and configurations for a period of time, then remove support for the old ones in a next major version.

Substitute for the exclusionary terminology

Our proposed preferable terminologies:

1. Use allowlist to replace whitelist
2. Use denylist to replace blacklist

Location for the terminologies:

• code comments
• internal variable, method and class names
• Gradle build environment configuration
• field, method, class and package names that are exposed as API in Java libraries (renaming will impact clients and plugins that depend on OpenSearch code base)
• setting names and values (including definition, yml configuration files)
• tests

Versions to introduce the change

• Add the new usages with inclusive terminology and deprecation notice for the old usages in minor versions of 1.x.
• Remove the support for the old usages (APIs and settings) in a major version, it could be either 2.0 or 3.0.

Explanation:

1. The ideal case is to apply inclusive naming to replace the exclusionary terminology as soon as possible, but the backwards compatibility is a key aspect to be considered, so with new usages added soon, the old usages will be kept for a while before removal to keep the compatibility.
2. Considering OpenSearch follows Semantic Versioning, deprecation notice MUST come at a minor version, and the breaking change, such as API removal MUST come at a major version.
3. Update in March 2022: In practice, new usages will be added in version 2.0, and old usages will be removed in at least version 3.0.

Procedure for the replacement

1. Replace the exclusionary words that won’t impact the compatibility.

(source of the idea)
Replace all the exclusionary words in the following location:

• code comments
• Internal variable, method and class names
• Gradle build environment configuration

2. Deprecate old usages that having exclusionary words and impact the compatibility, then create alternative usages with the inclusive terms.

(source of the idea)
This step includes the following effort:

1. Deprecate the setting which contains exclusionary words.
2. Add proper form of deprecation notice for every setting changes, including in code (such as adding @Deprecated annotation), console output and logs.
3. Create alternative settings where having name change, so that calls to the new names can have the same effect with the old names.
4. For settings, adding fallback logic to fallback to the existing setting with old name, when the corresponding new setting is not configured.

The location that will impact the compatibility when changed:

• Configuration, including setting name.

3. Add tests to check both old and new usages - in Settings.

(source of the idea)
Adding new unit tests to check the both old and new names have got the same functionality.

4. Replace the exclusionary words in log messages and update documentation.

This step includes the following effort:

1. Replace the exclusionary words in logs.
2. Update the user and developer documents to introduce the new usages, and notice the deprecation of old usages.

5. Replace the exclusionary words that will impact software programs depend on OpenSearch Java libraries - in Java APIs.

Replace the exclusionary words in all Java APIs, including field, method, class, and package names.
The "Java APIs" refers to those packaged in Java libraries and are published to Maven (https://search.maven.org/search?q=g:org.opensearch https://mvnrepository.com/artifact/org.opensearch)
Impact of this step:
All plugins, clients and tools that use OpenSearch Java APIs from OpenSearch Java libraries which contain non-inclusive terminologies have to make corresponding changes to call new APIs, if they want to upgrade the dependency to a future major version of OpenSearch.

6. Remove the deprecated settings in a future major version.

Sub-issues

• Replace "blacklist/whitelist" terminology in code comments, and internal variable, method and class names #1484
• Add alternative usage for the configuration that contains "blacklist/whitelist" terminology #1547
• Replace "blacklist / whitelist" terminology in Gradle build environment configuration  #2458
• Replace "blacklist/whitelist" terminology in Java APIs #1683
• Remove the usages with non-inclusive language.

Appendix

Location that contains the exclusionary terms and will impact the compatibility when changed

Settings
1 reindex.remote.whitelist (static)
Code: https://github.com/opensearch-project/OpenSearch/blob/1.0.0/modules/reindex/src/main/java/org/opensearch/index/reindex/TransportReindexAction.java#L60

Location that contains the exclusionary terms and will impact software programs depend on OpenSearch Java libraries

API of Java library
1 Painless SPI - multiple classes with name “Whitelist”
Code: https://github.com/opensearch-project/OpenSearch/tree/1.0.0/modules/lang-painless/spi/src/main/java/org/opensearch/painless/spi
Javadoc: https://opensearch.org/javadocs/1.0.0/OpenSearch/modules/lang-painless/spi/build/docs/javadoc/org/opensearch/painless/spi/package-summary.html

Related issues and PRs

Existing PR before the issue created: #744
Related issue: #472

Issues for other repositories in the organization:

• [FEATURE] Change the "blacklist / whitelist" nomenclature documentation-website#449
• Change the "Blacklist / Whitelist" nomenclature sql#464
• Deprecate the "Blacklist / Whitelist" nomenclature security#1665
• Change the "Blacklist / Whitelist" nomenclature alerting#321
• Deprecate the "Blacklist / Whitelist" nomenclature OpenSearch-Dashboards#1319

[stockholmux]

Hey - just noticed that this was on the roadmap for 1.3 yet it's marked "breaking." The body of this issue does describe what's happening but if you just look at the roadmap it's alarming to see "breaking" on a minor release.

Can we remove that tag?

[dblock]

@stockholmux I think we want the non-breaking changes in 1.3, so for now I removed breaking instead.

[morsik]

Waiting for official english dictionary to remove "black" and "white" colors from it due to "excluding people on the basis of gender, sexual preference, age, race, ability"…

How the hell "whitelist" and "blacklist" can even be excluding to anyone?! Don't you know the physics thing called "shadows"? It makes things invisible… dark… (shouldn't even mentiond "black" because someone will kill me in a second because of that!).

I just really want to understand the reasoning for this. And reasoning should be reasonable! Not some imaginary problem (you obviously have here).

[dblock]

@morsik #472 (comment)

On behalf of the maintainers of the project I agree that the term 'master' may be offensive to some people. This project is committed to being a community where everyone can join and contribute. This means using inclusive and welcoming language. The repo defaults to main for the branch and, in a similar vein, supporting inclusive language should extend to the API and the rest of the software as long as it can be made in a backward compatible way.

[morsik]

@dblock "may be offensive to some people" is not explanation. You can cover anything behind this sentence just for sake of not answering this question. I'm asking about real explanation, who and how is offended by this?

[dblock]

@dblock "may be offensive to some people" is not explanation. You can cover anything behind this sentence just for sake of not answering this question. I'm asking about real explanation, who and how is offended by this?

I have offended Engineers on my teams over the years by using non-inclusive language in technical conversations. I've referenced race, gender and sexual orientation, physical and mental ability. I am not going to ask these individuals to explain here how they were offended, or make a list of who they are. But I've been asked directly, 1:1, to change the words I use, therefore I support this change. Other people's experiences matter to me.

[morsik]

I am not going to ask these individuals to explain here how they were offended

This seems kinda weird for software engineer… every change I do in my software or infrastructure is made because of some reason which I have to understand why something happens and make clear decision based on circumstances. Based on what you said… you don't even know why you want to make this change.

I've referenced race, gender and sexual orientation, physical and mental ability.

Well, that's your problem that you were a bad person to others, not mine and not other people in community. I don't see why your personal problems should reflect to software. Not to mention that "blacklist" and "whitelist" don't refer to "race, gender and sexual orientation, physical and mental ability" at all.

[jcgraybill]

Hi @morsik, thanks for expressing your concern here. We definitely don't make breaking changes lightly, and I hear your feedback that every change has a cost.

We're committed to avoiding noninclusive language throughout the project. I recognize some members of the community may be encountering this perspective on "blacklist" and "whitelist" for the first time. Sometimes the realization that a familiar term is actually not the best can be surprising, and curiosity or skepticism about that is totally rational.

"Blacklist" and "whitelist" are recognized as noninclusive terms pretty broadly: this isn't a concept that originated from this project, and our intention is to recognize and respect this rather than to re-debate it.

In this case, I think this change has the benefit of making this terminology more accurate and understandable: as the lists are of things that are allowed and denied, not lists of things that are colors.

[dlvenable]

This issue came up in the community meeting yesterday. It sounds like the plan is to remove the terms "blacklist" and "whitelist" in 2.0. The reindex.remote.whitelist setting has been around since at least Elasticsearch 6, and I think a longer deprecation path is more appropriate.

It's good that the new fields will come soon in 1.3 so that developers can start making changes soon. But, removing the field from APIs in 2.0 will make migration harder.

[dblock]

This issue came up in the community meeting yesterday. It sounds like the plan is to remove the terms "blacklist" and "whitelist" in 2.0. The reindex.remote.whitelist setting has been around since at least Elasticsearch 6, and I think a longer deprecation path is more appropriate.

It's good that the new fields will come soon in 1.3 so that developers can start making changes soon. But, removing the field from APIs in 2.0 will make migration harder.

Let's start by implementing a backwards-compatible change and we can decide the deprecation -> removal separately.

[qreshi]

@dblock Regarding these changes for the plugins, do we have an ETA on when OpenSearch and OpenSearch-Dashboards will rename their references?

Alerting, for example, is referencing Painless and NodeListeners which have things like WhitelistLoader and localNodeMaster, etc. So we can't fully remove our references until core does.

[dblock]

@tlfeng ^

[tlfeng]

Regarding these changes for the plugins, do we have an ETA on when OpenSearch and OpenSearch-Dashboards will rename their references?

Hi @qreshi 😄 Such class/method names that in OpenSearch Java libraries will be renamed in a future major version. At least in version 3.0, but depends on the public acceptance. It is tracked in issue #1683. I will keep updated.
There was a discussion on renaming usages in the public Java APIs (#2453 (comment)). Although deprecating old usage and create new usage is the normal way, there are too many public class/method names that having the non-inclusive name, I'm likely to rename them in-place in a major version instead.

[qreshi]

@tlfeng Got it, so it's safe to say that these issues being tracked for the plugins don't need to be closed out by 2.0 preview or GA? Since this issue is currently tagged as 2.0.0 and on the roadmap should we disambiguate that by detailing a high-level summary of what we expect to be done by 2.0 and what is targeted for 3.0?

[tlfeng]

Since this issue is currently tagged as 2.0.0 and on the roadmap should we disambiguate that by detailing a high-level summary of what we expect to be done by 2.0 and what is targeted for 3.0?

@qreshi That's a reasonable suggestion! I will add more detail to disambiguate the current process. Thank you! 😁

[kartg]

This issues spans several changes, some of which are not in scope for 2.0.0. I'll go ahead and drop the v2.0.0 tag

[flavienbwk]

Our society has a serious problem with intellectual honesty: biased by ultra-minority political opinions from people who just yell loudly.

It's even worse here: OpenSearch engineers, only witnessing their close surrounding is bothered by this "black/white terminology", are totally disrupting the roadmap with a change that will take them months to make.

Have you conducted an scientific survey on this alleged problem ?

[anasalkouz]

With 2.4 release, we have deprecated all non-inclusive usages (i.e. master, blacklist and whitelist) and we are planning to remove those deprecated usages on 3.0 release.

[anasalkouz]

Closing the issue