Enable BELL SPI Visibility and BELL Properties

[dazavala]

Description

Internal and external customers have voiced interest in two enhancements to the Basic Extensions using Liberty Libraries (BELL) feature, AKA bells-1.0.

First, issue 16317 Liberty SPI packages should be made visible to libraries referenced in BELL configurations.

Liberty features declare SPI packages that allow product extensions (i.e. BELLs, user features) to extend server functionality through the implementation of a service interface included in these packages. Because SPI packages are visible only to user features, developers must deploy their own user feature to register an OSGi service that implements a service interface. Library SPI visibility will obviate the need for BELL developers to also create a user feature in order to extend a service interface.

Support for BELL library SPI visibility will entail changes to the bells-1.0 runtime, the library configuration, the classloading gateway, and the kernel region support.

Second, issue 17414 Liberty should support the configuration of BELL service properties and make the properties available to service implementations.

Currently developers must use JVM environment variables or system properties to configure BELL services, because they cannot otherwise configure Liberty properties. Adding support for BELL service properties will make the configuration for basic extensions consistent with other configuration types.

Support for BELL service properties will entail changes to the bells-1.0 runtime and configuration, and to service implementation classes which will use the properties.

---

Documents

When available, add links to required feature documents. Use "N/A" to mark particular documents which are not required by the feature.

• Aha: Option to pass parameters to BELLs (Aha)
  • BELLs support API only, not SPI
• UFO: (UPDATED) BELL SPI Visibility and BELL Properties (UFO template)
  • Set the Box link to be publicly accessible, with a long expiration (10 years)
    • Click "Share" > select "People with link" > click "Link Settings" > under "Link Expiration" select "Disable Shared Link on" > set an expiration date ~10 years into the future
    • If you lack permissions, contact OpenLiberty/release-architect
• FTS: 19368 Feature Test Summary #19357
• Beta Blog: BETA BLOG - BELL SPI Visibility and BELL Properties #21974
• GA Blog: GA BLOG - BELL SPI Visibility and Configuration Properties #22911

---

Process Overview

• Prioritization
• Design
• Implementation
• Legal and Translation
• Beta
• GA
  • Focal Point Approvals
• Other Deliverables

General Instructions

The process steps occur roughly in the order as presented. Process steps occasionally overlap.

Each process step has a number of tasks which must be completed or must be marked as not applicable ("N/A").

Unless otherwise indicated, the tasks are the responsibility of the Feature Owner or a Delegate of the Feature Owner.

If you need assistance, reach out to the OpenLiberty/release-architect.

Important: Labels are used to trigger particular steps and must be added as indicated.

---

Prioritization (Complete Before Development Starts)

The (OpenLiberty/chief-architect) and area leads are responsible for prioritizing the features and determining which features are being actively worked on.

Prioritization

• Feature added to the "New" column of the Open Liberty project board
  • Epics can be added to the board in one of two ways:
    • From this issue, use the "Projects" section to select the appropriate project board.
    • From the appropriate project board click "Add card" and select your Feature Epic issue
• Priority assigned
  • Attend the Liberty Backlog Prioritization meeting

---

Design (Complete Before Development Starts)

Design preliminaries determine whether a formal design, which will be provided by an Upcoming Feature Overview (UFO) document, must be created and reviewed. A formal design is required if the feature requires any of the following: UI, Serviceability, SVT, Performance testing, or non-trivial documentation/ID.

Design Preliminaries

• UI requirements identified. (Owner and UI focal point)
• ID requirements identified. (Owner and ID focal point)
  • Refer to Documenting Open Liberty.
  • Feature Owner adds label ID Required, if non-trivial documentation needs to be created by the ID team.
  • ID adds label ID Required - Trivial, if no design will be performed and only trivial ID updates are needed.
• Serviceability Requirements Identified. (Owner and Serviceability focal point)
• SVT Requirements Identified. (Owner and SVT focal point)
• Performance testing requirements identified. (Owner and Performance focal point)

Design

• POC Design / UFO review requested.
  • Owner adds label Design Review Request
• POC Design / UFO review scheduled.
  • Follow the instructions in POC-Forum repo
• POC Design / UFO review completed.
• POC / UFO Review follow-ons completed.
• Design / UFO approved. (OpenLiberty/chief-architect) or N/A
  • (OpenLiberty/chief-architect) adds label Design Approved
  • Add the public link to the UFO in Box to the Documents section.
  • The UFO must always accurately reflect the final implementation of the feature. Any changes must be first approved. Afterwards, update the UFO by creating a copy of the original approved slide(s) at the end of the deck and prepend "OLD" to the title(s). A single updated copy of the slide(s) should take the original's place, and have its title(s) prepended with "UPDATED".

No Design

• No Design requested.
  • Owner adds label No Design Approval Request
• No Design / No UFO approved. (OpenLiberty/chief-architect) or N/A
  • Approver adds label No Design Approved

FAT Documentation

• "Feature Test Summary" child task created
  • Use the Feature Test Summary Template
  • Add FTS issue link to the Documents section.

---

Implementation

A feature must be prioritized before any implementation work may begin to be delivered (inaccessible/no-ship). However, a design focused approach should still be applied to features, and developers should think about the feature design prior to writing and delivering any code.
Besides being prioritized, a feature must also be socialized (or No Design Approved) before any beta code may be delivered. All new Liberty content must be inaccessible in our GA releases until it is Feature Complete by either marking it kind=noship or beta fencing it.
Code may not GA until this feature has obtained the "Design Approved" or "No Design Approved" label, along with all other tasks outlined in the GA section.

Feature Development Begins

• Add the In Progress label

Legal and Translation

In order to avoid last minute blockers and significant disruptions to the feature, the legal items need to be done as early in the feature process as possible, either in design or as early into the development as possible. Similarly, translation is to be done concurrently with development. Both MUST be completed before Beta or GA is requested.

Legal (Complete before Feature Complete Date)

• (N/A) Changed or new open source libraries are cleared and approved, or N/A. (Legal Release Services/Cass Tucker/Release PM).
• Licenses and Certificates of Originality (COOs) are updated, or N/A

Translation (Complete 1 week before Feature Complete Date)

• PII updates are merged, or N/A. Note timing with translation shipments.

---

Beta

In order to facilitate early feedback from users, all new features and functionality should first be released as part of a beta release.

Beta Code

• Beta fence the functionality
  • kind=beta, ibm:beta, ProductInfo.getBetaEdition()
• Beta development complete and feature ready for inclusion in a beta release
  • Add label target:beta and the appropriate target:YY00X-beta (where YY00X is the targeted beta version).
• Feature delivered into beta
  • (OpenLiberty/release-manager) adds label release:YY00X-beta (where YY00X is the first beta version that included the functionality).

Beta Blog (Complete 1.5 weeks before beta eGA)

• Beta blog issue created and populated using the Open Liberty BETA blog post template.
  • Add a link to the beta blog issue in the Documents section.

---

GA

A feature is ready to GA after it is Feature Complete and has obtained all necessary Focal Point Approvals.

Feature Complete

• Feature implementation and tests completed.
  • All PRs are merged.
  • All epic and child issues are closed.
  • All stop ship issues are completed.
• Legal: all necessary approvals granted.
• Translation: All messages translated or sent for translation for upcoming release
• GA development complete and feature ready for inclusion in a GA release
  • Add label target:ga and the appropriate target:YY00X (where YY00X is the targeted GA version).
  • Inclusion in a release requires the completion of all Focal Point Approvals.

Focal Point Approvals (Complete by Feature Complete Date)

These occur only after GA of this feature is requested (by adding a target:ga label). GA of this feature may not occur until all approvals are obtained.

All Features

• APIs/Externals Externals have been reviewed or N/A. (OpenLiberty/externals-approvers)
  • Approver adds label focalApproved:externals
• Demo Demo is scheduled for an upcoming EOI or N/A. (OpenLiberty/demo-approvers)
  • Approver adds label focalApproved:demo.
• FAT All Tests complete and running successfully in SOE or N/A. (OpenLiberty/fat-approvers)
  • Approver adds label focalApproved:fat.
• Globalization Translation and TVT are complete or N/A. (OpenLiberty/globalization-approvers)
  • Approver adds label focalApproved:globalization.

Design Approved Features

• Accessibility Accessibility testing completed or N/A. (OpenLiberty/accessibility-approvers)
  • Approver adds label focalApproved:accessibility.
• ID Documentation is complete or N/A. (OpenLiberty/id-approvers)
  • Approver adds label focalApproved:id.

  • NOTE: If only trivial documentation changes are required, you may reach out to the ID Feature Focal to request a ID Required - Trivial label. Unlike features with regular ID requirement, those with ID Required - Trivial label do not have a hard requirement for a Design/UFO.

• Performance Performance testing is complete or N/A. (OpenLiberty/performance-approvers)
  • Approver adds label focalApproved:performance.
• Serviceability Serviceability has been addressed or N/A. (OpenLiberty/serviceability-approvers)
  • Approver adds label focalApproved:sve.
• STE Skills Transfer Education chart deck is complete or N/A. (OpenLiberty/ste-approvers)
  • Approver adds label focalApproved:ste.
• SVT System Verification Test is complete or N/A. (OpenLiberty/svt-approvers)
  • Approver adds label focalApproved:svt.

Remove Beta Fencing (Complete by Feature Complete Date)

• Beta guards are removed, or N/A
  • Only after all necessary Focal Point Approvals have been granted.

GA Blog (Complete by Feature Complete Date)

• GA Blog issue created and populated using the Open Liberty GA release blog post template.
  • Add a link to the GA Blog issue in the Documents section.

Post GA

• Replace target:YY00X label with the appropriate release:YY00X. (OpenLiberty/release-manager)

---

Other Deliverables

• Standalone Feature Blog Post A blog post specifically about your feature or N/A. (OpenLiberty/release-architect)
  • This should be strongly considered for larger or more prominent features.
  • Follow instructions in the blogs repo.
• OL Guides OL Guides assessment is complete or N/A. (OpenLiberty/guide-assessment)
• Dev Experience Developer Experience & Tools work is complete or N/A. (OpenLiberty/dev-experience-assessment)

---

[jhanders34]

The recording from the UFO socialization from Friday is here: https://ibm.ent.box.com/file/949800542801

Feedback from the socialization was:

Slide 18
Look at having only one setSpiTypeVisiblity. Scratch the second one.

Slide 17 to 19
Move to having this be an internal interface / Enum instead of exposing it in the SPI package. Or just cast to the impl.

Slide 21
Lots of discussion on how to configure this.

Options discussed:

• spiTypeVisibility="all"
• apiTypeVisibility="+spi"
• Add spiVisibility to the bell configuration instead of being on Library
• Make a new library type like spiLibrary or something that is used by Bells.

Have discussion at design issues call about these options.

Slide 28
Add some additional tests for use of Liberty SPIs to use a bell instead of using user features. Focus on the scenarios where this pattern makes a lot of sense. ServletContainertInitializer was one of the scenarios mentioned.

Slide 29
Possibly add something to SVT to convert / add a bell for what we are doing a user feature for today.

[dazavala]

Opened issue 20916 Enable BELL SPI visibility w/o Library configuration to address design feedback from the apr22 UFO socialization.

[yeekangc]

Notes from Part 2 of UFO Review (on May 20)

Slides 20 & 21

• Need a design issue to discuss and figure out what we should do here (for this feature and its UFO) especially around handling of service-specific properties
• A BELL 2.0 may be the option to consider to best support service-specific properties given the current design (and behaviours in BELL 1.0)
• Consider a design with service configuration elements under the element? Keep the current behaviour if no service config elements specified (as children).
• Typos: Erroneous comma (",") in the examples

Slide 28

• Consider system test that covers service properties too

Session recording: https://ibm.box.com/s/ao2hfn9wnpg7whphcbl5gom3q7170k6e

[NottyCode]

@dazavala In general I prefer to avoid using service properties with enable in their name. Has anything else been discussed as an option for enableSpiVisibility?

[dazavala]

Hello @NottyCode. The original proposal was spiVisibilityEnabled, which I changed to enableSpiVisibility after finding similar examples in the Liberty configuration metatypes. IIRC we have not discussed any other names for this attribute, but I think spiVisibilty (boolean) would do fine.

[dazavala]

@NottyCode I ran this by @tjwatson. We'll change the attribute name to spiVisibility (boolean).

[dazavala]

@OpenLiberty/demo-approvers Demo scheduled for EOI 22.16 (23aug2022)

[donbourne]

Serviceability Approval Comment - Please answer the following questions for serviceability approval:

1. UFO -- does the UFO identify the most likely problems customers will see and identify how the feature will enable them to diagnose and solve those problems without resorting to raising a PMR? Have these issues been addressed in the implementation?

Yes. See slides 32-35 of the updated UFO

2. Test and Demo -- As part of the serviceability process we're asking feature teams to test and analyze common problem paths for serviceability and demo those problem paths to someone not involved in the development of the feature (eg. L2, test team, or another development team).
   a) What problem paths were tested and demonstrated?

-Demo classloading exception for BELL service that implements SPI, but user does not enable BELL SPI visibility.
-Discuss library API visibility and stress design point that SPI visibility applies only to libraries that provide BELL services. Applications and resources can never see SPI.
-Demo unsupported use of the global library with BELL SPI visibility enabled.
-Demo syntax errors in BELL property strings that cause server.xml parse errors at startup and dynamic update.
-Demo declaration of multiple properties elements containing duplicate property names.
-Explain the error which occurs when a service implementation is not enabled to receive BELL configuration properties.

b) Who did you demo to?

L2 STE focal and SMEs, L3 SME: Thanh Giang, Krishna Jaladhi, Marcia Amico, Jarid Kvale, Varun Tallapragada

c) Do the people you demo'd to agree that the serviceability of the demonstrated problem scenarios is sufficient to avoid PMRs for any problems customers are likely to encounter, or that L2 should be able to quickly address those problems without need to engage L3?

Yes, with three takeaways:

1. The errors and messages emitted by open-liberty look effective and should prevent new skills cases.
2. The mustgather for the BELL enhancements did not change. As before, BELL feature mustgather includes Bells+Classloading trace and server introspections related to classloading.
3. I will deliver an STE deck for the BELLs feature so L2 can have education materials on-hand, but an STE presentation is not yet necessary as the BELLs feature is not heavily used.

3. SVT -- SVT team is often the first team to try new features and often encounters problems setting up and using them. Note that we're not expecting SVT to do full serviceability testing -- just to sign-off on the serviceability of the problem paths they encountered.
   a) Who conducted SVT tests for this feature?

N/A - We've no SVT requirement for the BELLs 1.0 feature.

b) Do they agree that the serviceability of the problems they encountered is sufficient to avoid PMRs, or that L2 should be able to quickly address those problems without need to engage L3?

N/A

4. Which L2 / L3 queues will handle PMRs for this feature? Ensure they are present in the contact reference file and in the queue contact summary, and that the respective L2/L3 teams know they are supporting it. Ask Don Bourne if you need links or more info.

The L2 and L3 classloading missions will service cases involving the BELLs 1.0 feature.
For L2, WASWIN currently owns the classloading mission, which should fully transfer to WASCET by 1Q2023.
For L3, Liberty Classloading.

5. Does this feature add any new metrics or emit any new JSON events? If yes, have you updated the JMX metrics reference list / Metrics reference list / JSON log events reference list in the Open Liberty docs?

No.

[chirp1]

Dave adding info to doc to OpenLiberty/docs#5724 Will approve epic after info is added.

[chirp1]

Info to doc provided in doc issue github.com/OpenLiberty/docs/issues/5724#issuecomment-1262520703 . Approving epic.

[tngiang73]

@dazavala: As we discussed in the meeting, L2 would like to have STE slides for this feature. The STE template can be found at the links below. You can use either one to create the education.
Slide Template: https://ibm.box.com/s/1an42g7zdgmaj84w7dft0indqfgi8ffm
Github Template: https://pages.github.ibm.com/WASL3/site/STE/about
Please upload the completed slides to the same 'STE Archive' BOX folder and let me know when they're complete or provide me the Github link. Thanks!

[dazavala]

Feature content is confirmed for the 22.0.0.11 GM build. Moving forward with user-doc revisions (OL doc issue 5724) and opened the ga blog issue.

[dazavala]

ID story 5724 is now labeled technical reviewed and all edits are merged in the draft document. User docs are ready for translation and publishing.

[dazavala]

The GA blog review is now complete. We can close this epic and the ga blog issue.