[10pt,7pt] Detailed Project Planning & UX Workshop for CALL#4 (Belgrade)

[RalphSteinhagen]

We will have an in-person workshop to do the detailed task and SP break-down for the next CALL#4 digitizer OP integration iteration in Belgrade. This issue is to track the tentative outline and SP effort needed for this.

Proposed Agenda for Workshop

Day 1: Project Planning and UX Design Discussion

Morning session (3 hours)

1. UX review and design optimization: Brief discussion on the current UX state and areas of improvement.
   • Touch Support & AR?
   • Corporate Look
2. Opencmw digitizer integration backlog: Review the integration backlog and prioritize tasks.
   • Revisit data aggregation (failure and error handling, circuit breaker pattern, ...)
   • DNS signal selection, timing-context and Subscribe-Source (snapshots, streaming, triggered, post-mortem) integration
   • UI Data Sink behaviour (streaming, triggered/DataSet, snapshot, etc.)
   • UI Contour Plots, Mountain-Range Plots, user-defined ad-hoc indicators, etc.
   • UI Display of meta-data

Afternoon session (3 hours)

1. Detailed UX design review round 2: A deeper dive into the UX designs focusing on polishing the UI for 'Susan'(DONE) & 'Paul'.
2. Automated CI/CD microservice management framework: Delve into the development of a robust framework for managing CI/CD operations.
   • Monitoring
   • Continuous Deployment Pipeline
   • System- & Deployment-Level Debugging
   • General Management

Day 2: GNU Radio and UX Iteration

1. GNU Radio: Discuss the migration of existing Digitizer GNU Radio (GR) modules from V3.7 to V4.0.
   • Modernization of gr-flowgraph (check what is missing)
2. UX Iteration: Based on the feedback from Day 1, revisit UX designs and discuss potential changes.
   • Signal selection
   • Timing-context
   • Charts/views
3. Wrap up and AOB: Consolidate the discussions over the two days and plan for next steps.

[wirew0rm]

Meeting Minutes: Belgrade meeting for Project Planning and UX Design Discussion

3.-4.7.2023, Belgrade
Participants:

• Björn Balazs (BB, @bbalazs, KDAB)
• Ivan Čukić (IC, @ivan-cukic, KDAB)
• Ralph Steinhagen (RS, @RalphSteinhagen, FAIR)
• Alexander Krimm (AK, @wirew0rm, FAIR)

0. General Project Status

• No MVP yet

  • Priority 1: Finish items in [0pt] intermediate step: make graph-prototype pre OP-usable for FAIR/OpenDigitizer-UI #46 by GRCon23 on 5th September.
  • Priority 2: Differentiating features required for the UX workshop by end-September/start-October:
    • Real or mocked digitizer GR data sources.
    • Selection/filtering of the signals provided by the DNS service.
    • Selecting the timing context e.g. FAIR.SELECTOR.C=<int1>:S=<int2>:P=<int3>.
      • Look-up of timing context integers to identifiers (e.g. P="INJECTION") to be provided by FAIR
    • Acquisition mode changeable at runtime (may trigger flowgraph restart).

• Deployment:

  • Initial deployment for 10-20 digitizers, eventually scaling up several hundreds.
  • Exploration of suitable infrastructure for clusters.
  • Mostly done on FAIR side for the immediate timeframe, but aspects might be needed in service part or parts might be done in follow up call.
  • Detailed discussion below in individual point.

1. General UX review and design optimization

• RS gave a recap of the previous UX workshop in Berlin.
• Issues:
  • Need for a more professional and graphics-rich UI.
  • Need to cater to both feature-rich and minimalistic UI expectations.
• Discussion Points:
  • Emphasis on targeting FAIR users, do not include new personae especially for UX workshop.
  • Identified inconsistencies/UX-problems in the UI -> listed in separate issue [EPIC] UI Improvement for OpenDigitizer #117
  • The single UI has to be flexible enough for both we don't have the resources to provide and maintain multiple versions.
  • Discussion on hiding interactions (e.g hover menus and interactors) concluding that for our case hiding them is OK since the UI is aimed at trained operators.
• Actions:
  • @ivan-cukic Coordinate with Giulio regarding ImGui estimates for realization of UX-improvements and potentially applying of a UI makeover by a UI designer.

2. UI for signal selection

• Discussion
  • Debates on the pros and cons of radial menus.
  • Consideration of search boxes inspired by shopping sites or issue filtering searchbar on gitlab.
  • Issue of multiple selections on a single level.
  • Smart Searchbar + Menu combination
  • Idea: Allow users to select signals based on experiment/filtering contexts.
• Collected ideas and requirements, detailed UX proposal to be drafted after the workshop.
• Actions:
  • @bbalazs: Team up with Nuno for UX of signal filtering (and maybe a bit of graphics UI design). [10SP]
  • @ivan-cukic: Coordinate with Giulio regarding ImGui estimates for the implementation of the signal filtering UI [5-10SP].

3. Transitioning between acquisition modes

• The needs to be able to change the acquisition mode of a signal in the flowgraph, as otherwise he would have to locate the same signal via the signal selection again which is tedious.
• Not a UI only task because the underlying data formats are different for different acquisition types.
• Transistioning between different acquisition modes at runtime is a differentiating requirements for the MVP.
  • only allow to change acquisition mode for very simple flowgraphs where the source is directly connected to the chart. Complex flowgraphs with postprocessing to be added later. (Also not possible in the GR3 based digitizer implementation)

4. Opencmw digitizer integration backlog

• RS provided insights into the 'revisit data aggregation' task.
• Determine which of the resilience patterns (e.g from resilience4j) to implement either as separate blocks or incorporated into aggregation blocks.
• How to expose information about (partial) failures to UI/monitoring (eg. failure pattern, communication, statistics, ...)

5. Automated CI/CD microservice management framework

• Discussions on deployment strategies.
• Monitoring of systems and use of tools like Grafana.

• Overview:

• Discussions on continuous deployment and machine image generation.
  • Deployment considerations for different versions of digitizers.
  • Monitoring using time-series DB.
  • Role-Based Access Control (RBAC) for privileges.

• general architecture
  • generic machine type specific netboot image (YOCTO)
  • self contained package images for opendigitizer
    • to be able to run multiple versions without rebooting everytime
  • base image and package images are built on FAIR infrastructure
• service management
  • has to start package images on machines
  • provide a UI to stop/start/change version/clear caches/... of the service or system
  • cgroups, maybe via libsystemd?
  • manage service settings(e.g. configured flowgraphs/dashboards/names), find systems that deviate from default config, provide way to persist current settings into settings repository
  • use RBAC for authenticating users before performing changes
• Monitoring
  • Single place monitoring of 200 systems
  • Expose a generic opencmw worker for generic service information (cpu/mem/temp)
  • Adapter to feed these signals into time-series DB (in addition to using them in dashboards like digitizer information)
  • Grafana for generic service monitoring dashboards

6. GNU Radio

• How to implement dynamic number of ports for nodes

  • needed for e.g. selector block or variable input adder

    struct selector : ... {
      in_port<I> in;
      out_port<O> out;
      vector<in_port<T>> ins;
      vector<out_port<T>> outs;
    
      process_bulk(in, ins, out, outs)
    };
    

  • See issue #69 for a more detailed discussion.

• Handling messages asynchronously with message ports

  • separate from regular ports, not included in available samples calculation
  • processed by work function, if messages are available call process_message before data processing
  • processing is asynchronous with respect to stream data
  • denoted by enum value Message...

• Flowgraphs with loops and feedback delays

  • detecting loops is simple with basic graph algorithms
  • to run graphs with feedback, samples need to be added into the loop
  • explicit
    • manually add delay block
    • error for loops without delay
    • runtime overhead of additional buffer (could be optimised away by scheduler)
  • implicit
    • extra sample(s) get added at initialisation
  • handle loops with blocks with min_samples > 1?

• Actions

  • @wirew0rm: Create issue regarding ports. -> #148
  • @ivan-cukic: Create issue for message passing and cyclic elements to ask for input from GR architecture audience

7. Code style and formating

While we could decide on formatting ourselves, we should invite a larger audience that will work with and use GR. Apart from formatting, naming and naming styles (CamelCase versus snake_case) should be dicussed.

The discussion should be controlled and prepared for. We need to review all the different clang-format settings,
group them if they are related not to allow voting results that end up choosing illogical combinations of some related values.

Voting should be SF/WF/N/WA/SA. We should discuss voting for no-yes-no parameters.

We need to be able to review settings on real code-base (graph-prototype) and not only rely on the examples in clang-format documentation.

See also some more discussion in [PR #132](fair-acc/gnuradio4#132.

8. Wrap up and AOB

Timeline:

• MVP before September:
  • DNS
  • Graph-prototype in UI (for the UX workshop)
• By end of October a bit breadth
  • writing modules.
• To be able to have hand-rolled out systems in November.

Priorities:

• DNS UI
• signal selection
• GP+UI integration.