Skip to content

Commit

Permalink
Entities can be dragged from the blueprint tree and streams tree to a…
Browse files Browse the repository at this point in the history 
…n existing view in the viewport (#8431)

### Related

* Closes #8266
* Related to #8267 

### What

This PR makes it possible to drag one or more entities from the
blueprint and streams tree to existing views in the viewport. This
involves introducing and/or adjusting a whole bunch of semantics around
selection and dragging.

#### `DragAndDropFeedback`

This PR introduced the notion of drag-and-drop feedback from the hovered
ui element.

Feedback may be either of:
- **Ignore** (default): hovered element is uninterested in the type of
drag payload, or in any payload at all, or is outright oblivious to all
that drag-and-drop stuff.
- **Reject**: hovered element is compatible with the _type_ of drag
payload, but not its actual _content_. For example, a view might already
contains the dragged entities.
- **Accept**: hovered element is compatible with both the type and the
content of the payload.

A drop should only ever happen in the latter case.

#### Payload visualisation

The payload pill display is now adjusted based on the feedback, both its
opacity (ranging from 50 to 80%) and its colour (grey or blue). The
mouse cursor is also adjusted based on the feedback.


#### Drop container visualisation

This PR slightly adjust the look of the drop container visualisation:
the blue frame is now 2px wide.

Note that the drop container is _not_ necessarily the thing that's
hovered by the mouse, see e.g. containers in the blueprint.


#### Selection handling during drag-and-drop

This PR slightly alter the current behaviour. Now:

- dragging a selected item drags the entire selection
- dragging an unselected item with `cmd` held adds that item to the
selection, and drags the entire selection
- dragging an unselected item drags that item, _without changing the
selection_ (new)


#### Entity-to-viewport-view drag semantics

This is the original goal of this PR.

- An existing view will accept a payload containing entities **IFF** any
of these entities—or their children—is both visualisable and not already
contained in that view.
- An existing view will reject a payload containing entities **IFF** all
of these entities are either non-visualisable or already contained.
- An existing view will ignore a payload containing anything else.
- When a drop succeeds:
- The view will add an inclusive ("…/**") rule for each of the dropped
entities that are both visualisable and not already included.
  - The view becomes selected.

Emphasis on that last point. This subtle UX behaviour (courtesy of
@gavrelina) makes the drop success and impact on the entity path filter
more explicit.


#### Theory of operation for drag-and-drop

With this PR, a "framework" slowly starts to emerge. For now, it's
mainly this bit of documentation:

```rust
//! ## Theory of operation
//!
//! ### Setup
//!
//! A [`DragAndDropManager`] should be created at the start of the frame and made available to the
//! entire UI code.
//!
//!
//! ### Initiating a drag
//!
//! Any UI representation of an [`crate::Item`] may initiate a drag.
//! [`crate::ViewerContext::handle_select_hover_drag_interactions`] will handle that automatically
//! when passed `true` for its `draggable` argument.
//!
//!
//! ### Reacting to a drag and accepting a drop
//!
//! This part of the process is more involved and typically includes the following steps:
//!
//! 1. When hovered, the receiving UI element should check for a compatible payload using
//!    [`egui::DragAndDrop::payload`] and matching one or more variants of the returned
//!    [`DragAndDropPayload`], if any.
//!
//! 2. If an acceptable payload type is being dragged, the UI element should provide appropriate
//!    visual feedback. This includes:
//!    - Calling [`DragAndDropManager::set_feedback`] with the appropriate feedback.
//!    - Drawing a frame around the target container with
//!      [`re_ui::DesignToken::drop_target_container_stroke`].
//!    - Optionally provide more feedback, e.g., where exactly the payload will be inserted within
//!      the container.
//!
//! 3. If the mouse is released (using [`egui::PointerState::any_released`]), the payload must be
//!    actually transferred to the container and [`egui::DragAndDrop::clear_payload`] must be
//!    called.
```

### TODO

- [x] release checklist to check the above semantics


https://github.com/user-attachments/assets/047c0d41-fead-424a-b673-b6cb1479d1fa
  • Loading branch information
@abey79
abey79 authored Dec 17, 2024
1 parent 952e397 commit 02ae9c5
Show file tree
Hide file tree
Showing 19 changed files with 788 additions and 299 deletions.
1 change: 1 addition & 0 deletions Cargo.lock
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6862,6 +6862,7 @@ dependencies = [
"bit-vec",
"bitflags 2.6.0",
"bytemuck",
"crossbeam",
"directories",
"egui",
"egui-wgpu",
Expand Down
24 changes: 19 additions & 5 deletions crates/viewer/re_blueprint_tree/src/blueprint_tree.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use re_types::blueprint::components::Visible;
use re_ui::{drag_and_drop::DropTarget, list_item, ContextExt as _, DesignTokens, UiExt as _};
use re_viewer_context::{
contents_name_style, icon_for_container_kind, CollapseScope, Contents, DataResultNodeOrPath,
DragAndDropPayload, SystemCommandSender,
DragAndDropFeedback, DragAndDropPayload, SystemCommandSender,
};
use re_viewer_context::{
ContainerId, DataQueryResult, DataResultNode, HoverHighlight, Item, ViewId, ViewerContext,
Expand Down Expand Up @@ -101,6 +101,7 @@ impl BlueprintTree {

// handle drag and drop interaction on empty space
self.handle_empty_space_drag_and_drop_interaction(
ctx,
viewport,
ui,
empty_space_response.rect,
Expand Down Expand Up @@ -192,6 +193,7 @@ impl BlueprintTree {
ctx.handle_select_hover_drag_interactions(&item_response, item, true);

self.handle_root_container_drag_and_drop_interaction(
ctx,
viewport,
ui,
Contents::Container(container_id),
Expand Down Expand Up @@ -275,6 +277,7 @@ impl BlueprintTree {
viewport.set_content_visibility(ctx, &content, visible);

self.handle_drag_and_drop_interaction(
ctx,
viewport,
ui,
content,
Expand Down Expand Up @@ -411,6 +414,7 @@ impl BlueprintTree {

viewport.set_content_visibility(ctx, &content, visible);
self.handle_drag_and_drop_interaction(
ctx,
viewport,
ui,
content,
Expand Down Expand Up @@ -629,6 +633,7 @@ impl BlueprintTree {

fn handle_root_container_drag_and_drop_interaction(
&mut self,
ctx: &ViewerContext<'_>,
viewport: &ViewportBlueprint,
ui: &egui::Ui,
contents: Contents,
Expand Down Expand Up @@ -672,12 +677,13 @@ impl BlueprintTree {
);

if let Some(drop_target) = drop_target {
self.handle_contents_drop_target(viewport, ui, dragged_contents, &drop_target);
self.handle_contents_drop_target(ctx, viewport, ui, dragged_contents, &drop_target);
}
}

fn handle_drag_and_drop_interaction(
&mut self,
ctx: &ViewerContext<'_>,
viewport: &ViewportBlueprint,
ui: &egui::Ui,
contents: Contents,
Expand Down Expand Up @@ -751,12 +757,13 @@ impl BlueprintTree {
);

if let Some(drop_target) = drop_target {
self.handle_contents_drop_target(viewport, ui, dragged_contents, &drop_target);
self.handle_contents_drop_target(ctx, viewport, ui, dragged_contents, &drop_target);
}
}

fn handle_empty_space_drag_and_drop_interaction(
&mut self,
ctx: &ViewerContext<'_>,
viewport: &ViewportBlueprint,
ui: &egui::Ui,
empty_space: egui::Rect,
Expand Down Expand Up @@ -792,12 +799,13 @@ impl BlueprintTree {
usize::MAX,
);

self.handle_contents_drop_target(viewport, ui, dragged_contents, &drop_target);
self.handle_contents_drop_target(ctx, viewport, ui, dragged_contents, &drop_target);
}
}

fn handle_contents_drop_target(
&mut self,
ctx: &ViewerContext<'_>,
viewport: &ViewportBlueprint,
ui: &Ui,
dragged_contents: &[Contents],
Expand All @@ -816,6 +824,8 @@ impl BlueprintTree {
false
};
if dragged_contents.iter().any(parent_contains_dragged_content) {
ctx.drag_and_drop_manager
.set_feedback(DragAndDropFeedback::Reject);
return;
}

Expand All @@ -826,7 +836,9 @@ impl BlueprintTree {
);

let Contents::Container(target_container_id) = drop_target.target_parent_id else {
// this shouldn't append
// this shouldn't happen
ctx.drag_and_drop_manager
.set_feedback(DragAndDropFeedback::Reject);
return;
};

Expand All @@ -839,6 +851,8 @@ impl BlueprintTree {

egui::DragAndDrop::clear_payload(ui.ctx());
} else {
ctx.drag_and_drop_manager
.set_feedback(DragAndDropFeedback::Accept);
self.next_candidate_drop_parent_container_id = Some(target_container_id);
}
}
Expand Down
126 changes: 7 additions & 119 deletions crates/viewer/re_selection_panel/src/view_entity_picker.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ use re_data_ui::item_ui;
use re_entity_db::{EntityPath, EntityTree, InstancePath};
use re_log_types::{EntityPathFilter, EntityPathRule};
use re_ui::{list_item, UiExt as _};
use re_viewer_context::{DataQueryResult, ViewClassExt as _, ViewId, ViewerContext};
use re_viewport_blueprint::{ViewBlueprint, ViewportBlueprint};
use re_viewer_context::{DataQueryResult, ViewId, ViewerContext};
use re_viewport_blueprint::{
create_entity_add_info, CanAddToView, EntityAddInfo, ViewBlueprint, ViewportBlueprint,
};

/// Window for adding/removing entities from a view.
///
Expand Down Expand Up @@ -73,10 +75,9 @@ fn add_entities_ui(ctx: &ViewerContext<'_>, ui: &mut egui::Ui, view: &ViewBluepr
re_tracing::profile_function!();

let tree = &ctx.recording().tree();
// TODO(jleibs): Avoid clone
let query_result = ctx.lookup_query_result(view.id).clone();
let query_result = ctx.lookup_query_result(view.id);
let entity_path_filter = &view.contents.entity_path_filter;
let entities_add_info = create_entity_add_info(ctx, tree, view, &query_result);
let entities_add_info = create_entity_add_info(ctx, tree, view, query_result);

list_item::list_item_scope(ui, "view_entity_picker", |ui| {
add_entities_tree_ui(
Expand All @@ -85,7 +86,7 @@ fn add_entities_ui(ctx: &ViewerContext<'_>, ui: &mut egui::Ui, view: &ViewBluepr
&tree.path.to_string(),
tree,
view,
&query_result,
query_result,
entity_path_filter,
&entities_add_info,
);
Expand Down Expand Up @@ -263,116 +264,3 @@ fn add_entities_line_ui(
}
});
}

/// Describes if an entity path can be added to a view.
#[derive(Clone, PartialEq, Eq)]
enum CanAddToView {
Compatible { already_added: bool },
No { reason: String },
}

impl Default for CanAddToView {
fn default() -> Self {
Self::Compatible {
already_added: false,
}
}
}

impl CanAddToView {
/// Can be generally added but view might already have this element.
pub fn is_compatible(&self) -> bool {
match self {
Self::Compatible { .. } => true,
Self::No { .. } => false,
}
}

/// Can be added and view doesn't have it already.
pub fn is_compatible_and_missing(&self) -> bool {
self == &Self::Compatible {
already_added: false,
}
}

pub fn join(&self, other: &Self) -> Self {
match self {
Self::Compatible { already_added } => {
let already_added = if let Self::Compatible {
already_added: already_added_other,
} = other
{
*already_added && *already_added_other
} else {
*already_added
};
Self::Compatible { already_added }
}
Self::No { .. } => other.clone(),
}
}
}

#[derive(Default)]
#[allow(dead_code)]
struct EntityAddInfo {
can_add: CanAddToView,
can_add_self_or_descendant: CanAddToView,
}

fn create_entity_add_info(
ctx: &ViewerContext<'_>,
tree: &EntityTree,
view: &ViewBlueprint,
query_result: &DataQueryResult,
) -> IntMap<EntityPath, EntityAddInfo> {
let mut meta_data: IntMap<EntityPath, EntityAddInfo> = IntMap::default();

// TODO(andreas): This should be state that is already available because it's part of the view's state.
let class = view.class(ctx.view_class_registry);
let visualizable_entities = class.determine_visualizable_entities(
ctx.applicable_entities_per_visualizer,
ctx.recording(),
&ctx.view_class_registry
.new_visualizer_collection(view.class_identifier()),
&view.space_origin,
);

tree.visit_children_recursively(|entity_path| {
let can_add: CanAddToView =
if visualizable_entities.iter().any(|(_, entities)| entities.contains(entity_path)) {
CanAddToView::Compatible {
already_added: query_result.contains_entity(entity_path),
}
} else {
// TODO(#6321): This shouldn't necessarily prevent us from adding it.
CanAddToView::No {
reason: format!(
"Entity can't be displayed by any of the available visualizers in this class of view ({}).",
view.class_identifier()
),
}
};

if can_add.is_compatible() {
// Mark parents aware that there is some descendant that is compatible
let mut path = entity_path.clone();
while let Some(parent) = path.parent() {
let data = meta_data.entry(parent.clone()).or_default();
data.can_add_self_or_descendant = data.can_add_self_or_descendant.join(&can_add);
path = parent;
}
}

let can_add_self_or_descendant = can_add.clone();
meta_data.insert(
entity_path.clone(),
EntityAddInfo {
can_add,
can_add_self_or_descendant,
},
);
});

meta_data
}
3 changes: 2 additions & 1 deletion crates/viewer/re_time_panel/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,7 @@ impl TimePanel {
} = ui
.list_item()
.selected(is_selected)
.draggable(true)
.force_hovered(is_item_hovered)
.show_hierarchical_with_children(
ui,
Expand Down Expand Up @@ -726,7 +727,7 @@ impl TimePanel {
&response,
SelectionUpdateBehavior::UseSelection,
);
ctx.handle_select_hover_drag_interactions(&response, item.to_item(), false);
ctx.handle_select_hover_drag_interactions(&response, item.to_item(), true);

let is_closed = body_response.is_none();
let response_rect = response.rect;
Expand Down
11 changes: 11 additions & 0 deletions crates/viewer/re_ui/src/design_tokens.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,17 @@ impl DesignTokens {
pub fn thumbnail_background_color(&self) -> egui::Color32 {
self.color(ColorToken::gray(S250))
}

/// Stroke used to indicate that a UI element is a container that will receive a drag-and-drop
/// payload.
///
/// Sometimes this is the UI element that is being dragged over (e.g., a view receiving a new
/// entity). Sometimes this is a UI element not under the pointer, but whose content is
/// being hovered (e.g., a container in the blueprint tree)
#[inline]
pub fn drop_target_container_stroke(&self) -> egui::Stroke {
egui::Stroke::new(2.0, self.color(ColorToken::blue(S350)))
}
}

// ----------------------------------------------------------------------------
Expand Down
7 changes: 2 additions & 5 deletions crates/viewer/re_ui/src/list_item/list_item.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -384,13 +384,10 @@ impl ListItem {
let bg_rect_to_paint = ui.painter().round_rect_to_pixels(bg_rect);

if drag_target {
let stroke = crate::design_tokens().drop_target_container_stroke();
ui.painter().set(
background_frame,
Shape::rect_stroke(
bg_rect_to_paint.expand(-1.0),
0.0,
egui::Stroke::new(1.0, ui.visuals().selection.bg_fill),
),
Shape::rect_stroke(bg_rect_to_paint.shrink(stroke.width), 0.0, stroke),
);
}

Expand Down
6 changes: 5 additions & 1 deletion crates/viewer/re_ui/src/ui_ext.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1094,7 +1094,11 @@ pub trait UiExt {
return *span;
}

if node.has_visible_frame() || node.is_area_ui() || node.is_root_ui() {
if node.has_visible_frame()
|| node.is_area_ui()
|| node.is_panel_ui()
|| node.is_root_ui()
{
return (node.max_rect + node.frame().inner_margin).x_range();
}
}
Expand Down
15 changes: 8 additions & 7 deletions crates/viewer/re_viewer/src/app_state.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ use re_smart_channel::ReceiveSet;
use re_types::blueprint::components::PanelState;
use re_ui::{ContextExt as _, DesignTokens};
use re_viewer_context::{
drag_and_drop_payload_cursor_ui, AppOptions, ApplicationSelectionState, BlueprintUndoState,
CommandSender, ComponentUiRegistry, PlayState, RecordingConfig, StoreContext, StoreHub,
AppOptions, ApplicationSelectionState, BlueprintUndoState, CommandSender, ComponentUiRegistry,
DragAndDropManager, PlayState, RecordingConfig, StoreContext, StoreHub,
SystemCommandSender as _, ViewClassExt as _, ViewClassRegistry, ViewStates, ViewerContext,
};
use re_viewport::ViewportUi;
Expand Down Expand Up @@ -215,8 +215,9 @@ impl AppState {
);

// The root container cannot be dragged.
let undraggable_items =
re_viewer_context::Item::Container(viewport_ui.blueprint.root_container).into();
let drag_and_drop_manager = DragAndDropManager::new(re_viewer_context::Item::Container(
viewport_ui.blueprint.root_container,
));

let applicable_entities_per_visualizer =
view_class_registry.applicable_entities_for_visualizer_systems(&recording.store_id());
Expand Down Expand Up @@ -278,7 +279,7 @@ impl AppState {
render_ctx: Some(render_ctx),
command_sender,
focused_item,
undraggable_items: &undraggable_items,
drag_and_drop_manager: &drag_and_drop_manager,
};

// We move the time at the very start of the frame,
Expand Down Expand Up @@ -350,7 +351,7 @@ impl AppState {
render_ctx: Some(render_ctx),
command_sender,
focused_item,
undraggable_items: &undraggable_items,
drag_and_drop_manager: &drag_and_drop_manager,
};

if *show_settings_ui {
Expand Down Expand Up @@ -517,7 +518,7 @@ impl AppState {
//

add_view_or_container_modal_ui(&ctx, &viewport_ui.blueprint, ui);
drag_and_drop_payload_cursor_ui(ctx.egui_ctx);
drag_and_drop_manager.payload_cursor_ui(ctx.egui_ctx);

// Process deferred layout operations and apply updates back to blueprint:
viewport_ui.save_to_blueprint_store(&ctx, view_class_registry);
Expand Down
Loading

0 comments on commit 02ae9c5

Please sign in to comment.