[Design proposal] Chart creation flow updates

[ghost]

Hi all,

This is a follow-up post to this discussion, which proposes allowing the user to create a chart directly from database tables and SQL queries instead of only datasets.

I'm looking for feedback on UI changes to the Explore creation flow where users can select from multiple chart sources (not just datasets) to create a visualization. (Note: there are some additional UI enhancements to Explore itself – feel free to comment, but they will be addressed in a separate project).

Here's the Figma file with full annotations
Here's the interactive prototype

You can leave feedback here or directly in Figma comments by clicking on the speech bubble icon in the upper left corner and then again on the screen where you would like to leave the comment.

You can also toggle existing comments on/off with Shift+C.

Thanks for reviewing!

[suddjian]

This is very very cool. I love how much more information it gives you about the datasets. That should become standard throughout the product.

[danisola]

Completely agree! But IMO it should also display the description of the dataset and the columns.

Many organizations are using dbt to document datasets, it would be awesome if that information could be displayed within Superset instead of having to navigate to dbt docs.

[MacNale]

This is good progress. I use Grafana regularly. I love how it allows me to custom design my own plugin to query the data. I know, Grafana and Superset are different kinds of tools and used for different purpose. I just like the flow of creating the panel and dashboard. It is super easy.

[srinify]

I've been involved with the Superset project for nearly 2 years and I've been dreaming about this workflow every single week. I'm super excited that this is happening!

The strengths of this design, as I see it:

• Elevates all data sources (Saved Queries, Virtual Datasets, Physical Datasets) in Superset to first-class citizens as potential inputs to any Chart I'm creating
• Discovering and internalizing the shape of datasets is much much easier. I no longer need to have a separate window open just to keep the dataset preview / SQL Lab available while I'm crafting my Chart

Some weaknesses & improvements I'd like to see addressed at some point:

• As @danisola pointed out, it would be great to have metadata context from dbt docs, openmetadata, datahub, or another solution populated into Superset and provide EVEN MORE context to Chart builders
• There's still no support for folders or nesting. I'm hoping this is seriously considered in the V2 design, because organizations with hundreds of registered datasets are still going to struggle.
• It would be ideal if charts that aren't a clear fit are somehow hidden / visually grayed out with some nudging context. For example, if the dataset chosen doesn't have a Temporal / time column, charts that require it should be made unavailable
• It would be great if Saved Queries list view also had information on who created the query (unless saved queries are restricted to the individual only? I forget!)
• There's now some new added friction to Experimenting / Playing with Charts because they're in different views / tabs. So we need to ensure the speed of moving around is insanely quick, or we could easily frustrate Chart builders!

[ghost]

Hi all, wanted to provide an update on this effort – 

We got some great feedback over the past week and a half and have iterated on the designs quite a bit. The main concern was the tabbed navigation pattern used in the initial designs might hinder flexibility and ease of workflow. We've explored some alternative designs and would love for you to take a look (keeping in mind there's still some other Explore-related design changes lumped in here as well):

Here's the Figma file with full annotations
Here's the interactive prototype

And here's a quick video walking through the flow (not super high-res, I encourage you to check out the prototype!):

GMT20220301-021337_Recording_1750x962.mp4

Our next steps will be usability testing with users. If someone at your org would be interested in participating (and hasn't seen these designs), please reach out to me on Slack!

[MacNale]

I am on a dashboard view. I wanted to add new chart to it. Will it allow me "add new panel" flow right there in the dashboard?

From Grafana -

[michael-s-molina]

@jess-dillard The behavior of the areas that open on top of the content would be similar to Ant Design drawers?

Screen.Recording.2022-03-03.at.2.14.15.PM.mov

[ghost]

@michael-s-molina Yep, exactly.

[ktmud]

This looks great! I like how the new iteration are getting closer to SIP-34. Maybe the north star is worth pursuing after all!

One of the strength of SIP-34 and what Superset had historically suffered deficiency of was a consistent "look and feel", i.e., a brand-defining design language--which is manifested in color palette, fonts (both font family and sizes), spacing, button sizing, icon styles, etc. Not sure if we are trying to update the design language here, but it seems the new design wasn't closely following either the current style nor SIP-34. I think it's OK to update the design language if we believe SIP-34 had become outdated or doesn't fit our needs anymore but I'm curious whether we are still working towards a design language "end state" like SIP-34.

[ghost]

Hi @ktmud – we're in a bit of a hybrid state between the new and the old as we work towards curating a holistic design system both in Figma and on the front-end. In regards to styling, here's our current approach:

Font – We are using the Inter font family and sizes, consistent with SIP-34
Icons – We are using Ant-D icons wherever possible for consistency and custom icons provided by Cartel where Ant-D does not have an appropriate icon
Color palette – We are trying to stick with the old color palette for now until we can do a holistic migration of colors, as currently the application uses a mix of custom design system and Ant-D colors
Components – We are using current-state components until we have a migration plan for new components, as this would introduce a lot of inconsistency in the UI

Is there anything in particular you're seeing that seems off?

[ktmud]

Sorry for nitpicking but I'm seeing some inconsistencies at least in font sizes and component spacing:

• There are 6 different font sizes and styles in a small vertical strip of space, which is probably fine by itself, but the visual hierarchy of them isn't very clear. For example, I'm not sure we'd want the tabs to look bigger than the global main nav.
• The two search inputs have different height and border radiuses. Search icons and placeholder text are also aligned differently.
• The status pills are in different sizes.

I also didn't see the the big icon cards style in Cartel and was wondering whether this really is a pattern we want to use.

It's understandable because Superset has never had a full suite of standardized components but this big redesign seems to be a good opportunity to invest more in building such a component library---ideally we should have a Figma component library corresponding to all the existing UI components we can use in the frontend then gradually update/replace them towards Cartel or another end state. I know this design is still in early stage, but currently I'm struggling to imagine how will this be implemented. It's not clear to me how strict do we intend to follow the design mockup given its refreshed look comparing to current design and divergences from Cartel---should we just ignore the inconsistencies and reuse existing styles as much as possible or should we tweak existing components to match---risking breaking the look of other parts of Superset and moving further away from Cartel?

Don't get me wrong, I like the direction where this is going but also wanted to caution on the implications on the actual future implementation.

[ghost]

Ah, a few caveats I may have mentioned further up the thread:

1. The Explore changes included here are actually part of another, ongoing project – part of it is posted here.
2. The UI generally definitely is not finalized (though it is pretty close). I'll take your feedback into account when finalizing the UI, thanks for going through it.

The design team at Preset is working on exactly the Figma component library you mention, with aspirations to get it all into Storybook. This is a huge undertaking that unfortunately gets deprioritized to feature design work, but we're hoping to have it in good shape in the coming months. I think for now we'll go with the "ignore inconsistencies and reuse existing styles as much as possible" approach –otherwise the scope of this project will expand largely as we also try to tackle design system issues.

[ktmud]

Thank you for the additional context. Glad we are working on that!