Umbrella: feedback building GitHub supports tools

[oliviertassinari]

This is a list of pain points that I faced while building for https://www.notion.so/mui-org/Set-GitHub-scope-labels-12a84fdf50e44595afc55343dac00fca. I thought that it might be interesting for the rest of the team to get a perspective on this. I have seen a lot of things that we could improve, but it doesn't mean that we should. I mean for a lot of them we need to consider when or even if it's something to fix 😁.

The apps:

1. https://master--toolpad.mui.com/_toolpad/app/cl6rqzry10009arlv9sto6qja/pages/ip23ggo
2. https://master--toolpad.mui.com/_toolpad/app/cl6rqzry10009arlv9sto6qja/pages/e943n9y

The pain points:

1. Changing query name reset all the settings  #1157
2. I have exhausted my GitHub with 5,000 requests/hour in almost no time. I'm not sure how. One of my assumptions is because of this option:

I have now disabled it to see if it's the origin of the problem.

3. Support entities #385 The GitHub API returns a total number of results that can be e.g. 124 but would only return pages of 30 rows. This can be quite confusing in the UI:

4. Support more data types in data grid #827 I'm using the data grid with cells that are links. It would be great if I could click them:

Similar to how it works with Metabase or Notion.

5. Support custom column type in data grid #1174 The point 4. made me wonder if it wouldn't be great to support custom column type, where you could create custom components and use them there. For example, to render https://mui-plus.vercel.app/components/Sparkline inside the cells.
6. I have added a quick description below the header. Having Support Markdown component #756 would help.
7. The default size of the button feels quite big. I'm missing a size option.
8. Support duplication of queries #1172 I had two create almost two identical queries. I wish I could have the option to duplicate them.
9. Typography component's default variant of body1 doesn't appear same in the UI #1175 When you drop a Typography component in a page, it has a default variant of body1 but the UI doesn't reflect it:

10. The UX to add and remove columns is frustrating. I can't reorder columns. I can't delete all the columns at once. If I spam click on the delete button, the dialog moves each time I delete a column, sometimes it opens the default view. I have tried Retool tool, but it's super painful with them.
11. Undo/Redo action #226 I had a couple of cases where I wanted to undo my change, but couldn't.

At the end of the day I could solve my ops problem with Toolpad, it matters more :). I have tried a bit to recreate this on Retool, but I'm not convinced their DX is better.

[Janpot]

1. 👍
2. We should set a staleTime in react-query in the runtime. I think it's by default 0 and it makes the page constantly revalidate queries. Additionally, we could investigate whether we can integrate their devtools in the toolpad editor. I believe we can get rid of the "refetch on window focus" option. Add query cache control #1093
3. We're currently missing an abstraction to represent an abstract collection. See Support entities #385. We could try to get a bit more mileage out of the queries before we dive into that.
4. I believe it's a matter of adding more column types. See Support more data types in data grid #827. Cataloging what's missing is probably step one.
5. We could create a "waiting for 👍" ticket
6. 👍 . would also be nice if we could just write inline markdown and have rows with app components in between (i.e. Toolpad/Notion mashup)
7. We could also make this part of your theme to encourage uniformity
8. 👍
9. I believe that's a bug
10. Yes, this UI always was sort of temporary, I think there's an opportunity to figure out a good UX from the ground up. FYI, you can also reorder the columns on the grid itself, but I agree that the columns definition interface should support this
11. Undo/Redo action #226

I have tried a bit to recreate this on Retool, but I'm not convinced their DX is better.

But is Toolpad's better? or equally meh?

[oliviertassinari]

But is Toolpad's better? or equally meh?

I'm used to how Toolpad works and not soo much with Retool, so it was definitely better on Toolpad for me. For example, it took me a while to figure out how to write a REST fetch query in Retool, and then their UI was way too small to preview the result,s and then I couldn't get a live preview of how I was binding my props, eww.

Compared to pro-code, since the app is really simple, the value I got is:

1. not having to care about a bunch of things: hosting, git, prettier, updating dependencies
2. learning new APIs/spending time on the docs of them to copy & paste demos (e.g. react-query, CSS, data grid)

Regarding the direct friction points with the tool itself, I listed them above. This also got me some exposure to what could be bold changes that could make this experience a lot better. I didn't see anything obvious, the only thing for me is this notion of deeper integration with pro-code workflows, with emphasis on the latter word. Sometimes having text to describe your state is nicer than something visual. I wouldn't be surprised if https://nhost.io/, https://supabase.com/, https://appwrite.io/ aren't facing this exact same challenge of visual vs. text configuration: to see how they cope with it! And maybe copy what's great.

[Janpot]

Sometimes having text to describe your state is nicer than something visual.

I believe this is one of the primary hurdles to make pro-code developers use low-code tools. I think it goes slighty further than that, for textual representation to be useful I need it to integrate frictionless with my other tools (code editors, source control, linters,...).

I wonder whether we rather should've started by creating a declarative configuration file format to build internal apps/admin panels, with a CLI that generates project files from this configuration, and build Toolpad as a GUI that manipulates the configuration files through drag&drop.

[oliviertassinari]

@Janpot Actually, I have opened a support ticket to Notion for this exact same challenge:

In wikis it is impossible to make proposals that touch multiple parts of a page and/or multiple pages.

https://about.gitlab.com/handbook/handbook-usage/#wiki-handbooks-dont-scale (GitLab even have a 5 minutes video about this challenge). I have asked Notion an option to git sync some of the pages, I want to be able to cherry-pick the best model based on the nature of the content 😁.

[oliviertassinari]

2. Oh wow, this one had an unintended consequence, all the Netlify deploys are broken now 😆

Error: {"message":"API rate limit exceeded for user ID 3165635.","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}

Source. That ID, it's me https://avatars.githubusercontent.com/u/3165635. I'm creating a new token with another account for Netlify.

[Janpot]

I'm opening #1167 to deal with this short term

[prakhargupta1]

I wonder whether we rather should've started by creating a declarative configuration file format to build internal apps/admin panels, with a CLI that generates project files from this configuration, and build Toolpad as a GUI that manipulates the configuration files through drag&drop.

Looks like this discussion. May we can discuss it this week.

[prakhargupta1]

3. We're currently missing an abstraction to represent an abstract collection. See Support entities #385. We could try to get a bit more mileage out of the queries before we dive into that.

This can help me here. The API is returning more than 500 records. 500 being the limit of 1 hit, to get the second batch I have to set an offset=500. If the entire data could come in a collection then doing further operations would be easy.

[oliviertassinari]

This issue's description now has almost all the pain points linked to standalone GitHub issues 👌.

I think that we can close this issue once we solve the GitHub API rate limit problem. It's probably the most important of all of the items, the app is usable without.

[oliviertassinari]

the GitHub API rate limit problem.

@Janpot I wonder if the origin isn't different, while I was building this app, I was also trying https://airbyte.com/ to sync all the data of our GitHub repositories to a database we could run SQL queries. This might have been very API request intensive.

[oliviertassinari]

I'm closing for now, I need to figure out what's at the origin of my GitHub account API calls being rate limited. I have revoked all my API keys, to start from a clean state and be able to see correlations.

[oliviertassinari]

It looks like my issue with GitHub was Notion, not Toolpad. What the support told me: