UX: Mobile editing #149
Comments
How to play with samples in mobile SafariTo use a physical device like iPhone or iPad with live console/elements debugging and to get rid of caching issues:
Note: You can install XCode to use device simulator instead but it's not as smooth and "tangible". |
|
Wow, amazing research! |
|
I am in general fond of the idea that revolve around taking the majority of the screen. Coming close to the real estate that message/e-mail clients take. This has its draw-backs in context, but generally provides for a more enclosed experience. I am a bit confused by the examples, which by their resolution diminish the impact screen size on the UX - which is immense. The scalable in–place toolbar mock would be close of almost the same as the native experience, you could likely review this by just going with the default behaviours finding/reading on experiences. Where would the WYISIWYG options show up? |
|
After some brief fun with Android Studio simulator I found out that what Google Chrome does is much more predictable from WYSIWYG point of view, when it comes to mocking a fullscreen editor.
It simply shrinks the viewport to the available size when software keyboard is activated and hence preserves positioning, which is a good news for the editor because the toolbar stays where it was and there's no additional scrolling.
@Bojhan: It's not clear at the moment. My initial idea was that the toolbar should stay sticky at the top of the screen while user navigates the content in the fullscreen editor. Something like a native app, but as I said, it was a draft. As for the size, complexity, contents and behaviour of the toolbar (or "WYSIWYG options"), these topics should be discussed when all technical restrictions are clear because we already know that some solutions (like We may even end up with a single button in the corner of the screen which would then open a dialog–like toolbox, it's hard to tell. We're open to suggestions and ideas at this stage but let's explore Android and Windows Phone first to know what is possible and what is not. |
|
First of all, thanks for the research and all the pics :). I can now better understand the problems that you had. The idea of using fullscreen API or at least mocking it seemed great, so it's really sad that none of this is possible. As far as I understood you haven't been able to achieve any results which would be stable and wouldn't require ugly hacks. Do you have more ideas about it? From the solutions I checked I liked this the most: https://github.com/ckeditor/ckeditor5-design/tree/mobile-editing/sample/incontent-scalable-sticky-toolbar because it's less hacky, but of course One note here: I still think that we can ignore the case when someone zoomed in. We should assume that the website is styled correctly so it's mobile friendly. On such a website people should not need to zoom much. So even if something goes beyond the viewport, it's ok.
Remember about the native balloon. It's going to collide with your toolbar, so I don't think that this solution could work. |
|
One idea came to my mind today after seeing http://app.ft.com. I noticed that it overrides native scroll in iOS and I was curious why. It struck me that this gives you a more control over the viewport. Unfortunately, I noticed that because it didn't work very well and it was actually a bit irritating... :( I asked one of the devs why is the native viewport overriden and it turns out that to make it possible to control swiping. That's actually another idea that came to my mind recently – that the toolbar could be a swipe'able panel... I'm not sure how we could ensure its discoverability, but perhaps it's just a matter of some initial hint to the user (as apps often show). Unfortunately, to be able to detect swipes we need to override scroll, which looks bad :(. See the thread on Twitter: https://twitter.com/reinmarpl/status/745671568265711616 |
The "Scalable fullscreen interface mock" is stable and not quite hacky as it looks. It uses v4 Fullscreen plugin strategy to ensure the content of the webpage will not scroll while the editor is fullscreen and then uses The problem with this solution (any solution in iOS) that users will face double scrolling when zoomed and/or using soft keyboard: scrolling of editable and scrolling of the "viewport". It works because Apple anticipated such case but it's not a very good UX TBH. It requires a longer tap&swipe to scroll the viewport and reach the toolbar or the bottom of the editable. It's not very intuitive and quite frustrating if you don't know about it because it's not so easy to discover. And considering that some button in the toolbar would be the only way to exit the fullscreen mode of the editor, it's a terrible trap for the users because they're stuck and they cannot go back to the webpage since the button and the toolbar are out of reach (out of sight).
It looks like once soft keyboard appears all The problem with JS is that the whole transition to the double-scrolling mode when soft keyboard appears is seamless to the DOM. I checked a number attributes of the
Yes, we can ignore it but people will zoom anyway. And blocking zoom with JS is out of question for many reasons. Besides, ignoring zoom doesn't solve the problem of the soft keyboard and double-scrolling. It's the same case we're talking about, after all.
Yes, that's another idea. But it feels like solving problems with RegExp: now you have twice as many problems maintaining it that you had before you decided to use it. I think that to keep it simple and maintainable CKE5 should stick to native solutions. It's not that I reject other possibilities like blocking some native behavior etc. It's that my experience tells me that it will be a mess to code, hard to test and maintain and still it will not be perfect. Oh, by the way, it looks like FT blocked pinch to zoom with JS to achieve what they have at the moment. Unfortunately it's out of question in CKE5 because:
|
:D
Yep, discoverability of this solution worries me the most. I think that we need to test it on users. Cause all of us know what to do because we've read your posts, so it's impossible to predict what real users will do when faced disappearing toolbar. |
|
What an amazingly insightful discussion! I pinged this issue to two people who work at Apple: https://twitter.com/wimleers/status/745938911717896193 — hopefully they'll provide some feedback! |
|
Great research, Olek! (Yeah, I know I am kinda late to the party :P.) I, too, agree that blocking native scrolling brings more problems than expected. I tried this approach when I was toying with fullscreen mode on iOS for CKE4. At first I thought it is a great solution but it did not ended up well. |
|
It looks like Safari 10.1 in macOS 10.12.4 and iOS 10.3 betas moves us one step closer to the mobile interface we actually need https://developer.apple.com/library/prerelease/content/releasenotes/General/WhatsNewInSafari/Articles/Safari_10_1.html
It could be a great news because it sounds like Android and iOS will finally be consistent in that matter. I'll check it out asap. |
|
Commenting on #149 (comment): At first, I hoped we're done with iOS but things turned out not so well. Safari 10.1 does fix the issue where
The box with the red outline has the following styles position: fixed;
top: 30px;
left: 30px;
right: 30px;
bottom: 30px;I'm afraid I'm slowly running out of ideas how to deal with the problem. |
|
@oleq Pardon me that I didn't read the entire thread but is adding a Codepen showing example: http://codepen.io/gijs/pen/MpLddJ |
|
@gijsroge Unfortunately, no :( The web page (DOM) is totally oblivious to the software keyboard. When it pops out, the web page becomes scrollable in the level "above DOM" (I'd call it the "app level") because In other words
It's a unique behavior of Safari in iOS and I've never seen anything similar in another browser. For example, Chrome works just like desktop browsers in that matter (screenshot to the right) and that's what we need to create a true full-screen editor. |
|
I've just stumbled upon https://quirksmode.org/presentations/Autumn2017/viewports_sofia.pdf and https://www.quirksmode.org/mobile/viewports.html. PPK is a great source of information if someone tries to understand the topic of viewports (and many other topics too, ofc :P). |
|
Cleaning up old discussions. See #186. |
|
After getting some feedback from Apple devs f2f, I finally reported the problem with the viewport mechanics officially: https://bugs.webkit.org/show_bug.cgi?id=191204. Let's hope it'll get some traction. |
Unlike CKEditor 4, the new "5" must be mobile–friendly by providing dedicated interface and editing experience to the users.
Research
There are multiple factors to consider in this issue like screen size (RWD) or input method (touch vs. click), but most importantly, it's all about confronting the UX/UI ideas with technical limitations of the most popular mobile web browsers.
Note: The following research involves Safari on iOS only so far.
Native Fullscreen API editing
The initial idea of mobile editing involved the usage of Fullscreen API. Users would click the button (or some area) on the webpage to start editing and then the editor would show up in fullscreen mode, non–scalable with a toolbar on the top of the viewport. The editable area of the editor would be scrollable when content exceeds available screen height.
This solution solves UI/UX conflict between webpage and editable content – the editing experience is similar to a standalone (native) text processor application, undisturbed by floating webpage elements not related to the editing process.
Unfortunately, the Fullscreen API is unavailable in iOS Safari and unlikely to be implemented in the nearest future. It's hard to tell why Apple falls short in this matter but it could be that a conflict of interests between fullscreen web pages and native App Store content might be the case. It could also be that since iOS devices come with a single hardware button (there's no direct "back" button), Apple is reluctant to enable Fullscreen API for all element due to UX concerns like fullscreen "hijacking", etc.
Ideally, CKEditor 5 should use an universal mobile editing solution for each environment. Still, using Fullscreen API in Android devices (research needed), and mocking it in some way in iOS is also an option.
Fullscreen mocking and alternative interfaces
Because Fullscreen API is unavailable in mobile Safari, some mocks and alternative approaches have been researched. Samples can be found in mobile-editing branch.
A quick note on viewport mechanics in mobile Safari (iOS 9+)
(Mocked) Fullscreen WYSIWYG editing in mobile Safari is quite tricky because of the soft keyboard.
As long as the keyboard stays invisible (editable blurred), the whole area of the editor is available to the user. Editable can be scrolled to reveal more content off the screen (green scrollbar).
Then the keyboard appears (editable focus), Safari compensates its height with an extra scrolling area (blue, virtual scrollbar). The problem is that the process happens on some higher "higher" level, which means that the whole web page is wrapped with some scrollable layer in Safari.
What does it mean? It means a lot. For instance it means that all the elements which have
position:fixedorabsoluterelative to the<body>will go off-screen. They're positioned to the web page, which is now scrolled as a whole within the keyboard–height–compensating Safari layer.Wanna use
position:sticky? It's sticky as long as the keyboard is not visible. Then it goes off-screen as everything else. It makes sticky toolbar and such structures pretty useless because they are not visible to the user most of the time.On the desktop it would be as if something scrolled the entire browser tab within the browser to accomodate some extra structure appearing inside. It's an application level scrolling, not web–page level.
So now the user has 2 levels of scrolling: editable and keyboard–compensating. The problem is that they don't go well together. Or maybe: they require adaptation from the user. Editable scrolls smoothly, but when the user scrolls towards its bottom, some content is covered by the keyboard. To reveal it, a slightly longer "touch&move" is required. It's quite annoying and besides... the toolbar has gone off the screen now (panic, how do I exit the editor?!).
So to go the top of the keyboard–compensating layer and reveal the toolbar, the user starts scrolling again in the opposite direction. If not familiar with the viewport mechanics, they scroll the editable instead, which is also pretty frustrating. Another longer "touch&move" is required to move the whole webpage to the top and see the toolbar. Again, not very nice UX, although it works.
So how about zooming now? Yes, it works at the same layer as keyboard–compensating scrolling and engages higher–level scrolling in both directions. It means that it's even easier for the user to get lost in the editor, both horizontally and vertically, especially when the keyboard is visible.
So maybe it's possible to compensate that funny scrolling offset which appears when keyboard gets activated or user zoomed? Nope. The web page and DOM JS interface knows nothing about what happens when the keyboard kicks in. All the geometry/scroll/etc. values remain the same as if nothing happened. It happens in some other layer.
Madness.
Non–scalable fullscreen interface mock
Samples:
fullscreen-non-scalable-*/index.htmlNote: During the research Apple announced that due to accessibility concerns Safari in iOS 10 will ignore
user-scalable=nonemeta declaration. The news has been confirmed – in iOS 10 beta1 it's impossible to block pinch to zoom using<meta>tag. In theory it could by bypassed by blockinggesture*events likebut it's not a good idea really. In practice it's a source of event more problems because the editor is still unable to enforce
1:1scale when in going fullscreen and there's no way to manually zoom out.Long story short, both
fullscreen-non-scalable-*samples in mobile-editing branch can be discarded because soon they will not correspond with the API of iOS Safari.Scalable fullscreen interface mock
Sample:
fullscreen-scalable-smooth-scroll/index.htmlAfter some tinkering, Safari will display the editor like it was a native Fullscreen API mode. The result provides a decent UX, at least for a hack. Toolbar sticks to the top of the screen most of the time, but it is prone to disappearing due to the viewport mechanics (see "A quick note about viewport mechanics in mobile Safari").
At the moment, it's the only sample that provides some decent UI/UX in mobile Safari.
Scalable fullscreen interface mock (with autogrow)
Sample:
fullscreen-scalable-autogrow/index.htmlThe idea is to disable scrolling within editable by auto–growing it as the new content arrives. It would certainly improve the UX because the users would not need to struggle with double scrolling mechanics (see "A quick note about viewport mechanics in mobile Safari").
The problem is that the trick which mocks fullscreen mode assumes that
body,htmlhavewidthandheightequal0px, so the content of the web page which does not belong to the editor does not activate another web-page layer of scrolling (then would be 3 of them, yay!). To have an editable which expands vertically, these restrictions cannot be applied and also the editor itself cannot haveposition: fixedin respect to the bottom of the web page.The result is that when the content of the web page is much longer than the content of editable, it becomes visible below the editor layer (which has
position: fixed), at the bottom of the editor. Possibly, if the height ofbodywas automatically adjusted to equal the height of autogrowing editor, this sample could be a valid solution.Scalable interface mock (with sticky toolbar)
Sample:
incontent-scalable-sticky-toolbar/index.htmlIt's an implementation of ckeditor/ckeditor5#135.
Unfortunately
position: stickydoes not work as desired when the keyboard appears (see "A quick note about viewport mechanics in mobile Safari"). So it's as good as no special solution at all.Scalable in–place toolbar mock (TODO)
Sample: TODO
Since any kind of CSS positioning does not go well with the presence of soft keyboard and user zoom, pinning the toolbar to the selection could be a solution instead. It would certainly limit the amount of space available for toolbar controls and perhaps cover some content, but it would clearly follow the user as they edit and format the content.
What's next?
The text was updated successfully, but these errors were encountered: