component loader) modern equivalent to require/amd

[avickers]

So Chrome and Safari already support dynamic imports, i.e. import('/lib/knockout-latest.js').

It can be polyfilled.

It's basically AMD without requirejs, and it lets you asynchronously import es6 modules dynamically.

Would anyone like to see an update of
ko.components.register('home-page', { require: 'components/home-page/home' });

To something like
ko.components.register('home-page', { import: 'components/home-page/home' });

Using the fetch API to get the template and dynamic import() to load the view model?

Does anyone still use the require style registration? Would the community like a solution that rids it of the requirejs dependency?

[brianmhunt]

Thanks @avickers

I think this is a good idea, and should be straightforward to implement with dynamic imports.

[avickers]

@brianmhunt, I was going to submit a pull request for this, but gave up after recursive, nested callback functions spread across multiple modules hurt my brain. It reminded me of why programmers used to need multi-monitor setups.

The component loader code is legacy code from Knockout 3.x. Did you want to keep as much of the legacy code unchanged as possible or would you be OK with it being refactored to use promises or async function?

[brianmhunt]

@avickers Thanks for giving it a whirl.

I've tried to refactor the legacy code, but it was a bit convoluted because a Component may be synchronous or asynchronous, which adds oodles of complexity.

So that's something to keep in mind.

You might also want to work off the #54 branch b/c that's where I'm making changes to ensure backwards-compatibility right now, and some changes affect Components.

Let me know if you have any issues/questions.

[avickers]

I think Knockout 4/TKO should drop support for synchronously loading components entirely:

• Even when sync is set to true, there is no guarantee that the component will load synchronously;
• the Javascript standard has now fully embraced the default behavior as best practices and provides first class support (raison d'etre of the issue);
• synchronous loading of components feels like realized technical debt to me, and I'd rather try to rework the library to better handle async code;
• except that, as near as I can tell, KO 3.4 (microtasks) and 3.5 (descendantsComplete) have fully paid down that technical debt, and this is a deprecated feature that exists for no reason other than backwards compat.

I realize that the stability of the API is one of the most prized features of Knockout, but at the same time, major version changes are rare opportunities to improve the usability and maintainability of libraries. I feel like synchronous loading of components should be flagged as deprecated in 3.5 and removed in 4. Unless there is something that I am overlooking?

cc: @mbest

[brianmhunt]

@avickers I agree, subject to input from others.

What are your thoughts @mbest?

[mbest]

I also agree. I'll add a note to mark it as deprecated.

[brianmhunt]

The one concern I have is the “async flicker”; I want to find the component-sync issues and discussions (on mobile phone right now), just to be sure we aren’t overlooking something.

[avickers]

• Knockout 3.4 breaks component partial loading knockout#1965

Is this the discussion?

[brianmhunt]

Here: knockout/knockout#1504 . The issues with async + DOM are:

1. janky loading
2. layout thrashing (CPU usage, UX lag)

So perhaps we might want two separate components, one that's always-async and one that's always-sync. There's some code re-use, but most of the complexity comes from "sometimes async".

In any case, we want to be mindful that there was a lot of demand for synchronous components, and this might be a significant regression for a number of use cases.

Thoughts?

[avickers]

Internet years are like dog years and 2014 was along time ago. The thread does help place the debate into context, but I do think there is a bigger issue in 2018.

Initial load. The initial load is like the job interview or the first date. It's the most important interaction because if you don't make a good impression there won't be any more. And, frankly, KO has some grey hairs showing.

Even with a pared down CSS framework, i.e. not TWBS, and minimal real world dependencies, KO components are quite likely to be 5,000ms+ to first paint and interactivity. In a world where every 100ms is precious, that's painful.

It's worth pointing out that Google is highly opinionated on this issue, and they no longer want you to prerender your SPAs for them. Their spiders evaluate the performance of your app, and if it's bad, you're going to pay for it. And they are within their rights to do so because it's very highly correlated with bounce rate, which they will further penalize for.

So, yeah, the argument for synchronous is well and good but by the time that fluid UI loads, you're going to be ranked on page 5 and anyone that still finds it will have hit back in search of more cat vids and alternative facts. I wish this was more hyperbolic than it likely is.

The initial rendering has to be faster. I think we should take a page out of the Progressive Web App play book, and support partial/progressive renders.

Allow developers to create yet another json based manifest (because we need more of those) that defines the basic structure of the app wrt critical paths. A universal critical path for things like header/nav, sidenav, footer, etc., and a critical path for every route that is anticipated as a landing page under normal circumstances. (*loader will need to suppress warnings for components in more than one critical path.)

Each component in a critical path will have a stub template, in a separate array, that should generally be little more than an outer div with critical classes/styling, e.g. <navbar class="navbar col-12 teal text-pink"></navbar>, and absolutely no bindings. That way, knockout can begin to immediately and synchronously layout the skeleton of the app. (Maybe a few static assets like logo and some basic links?)

Then, knockout will look for a bundle with the name of the critical path, and attempt to load and register all the components therein, expecting them to be exported as the new component classes.

Then, knockout will lazily load any other modules on the page or in the code. For lazily loaded modules, maybe it's worth looking into a hook for a spinner, progress bar, or quotation/tip cards, etc..

The build process should generate bundles predicated upon the manifest, and probably in-line the manifest itself into the entry js file.

I actually have more thoughts, but I feel like this is long enough. I'd just like to reiterate that I think that the future of the front end is more asynchronous, more modular, and an experience that rivals native apps. I think the discussion should really be about how to realize that. The partial rendering strategy is pretty much what Google expects, and gives some of the best of both worlds if well conceived and implemented.

None of this would be required, of course. If TKO does include a router, starting the app could be as easy as ko.defaultRouter( manifest ), but since KO isn't terribly opinionated an example could be given of how to configure a custom router and/or work with a service worker.

IDK. It's just a thought. I do feel like progressive rendering is the most viable strategy for balancing the myriad of performance issues and user expectations.

[caseyWebb]

Allow developers to create yet another json based manifest (because we need more of those) that defines the basic structure of the app wrt critical paths. A universal critical path for things like header/nav, sidenav, footer, etc., and a critical path for every route that is anticipated as a landing page under normal circumstances. (*loader will need to suppress warnings for components in more than one critical path.)

Each component in a critical path will have a stub template, in a separate array, that should generally be little more than an outer div with critical classes/styling, e.g. , and absolutely no bindings. That way, knockout can begin to immediately and synchronously layout the skeleton of the app. (Maybe a few static assets like logo and some basic links?)

Then, knockout will look for a bundle with the name of the critical path, and attempt to load and register all the components therein, expecting them to be exported as the new component classes.

Then, knockout will lazily load any other modules on the page or in the code. For lazily loaded modules, maybe it's worth looking into a hook for a spinner, progress bar, or quotation/tip cards, etc..

The build process should generate bundles predicated upon the manifest, and probably in-line the manifest itself into the entry js file.

This can be done today with webpack and a component loader; it's essentially what we do.

In the directory that has our components, we have the following...

import * as ko from 'knockout'
import { ComponentLoader } from 'lib/utils/component-loader'

import './app-nav'
import './app-error'
import './app-footer'
import './beacon'
import './breadcrumbs'

const context = require.context('./', true, /\.\/[^/_]+\/index\.(j|t)s$/, 'lazy')

ko.components.loaders.unshift(new ComponentLoader(context))

Those imports up top are to what you refer to as critical components, i.e. the first things that should be on the page, ASAP. They are included in the entry bundle that webpack produces.

Below that, we create a lazy webpack "require context" which is passed to this loader:

export class ComponentLoader implements KnockoutComponentTypes.Loader {
  private manifest = new Map<string, () => Promise<KnockoutComponentTypes.ComponentConfig>>()

  /**
   * @param context require.context() return value
   */
  constructor(context: RequireContext) {
    context
      .keys()
      .map((k) => ({
        path: k,
        name: k.match(/[\\\/]([^\\\/]+)/)[1]
      }))
      .forEach(({ name, path }) => {
        // see http://knockoutjs.com/documentation/component-custom-elements.html#registering-custom-elements
        ko.components.register(name, {})

        this.manifest.set(name, () => context(path))
      })
  }

  public getConfig(name: string, cb: (config: KnockoutComponentTypes.ComponentConfig) => void) {
    if (!this.manifest.has(name)) return cb(null)
    this.manifest.get(name)().then((config: any) => {
      cb({ ...config, synchronous: true })
    })
  }
}

This loader handles fetching the lazy-loaded chunks (produced by webpack thanks to the require.context call), and loads them on-demand. It would be fairly trivial to change this loader to return a wrapper template and viewModel from getConfig immediately and handle fetching the component and initializing it "out-of-band". For example, you could return a simple template like <span data-bind="if: ready"><div data-bind="component: { name: componentName, params }'"></span>, and a viewModel that fetches the component, registers it directly with a GUID, sets that guid to componentName, and proxys tha params down though. Hopefully that makes some sense, but if not this is what I'm thinking (note, this snippet isn't in any way shape or form tested, it's more thinking out loud)

export class ComponentLoader implements KnockoutComponentTypes.Loader {
  private manifest = new Map<string, () => Promise<KnockoutComponentTypes.ComponentConfig>>()

  /**
   * @param context require.context() return value
   */
  constructor(context: RequireContext) {
    context
      .keys()
      .map((k) => ({
        path: k,
        name: k.match(/[\\\/]([^\\\/]+)/)[1]
      }))
      .forEach(({ name, path }) => {
        // see http://knockoutjs.com/documentation/component-custom-elements.html#registering-custom-elements
        ko.components.register(name, {})

        this.manifest.set(name, () => context(path))
      })
  }

  public getConfig(name: string, cb: (config: KnockoutComponentTypes.ComponentConfig) => void) {
    if (!this.manifest.has(name)) return cb(null)

    const ready = ko.observable(false)
    const componentName = 'some-random-component-name'

    cb({
      template: `<span data-bind="if: ready"><div data-bind="component: { name: componentName, params: params }"></div></span>`,
      viewModel: {
        createViewModel(params: any) {
          return {
            ready,
            params,
            componentName
          }
        }
      }
    })

    this.manifest.get(name)().then((config: any) => {
      ko.components.register(componentName, config)
      ready(true)
    })
  }
}

This is similar to how I implement HMR in my project:
https://medium.com/@notCaseyWebb/knockoutjs-hot-component-replacement-e1535768ac3f

Also, you mentioned a router... The router in https://github.com/Profiscience/knockout-contrib (formerly https://github.com/Profiscience/ko-component-router) is designed with this sort of UX in mind ;)

[avickers]

Thanks, @caseyWebb, that is helpful.

I have been thinking about the a/sync component thing for awhile.

I think that there may be a cleaner solution. Allow components to load asynchronously, and create a new, abstraction layer/class between components and paints--call it the Virtual Dom, Heartbeat, Mediator, or whatever.

When a component is created, instead of just having 'name' and 'params', it can also have a declared Virtual Dom. That VD (ugh) will collect all the virtual elements from all of the child components assigned to it and do the React diff thing to control intelligent rendering with a default--and presumably adjustable--"rateLimit." Components created without being declaratively assigned to a VD/Med/HB will default to their own and a rateLimit of 0.

Potentially, the VD could provide a parent view model for all child components, such that all shared states could be declared in that view model and changes to all children rendered in one paint. (The Mediator pattern. Differs from postbox because it thinks about dom flow rather than just sharing state across VMs.)

This would also provide an opening to address #77 because the VD/Med/HB could wait until all components had signaled they were ready. Each component class could, by default, declare itself ready, but developers could override this and return a promise. This should accomplish the desired experience from synchronous module loading in a more consistent and, imo, conceptually straightforward way. Each VD is essentially a bundle of components loosely coupled together to serve a particular view.

[rposener]

Not 100% certain I followed all that @avickers. If I'm understanding - would there be a way to do the same by just promises, not a rateLimit function - that way everything can be done at exactly the right time, instead of leaving a crack for janking to re-appear, which it feels like a 'rateLimited' functionality could cause. Not sure I'm following all the benefit of the VD bits - maybe that's somehow solving it...

If the benefit of the V-Dom thing is to build a bunch of stuff that then paints once after the loops and all iterations are completed - why not just allow components to be composed of child components, which could then require resolution of their promise, until the top level component item is ready and then only apply/paint the V-Dom at once? Maybe tko can just create/hold a V-Dom at the top-level binding for each component, and child components participate in the V-Dom creation/managment similar to context/child-context?

Maybe I just don't know the inner workings well enough?

[avickers]

Well, what I'm proposing is essentially borrowing some of the tools and techniques of React.

Changes would manifest first in the virtual dom. Then you figure out what nodes are currently in the viewport. Then you run a diff function to figure out the minimal update needed. This means responding to both changes in the aggregate view model and events like scroll position, resize, etc.

This should really help Knockout perform better in those framework comparison tools that all check edge cases like "adding" and "removing" 10k or 100k nodes. That is relatively painless in a virtual dom and very painful in the actual dom.

Components with child components would probably work as well, as long as it's clear that the top-level should be whatever constitutes a particular view in the application. Now that I think about it, if an additional class were to be created, it should probably just be called View.

[bradwbradw]

@avickers why close? Is this resolved, or not needed anymore? I would love to see an example for how to use dynamic import() to load ViewModels in knockout

[brianmhunt]

@bradwbradw this is something I'd like to do, too (eventually, as time permits). Feel free to open a new issue noting this.