Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] change when upgrade/connected happens during parsing, so that it is the first event in the following microtask #787

Closed
trusktr opened this issue Jan 29, 2019 · 29 comments

Comments

@trusktr
Copy link
Contributor

trusktr commented Jan 29, 2019

At the moment, it seems that during parsing, a custom elements code can not defer to a microtask in order to observe children (and expect the children to be upgraded).

I explained this in detail here: #550 (comment)

The main issue, put shortly, is that using Promise.resolve().then() to defer to a microtask does not allow a custom element to observe their children after their children have already been upgrade. The only way is to defer to a macrotask with setTimeout, which has the unwanted side effect of causing custom element code to execute after the code of any <script> tags that are located below the custom elements in parsing order.

I'm not sure if this is the right place, but this is a feature request to change the behavior so that elements that are about to be upgraded will be upgraded in the first microtask following the connectedCallback of the current element. This way at least, Promise.resolve().then() can be used by custom elements authors to work with already-upgraded children, without having to defer code beyond the execution of end-user code which may following the custom elements in parsing order.

@domenic
Copy link
Collaborator

domenic commented Jan 29, 2019

As was explained in that issue, this is not the correct pattern to use for working with children, and we will not be changing things to accomodate broken patterns that try to inspect the children in the constructor or connectedCallback.

@domenic domenic closed this as completed Jan 29, 2019
@trusktr
Copy link
Contributor Author

trusktr commented Jan 29, 2019

Is there a pattern you'd recommend in particular for working with children?

I'm thinking about having children emit an event to tell parents they are ready (in those children's connectedCallback).

If MutationObserver would fire connected during parsing, that could help too, as I already have that in place for future connected/disconnected children.

sidenote: As @rniwa mentioned there too, I hadn't thought about the parser pausing while waiting for the network: in this case it can be paused on an opening tag of a custom element, after having upgraded it and called its connectedCallback, without knowing if markup for a possible child element will ever arrive.

@domenic
Copy link
Collaborator

domenic commented Jan 29, 2019

A MutationObserver watching for childList changes is the approach I would recommend. No built-in elements with child/parent relationships care about the connectedness of their children (as opposed to just looking at the connectedness of the parent), so I don't have a framework to advice you on the design you seem to be going for where that does matter for some reason.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 29, 2019

A MutationObserver watching for childList changes is the approach I would recommend

But it doesn't work during parsing. This codepen example shows that the console.log in the childConnectedCallback (implemented with MutationOBserver) will never fire for the initial children.

So we need another way to work with initial children.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 29, 2019

I wish the MutationObserver approach just worked.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 29, 2019

Alright, here's an example using CustomEvent to implement childConnectedCallback work, and it catches initial children as well as future children.

How do you feel about this CustomEvent pattern instead of the MO one?

I think I'm liking this pattern because it is simpler, and could be applied to a custom element as a class-factory mixin which would also show what it could be like to ship new builtin features as mixins (referring to #758).

@rniwa
Copy link
Collaborator

rniwa commented Jan 29, 2019

I'm not sure what you mean by mutation observer doesn't get called for the initial children. It definitely gets called at least once in the following code as I tested in Safari 12.0.2 and Chrome 71.0.3578.98. It logs "child changed!" once then "child changed!" one second later as more child is inserted:

<!DOCTYPE html>
<html>
<body>
<pre id="log"></pre>
<script>

customElements.define('test-parent', class extends HTMLElement {
    constructor()
    {
        super();
        this._observer = new MutationObserver(() => this.childChangedCallback());
        this._observer.observe(this, {childList: true});
    }

    childChangedCallback()
    {
        log.textContent += 'child changed!\n';
    }
});

setTimeout(() => {
    document.querySelector('test-parent').appendChild(document.createElement('div'));
}, 1000);

</script>
<test-parent><div></div><div></div></test-parent>
</body>
</html>

@domenic
Copy link
Collaborator

domenic commented Jan 29, 2019

Look at your initial children using .children.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

Look at your initial children using .children.

@domenic That doesn't work, children are not yet upgraded during the connectedCallback of a parent during parsing, so the parent can not see any relevant properties or methods.

Otherwise I would be doing this and not opening these issues.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

@rniwa Yes, however you're missing a key point there: to avoid leaking memory and keep things proper ensure things are cleaned up, doesn't the observer need to be created in connectedCallback and reciprocally disconnected in disconnectedCallback? If you do it in connectedCallback, you'll the observer doesn't work.

@domenic
Copy link
Collaborator

domenic commented Jan 30, 2019

That seems like a very wrong way to use MutationObservers. @rniwa's approach is the correct one; I'm not sure where you got the idea that they "leak memory" or are "kept proper" by for some reason tying them to connectedCallback and disconnectedCallback. Mutations can happen any time; connected or not.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

@rniwa's approach is the correct one

Oh, so if we don't call disconnect on the observer, that's totally fine and it will be garbage collected later once we're done with the element? If so, it'd be nice if that was clear in the documentation that I can find for MutationObserver.

for some reason tying them to connectedCallback and disconnectedCallback

Because in general I try to allocate memory (create references to new objects) when needed, then unallocate memory (lose references to those objects and call relevant cleanup methods) so that I am sure things clean up.

I have too much experience with devtools heap profiler to know that if I don't generally follow this pattern that I see things leak.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

Mutations can happen any time; connected or not.

That's why I simply want to handle initial children in a manual call (hence the deferrals, and .children would work if children were upgraded).

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

So by "proper" I meant "good practice in defining how things are created AND reciprocally destroyed".

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

For example, I would really hate to have a leak because of https://bugs.chromium.org/p/chromium/issues/detail?id=315190 and forgetting to clean things up.

In general I'm trying to be cautious by reciprocally undoing whatever I do in all my components. I've seen to many leaks that were simply solved by just making sure to reciprocally clean things up.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

Alright, here's an example using CustomEvent to implement childConnectedCallback work, and it catches initial children as well as future children.

How do you feel about this CustomEvent pattern instead of the MO one?

I'm still curious to know, what are both of your thoughts on that pattern? (considering it can be cleaned up in disconnectedCallback)

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

Mutations can happen any time; connected or not.

From my observations in Chrome, connectedCallback is only fired for a root node and all of it's children and grand children when the root node is finally placed into a document.

Similarly, all the disconnectedCallbacks of a root node and all its grandchildren are fired when the root node is removed from a document tree.

Finally, connectedCallback or disconnectedCallback is not fired for any grandchildren of the root node if the root node is not in a document regardless if grandchildren are added or removed.

On this note, I'd like to observe things in connectedCallback and clean up in disconnectedCallback because from the behavior I observe, that's the correct pattern. Otherwise manipulating a tree that isn't in the document would be expensive if we were to be calling all connected and disconnected callbacks.

So the same sort of concept can apply to a childConnectedCallback/childDisconnectedCallback concept, and doing it in the constructor feels to be against the patterns defined by the implementation of connectedCallback/disconnectedCallback.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

Alright, I thought about it, a workaround for using the MutationObserver approach and also being able to clean up on disconnectedCallback and re-observe on connectedCallback, is to:

  • create the observer in the constructor so that we get the benefit of it during parsing,
  • disconnect and dispose of it in disconnectedCallback,
  • and in connectedCallback create a new observer if one doesn't exist. So connectedCallback will handle cases of imperative manipulation of the elements (suppose they are cached and will be re-used by React's (or other lib's) internals), while the constructor will handle the parsing case.

A very big downside of this approach is that an observer is created even if the node is outside of a tree, and like I mentioned above in the previous comment, I'd rather make it behave the way connectedCallback and disconnectedCallback do with respect to a those callback being called on a tree only when the tree is inserted into a document.

This is bad because:

  • for a tree outside of a document,
  • childConnectedCallback will be called on parents, while connectedCallback will not be called on children.
  • This will cause drastically different behavior from nodes that are in a document, and will definitely be a source of bugs that require extra complexity to solve.

The CustomEvent approach has the advantage that because it is implemented in connectedCallback and disconnectedCallback, it happens only when the tree is inserted into a document, which is 👍, so:

  • The connectedCallback of a child always corresponds one-to-one with a childConnectedCallback call in the parent.
  • Same with disconnectedCallback and childDisconnectedCallback
  • this prevents the above problem.

So taking @rniwa's example, run the following updated version showing the constructor + connectedCallback approach. Look in the console to see the problem I just described, and you'll see that for the elements which are not being placed into the document, the execution of code is imbalanced:

<!DOCTYPE html>
<html>
    <body>
        <pre id="log"></pre>
        <script>
            let instanceCount = 0
            
            class TestElement extends HTMLElement {
                constructor() {
                    super()
                    this.instanceNumber = ++instanceCount
                    this.changeCount = 0
                    this.createObserver()
                }

                connectedCallback() {
                    console.log('PARENT CONNECTED CALLBACK')
                    this.style = 'display: block'
                    this.createObserver()
                }

                disconnectedCallback() {
                    this.destroyObserver()
                }

                childConnectedCallback(child) {
                    console.log('CHILD CONNECTED CALLBACK')
                    this.changeCount++
                    this.appendChild(
                        document.createTextNode(
                            `child changed! Change count: ${
                                this.changeCount
                            }, Child tag: <${child.tagName.toLowerCase()}>, Child number: ${
                                child.instanceNumber
                            }`,
                        ),
                    )
                }

                createObserver() {
                    if (this._observer) return
                    this._observer = new MutationObserver(changes => {
                        for (const change of changes) {
                            change.type === 'childList' &&
                                change.addedNodes &&
                                Array.from(change.addedNodes).forEach(child => {
                                    if (!(child instanceof Element)) return
                                    this.childConnectedCallback(child)
                                })
                        }
                    })
                    this._observer.observe(this, { childList: true })
                }

                destroyObserver() {
                    if (!this._observer) return
                    this._observer.disconnect()
                    this._observer = null
                }
            }

            customElements.define('test-element', TestElement)

            setInterval(() => {
                document
                    .querySelector('test-element')
                    .appendChild(document.createElement('test-element'))

                console.log(' --- Test tree outside of the document:')
                // shows that the MutationObserver still operates on Nodes that are not in a document. :(
                const one = document.createElement('test-element')
                const two = document.createElement('test-element')
                const three = document.createElement('test-element')
                one.appendChild(two)
                two.appendChild(three)
            }, 1000)
        </script>
        <test-element>
            Test1
            <test-element>Test2</test-element>
            <test-element>Test3</test-element>
        </test-element>
    </body>
</html>

That's not a nice problem to have. The both APIs should be symmetrical!

@trusktr
Copy link
Contributor Author

trusktr commented Jan 30, 2019

@rniwa @domenic How would you solve that problem? So far, the CustomEvent approach seems the best way.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

How would you solve that problem?

I see a way: check if this.getRootNode() is a #document. If not, then don't act on MutationObserver changes, so we act on them only while connected in a document.

@rniwa
Copy link
Collaborator

rniwa commented Jan 31, 2019

Or just check isConnected. Note that it would return false in disconnectedCallback.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

Or just check isConnected.

Ah, nice, thanks.

The following is the CustomEvent-based version, which I just realized works only with Custom Element children that cooperate (emit the event), and not with builtins:

CustomEvent example
<!DOCTYPE html>
<html>
    <body>
        <script>
            let instanceCount = 0

            const childConnectedEvent = new CustomEvent('child-connected', { bubbles: false, composed: false })
            const childDisconnectedEvent = new CustomEvent('child-disconnected', { bubbles: false, composed: false })
                        
            class TestElement extends HTMLElement {
                constructor() {
                    super()
                    this.changeCount = 0
                    this.instanceNumber = ++instanceCount
                    this.__lastKnownParent = null
                    this.__childConnected = null
                    this.__childDisconnected = null
                }
                
                connectedCallback() {
                    this.__lastKnownParent = this.parentElement
                    this.__emitChildConnectedEvent()
                    this.__registerListeners()
                    this.style = 'display: block'
                }

                disconnectedCallback() {
                    this.__emitChildDisconnectedEvent()
                    this.__unregisterListeners()
                    this.__lastKnownParent = null
                }

                childConnectedCallback(child) {
                    console.log('CHILD CONNECTED CALLBACK')
                    this.changeCount++
                    this.appendChild(
                        document.createTextNode(
                            `child changed! Change count: ${ this.changeCount }, Child tag: <${child.tagName.toLowerCase()}>, Child number: ${ child.instanceNumber }`,
                        ),
                    )
                }

                __emitChildConnectedEvent() {
                    childConnectedEvent.element = this // re-use a single event object (is this okay?)
                    this.__lastKnownParent.dispatchEvent(childConnectedEvent)
                    childConnectedEvent.element = null
                }

                __emitChildDisconnectedEvent() {
                    childDisconnectedEvent.element = this // re-use a single event object (is this okay?)
                    this.__lastKnownParent.dispatchEvent(childDisconnectedEvent)
                    childDisconnectedEvent.element = null
                }

                __registerListeners() {
                    this.__childConnected = event => this.childConnectedCallback(event.element)
                    this.addEventListener('child-connected', this.__childConnected)

                    this.__childDisconnected = event => this.childDisconnectedCallback(event.element)
                    this.addEventListener('child-disconnected', this.__childDisconnected)
                }

                __unregisterListeners() {
                    this.removeEventListener('child-connected', this.__childConnected)
                    this.removeEventListener('child-disconnected', this.__childDisconnected)
                }
            }

            customElements.define('test-element', TestElement)

            setInterval(() => {
                document
                    .querySelector('test-element')
                    .appendChild(document.createElement('test-element'))

                console.log(' --- Test tree outside of the document:')
                // Nice! shows that nothing fires when the a tree is not in the document.
                const one = document.createElement('test-element')
                const two = document.createElement('test-element')
                const three = document.createElement('test-element')
                one.appendChild(two)
                two.appendChild(three)
            }, 1000)
        </script>
        <test-element>
            Test1
            <test-element>Test2</test-element>
            <test-element>Test3</test-element>
        </test-element>
    </body>
</html>

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

@rniwa It keeps getting more complicated: now if we create an instance with new, f.e. new TestElement, then add children to it but avoid calling childConnectedCallback because isConnected is false, then we have to somehow ensure that when the element is connected we'll fire childConnectedCallback for its children.

Now we're back at square one again, trying to do the same thing we couldn't do before without a macrotask! So if we want to fire childConnectedCallback initially for the element's children in connectedCallback because isConnected being false prevented us from firing it before connecting, then we have no choice but to use setTimeout!

Is there another way?

Now I feel like I've ran in a circle.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

And then there's the issue of whether or not those children are defined yet or not, if they're elements with hyphens in their names.

So if we are using React, which will create elements without them being isConnected and will eventually place them into the DOM, we'll certainly run into this problem.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

I think the only option is to tell end users of the elements to ensure that the custom elements are always defined before they are ever used (f.e. define them in the <head> of the page). This way, we can rely on the custom elements' children being already upgraded when the parent's connectedCallback manually observes the children.

Mainly I was trying to make everything "just work", even if for example the end user of the custom elements placed the script tag at the bottom of the page (defined the custom elements at the end of the body). To make this work is turning out to be such a PITA.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

This is bad, because what if an author doesn't control the load order of scripts? What if all they can do is import the element classes and define them in their module, which might just happen to be loaded at the end of the body.

@trusktr
Copy link
Contributor Author

trusktr commented Jan 31, 2019

Another solution is to tell the user to bring back jQuery:

$(document).ready(() => {
  // now end user of the custom elements can begin manipulating them.
})

@trusktr
Copy link
Contributor Author

trusktr commented Feb 1, 2019

Alright, the following is the final solution with MutationObserver which seems to work in all the cases where the custom elements are defined BEFORE they are used, cleans itself up, and triggers childConnectedCallback and childDisconnectedCallback following the same pattern as connectedCallback and disconnectedCallback wherein they are only called when element is connected/disconnected into/from a context (document or shadow root).

Don't mind the Class() syntax which is using class inheritance from lowclass. The Mixin() helper simply makes a mixin so that the class has a static mixin method). Other than this, you can imagine it using plain class syntax.

The implementation of observeChildren is here, which is just using MutationObserver with childList: true to call the connected and disconnected callbacks that are passed into observeChildren.

import Class from 'lowclass'
import Mixin from 'lowclass/Mixin'
import { observeChildren } from '../core/Utility'

export default
Mixin(Base => Class('WithChildren').extends(Base, ({ Super, Private, Public }) => ({

    // not sure if the custom element polyfills include this property, so we define it just in case.
    isConnected: false,

    constructor(...args) {
        const self = Super(this).constructor(...args)
        Private(self).__createObserver()
        return self
    },

    connectedCallback() {
        this.isConnected = true
        Super(this).connectedCallback && Super(this).connectedCallback()

        const priv = Private(this)

        // NOTE! This specifically handles the case that if the node was previously
        // disconnected from the document, then it won't have an __observer when
        // reconnected. This is not the case when the element is first created,
        // in which case the constructor already created the __observer.
        //
        // So in this case we have to manually trigger childConnectedCallbacks
        if (!priv.__observer) {
            const currentChildren = this.children

            // NOTE! Luckily Promise.resolve() fires AFTER all children connectedCallbacks,
            // which makes it similar to the MutationObserver events!
            Promise.resolve().then(() => {
                for (let l=currentChildren.length, i=0; i<l; i+=1) {
                    this.childConnectedCallback && this.childConnectedCallback(currentChildren[i])
                }
            })
        }

        Private(this).__createObserver()
    },

    disconnectedCallback() {
        this.isConnected = false
        Super(this).disconnectedCallback && Super(this).disconnectedCallback()

        // Here we have to manually trigger childDisconnectedCallbacks
        // because the observer will be disconnected.
        const lastKnownChildren = this.children

        // NOTE! Luckily Promise.resolve() fires AFTER all children disconnectedCallbacks,
        // which makes it similar to the MutationObserver events!
        Promise.resolve().then(() => {
            for (let l=lastKnownChildren.length, i=0; i<l; i+=1) {
                this.childDisconnectedCallback && this.childDisconnectedCallback(lastKnownChildren[i])
            }
        })

        Private(this).__destroyObserver()
    },

    // private fields
    private: {
        __observer: null,

        __createObserver() {
            if (this.__observer) return

            const self = Public(this)

            this.__observer = observeChildren(
                self,
                child => {
                    if (!self.isConnected) return
                    self.childConnectedCallback && self.childConnectedCallback(child)
                },
                child => {
                    if (!self.isConnected) return
                    self.childDisconnectedCallback && self.childDisconnectedCallback(child)
                },
                true
            )
        },

        __destroyObserver() {
            if (!this.__observer) return
            this.__observer.disconnect()
            this.__observer = null
        },
    },
})))

And here's how to use it: take your existing class,

class MyElement extends HTMLElement {
    connectedCallback() { /* ... */ }
    disconnectedCallback() { /* ... */ }
}

and mix the functionality in when you need it:

import WithChildren from './WithChildren'

class MyElement extends WithChildren.mixin(HTMLElement) {
    connectedCallback() { /* ... */ }
    disconnectedCallback() { /* ... */ }

    childConnectedCallback(child) { /* ... */ }
    childDisconnectedCallback(child) { /* ... */ }
}

Next I'd like to investigate if #789 can help make this cleaner.

Sidenote: In reference to #758, I like the idea of selectively adding new Custom Element features/APIs to my classes via mixins.

@trusktr
Copy link
Contributor Author

trusktr commented Feb 2, 2019

My assumption is that I need to clean up the observer by calling disconnect() on it when I no longer need it. Is this true?

If we don't have to call disconnect() and can rest assured that it will be GC'ed when there's no references to the Node, Node's children, the observer, the observer callback functions, or anything in the observer callback functions' scopes, then the above example could be simplified to the following.

(For the sake of anyone not familiar with lowclass, I converted it to a regular class)

import Mixin from '../core/Mixin'
import { observeChildren } from '../core/Utility'

export default
Mixin(Base => class WithChildren extends Base {

    constructor(...args) {
        super(...args)
        
        // TODO, if the polyfills cover this remove it, isConnected is already
        // part of the window.Node class (in the specs) and serves the same
        // purpose. https://github.com/webcomponents/webcomponentsjs/issues/1065
        this.isConnected = false
        
        this.hasBeenDisconnected = false
        
        this.__createObserver()
    }

    connectedCallback() {
        this.isConnected = true
        super.connectedCallback && super.connectedCallback()

        if (this.hasBeenDisconnected) {
            this.hasBeenDisconnected = false
            // We have to manually trigger childConnectedCallbacks on connect.
            const currentChildren = this.children
            
            // NOTE! Luckily Promise.resolve() fires AFTER all children connectedCallbacks,
            // which makes it similar to the MutationObserver events!
            Promise.resolve().then(() => {
                for (let l=currentChildren.length, i=0; i<l; i+=1) {
                    this.childConnectedCallback && this.childConnectedCallback(currentChildren[i])
                }
            })
        }
    }

    disconnectedCallback() {
        this.isConnected = false
        this.hasBeenDisconnected = true
        super.disconnectedCallback && super.disconnectedCallback()

        // Here we have to manually trigger childDisconnectedCallbacks
        // because the observer will be disconnected.
        const lastKnownChildren = this.children

        // NOTE! Luckily Promise.resolve() fires AFTER all children disconnectedCallbacks,
        // which makes it similar to the MutationObserver events!
        Promise.resolve().then(() => {
            for (let l=lastKnownChildren.length, i=0; i<l; i+=1) {
                this.childDisconnectedCallback && this.childDisconnectedCallback(lastKnownChildren[i])
            }
        })
    }
    
    __createObserver() {
        observeChildren(
            this,
            child => {
                if (!this.isConnected) return
                this.childConnectedCallback && this.childConnectedCallback(child)
            },
            child => {
                if (!this.isConnected) return
                this.childDisconnectedCallback && this.childDisconnectedCallback(child)
            },
            true
        )
    }
})

Again, note that observeChildren is just a small wrapper around creating the MutationObserver, assume that it does not cause any references to be held.

Will this work? Can we expect things to be garbage collected when the node is not used anymore, without calling observer.disconnect()?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants