[css-overflow-3] Specify extent of ink overflow

[chrishtr]

The spec defines ink overflow, but does not specify the extent of it, because some effects in CSS, such as blur radius or box shadow radius, are theoretically infinite.

However, issues such as #8597 and w3c/IntersectionObserver#295 are good reasons to define it, for the purpose of knowing the size of an animated image of a DOM subtree for the purpose of creating a bitmap image, or occlusion testing.

I think we can define it in cases of blur or box shadow in a reasonable way, such as defining the ink overflow to go out to a certain specified standard deviation on its Gaussian shape.

[fantasai]

If it's possible, it would be better to find a way to solve these problems without needing to define the extent of ink overflow. The ink overflow extent isn't something that can be made necessarily interoperable (it depends on where you cut infinite blur etc, but also on what fonts are used, and possibly other effects). So pages that depend on it being a certain value could then break... I'd rather not expose it to the Web if we can avoid it.

[szager-chromium]

If it's possible, it would be better to find a way to solve these problems without needing to define the extent of ink overflow. The ink overflow extent isn't something that can be made necessarily interoperable (it depends on where you cut infinite blur etc, but also on what fonts are used, and possibly other effects). So pages that depend on it being a certain value could then break... I'd rather not expose it to the Web if we can avoid it.

I'd like to press on this issue a bit if I may. There are now a couple of in-development specs that would like to rely on ink overflow extent, and it seems like a legitimately useful thing for any visually rich application environment. It isn't necessary, I think, to specify ink overflow down to the pixel; it would be enough to specify the maximum possible extent of any ink overflow, and by that measure many types of ink overflow are already de facto interoperable and specifiable. Glyph overhang is a notable exception, but perhaps there is a way to specify the maximum possible glyph overhang for a piece of text?

If necessary, we could limit the spec language to the lowest-hanging fruit, i.e., types of ink overflow that are trivial to describe. But I'd like to see if we can do something.

[drott]

From the font technology perspective, for OpenType fonts the maximum glyph extent and maximum per-glyph bounding box should be specified in the HEAD table in the xMin, xMax, yMin, yMax fields. Compare here. This a per-font metrics, as opposed to a per-glyph metric.

But: I am not sure if applications are practically limited by that or restricting glyph drawing to the extents of this top-level bounding box. The individual bounding box of a glyph can still be determined by calculating a bounding box of its glyph shape and thus, a drawing backing surface can be allocated for an individual glyph that's potentially larger than what the font specifies in the HEAD table.

I haven't checked a set of fonts on whether they specify reasonable values in there. So what usually happens if relying on a font metrics that's not been used in a strict way before: We may find many existing fonts having incorrect values in these field and if they were used as a strict restriction, glyphs may be cut off.

CC @bungeman @behdad

[jfkthame]

I've definitely seen fonts with incorrect bounding-box data, though I don't recall the specific examples, and this did result in clipped rendering in some context (which is what led to investigating the font data), while in other environments the glyphs rendered fully.

One point to note with regard to OpenType bounding boxes is that they represent the bounding box of the glyph geometry, but do not account for the fact that at rendering time, the effects of grid-fitting and antialiasing may mean that pixels just outside the "theoretical" glyph bounds are actually affected.

And how do the extents in the 'head' table work in the case of variable fonts? Do they indicate the maximum bounding box under any combination of variation settings, or the bounding box of the default instance? The spec does not appear to provide for applying deltas to these fields, AFAICS.

[drott]

And how do the extents in the 'head' table work in the case of variable fonts? Do they indicate the maximum bounding box under any combination of variation settings, or the bounding box of the default instance? The spec does not appear to provide for applying deltas to these fields, AFAICS.

Good point, the spec says further below:

"Also, in a variable font, [...] the xMin, yMin, xMax and yMax values in this table might or might not encompass the derived glyph outlines for an instance. Also, variation deltas are not provided for these values. If an application requires a bounding rectangle that encompasses the glyphs for a non-default instance of the font, the derived glyph outlines for the instance should be processed to determine a bounding rectangle."

So these HEAD table font-level metrics are not particularly reliable and even unusable for variable fonts. Glyph-level determination of bounding boxes is more accurate, and can take variations into account, but is relatively expensive to compute.

[szager-chromium]

@drott I presume all browsers must already compute glyph overhang extent, for the purpose of determining the size of composited layers, no? If I'm not mistaken, here is where chromium does that.

This code is unfamiliar to me, but it appears that chromium iterates over individual glyph bounds rather than relying on xMin etc. for the font. That probably doesn't bode well for the prospect of specifying glyph overhang interoperably.

I'm quite ignorant about font specifications, so I'm asking the most naive questions, but: can you think of any reliable way to describe the maximum possible extent of glyph rendering for a given block of text and a given font across all browsers?

[szager-chromium]

I'm hoping the CSSWG can resolve on whether to land these PR's, which relate to this issue and also #10066:

#9823
#9842
#10085

[fantasai]

@szager-chromium I think it would help if you could describe what you're trying to accomplish with these PRs and for what purpose. I'm not sure I agree with the changes, and I'm not convinced that defining ink overflow in this manner is reasonable or warranted; but it would help to understand what you are trying to do. Can you give specific examples of why you need the ink overflow extent defined in this way?

[szager-chromium]

@fantasai I refer to the issue links in the first comment, and to my comments below it. I'm not very familiar with view transitions so I don't have much to add about #8597, but I can speak to the IntersectionObserver issue, regarding the V2 feature set. A critical aspect of that API is that it reports whether an element of interest is occluded by other content, and the occlusion detection is based on ink overflow rectangle. V2 shipped in chromium in 2020, and it is increasingly popular. One of the issues that came up in spec conversations about V2 is that it's difficult to guarantee interoperability because the extent of ink overflow is insufficiently specified.

IntersectionObserver V2 is most useful in a cross-origin embedding context, where the embedee doesn't trust the embedder to display the embedded content faithfully. In this situation, the burden falls on the embedder to ensure that the embedding page doesn't inadvertently occlude embedded content, and it's difficult for authors to do this reliably and interoperably if they can't depend on some upper bound on ink overflow rects generated by their content. For that reason, the focus of my work has been to provide a reliable upper bound on ink overflow, and to provide practical guidance that authors can rely on.

[fantasai]

@szager-chromium I think the bounds that you would want for occlusion detection and the bounds we'd want for painting are not necessarily the same. For example, the theoretical bounds of a blur radius are infinite, although practically speaking we clip those. But where we clip the blur effect and where you decide "the stuff below this is noticeably occluded" might not be the same.

I also don't understand what sort of real-world interop issues you're running into that are motivating your proposed changes to the specs. The concept of ink overflow is reasonably well defined, and aside from where to consider the boundary of blur effects I don't see anything that needs further specification.

[szager-chromium]

@fantasai At least in the case of IntersectionObserver V2, which is security-focused, we always want to err on the side of caution and report content as "possibly occluded" whenever the UA cannot absolutely guarantee that not a single pixel gets shaded by other content. It is prohibitively expensive to determine occlusion precisely; that's why "maximum possible extent" is an useful concept. We can analyze the maximum possible extent of all ink overflow on a page and conservatively mark content as "possibly occluded" whenever there is any overlap whatsoever.

We have done some monitoring and analysis of IntersectionObserver V2 as deployed on partner sites, and have concluded that this conservative approach based on maximum possible extent of ink overflow works well in practice. There is a very low incidence of embedders inadvertently causing V2 to report content as "possibly occluded." So we have confidence in this approach. However, other browsers have rightly pointed out that the lack of specificity in the HTML spec around the extent of ink overflow makes it hard to guarantee interoperability of this feature.

Blur radius is the quintessential example of a feature that runs exactly counter to the "maximum possible extent" approach, but fortunately all the major browser implementations are de facto interoperable because they all use the same math to determine the extent of a blur effect. The change I landed to the fxtf spec was simply to non-conformingly codify and document that.

Text and text decorations are the only sources of ink overflow for which there isn't a completely satisfying answer to the "maximum possible extent" question. However, in conversation with @drott and @kojiishi, I landed on the language in #9842. The tl;dr is: "for a given well-defined font and text input, across browser implementations, the maximum possible extent of ink overflow from hanging glyphs is no more that a 2px outset from the painted area determined by visual inspection in any conforming browser." Or, even more succinctly: "look at your content in any browser, and make sure there's a 2px gap between any painted text and any element you want to avoid occluding."

Text decorations are even harder than text in this regard: the extent of ink overflow is a combined product of the font rendering system and the browser implementation. #10085 is, I admit, not wholly satisfying, but it is the best guidance I could come up with for authors trying to reason about the maximum possible extent of ink overflow.

[jfkthame]

"for a given well-defined font and text input, across browser implementations, the maximum possible extent of ink overflow from hanging glyphs is no more that a 2px outset from the painted area determined by visual inspection in any conforming browser."

This seems a bit unsatisfactory to me, because I think there are a number of factors it ignores. As an author, you can't actually be sure what font will be used to render the content on the user's machine. Your carefully-chosen webfont may have failed to load, for any of various potential technical, policy, or user-preference reasons. So the user may end up seeing a substitute or fallback font that has very different overhang characteristics.

In addition, the user may have set a min-font-size preference in their UA (e.g. for accessibility reasons) that mean the text is being rendered at a different size than you expected, resulting in greater overflow. Or they may have an accessibility feature that adds a high-contrast opaque background behind text elements, potentially extending more than 2px outside the glyph bounds.

So if it's necessary for authors to reason with absolute certainty about this for security reasons, I think the proposed text here falls short, and potentially gives a false assurance.

[szager-chromium]

@jfkthame Those are legitimate concerns with respect to IntersectionObserver V2. If UA settings or accessibility features cause an element of interest to be occluded, then the correct behavior is for V2 to report the content as "possibly occluded". I think it's a good idea to for the spec to call out these issues in non-normative text, but I would lean towards adding it to the IntersectionObserver spec.

I still think it's worth exploring whether there is some language we can add to css-text that is helpful to developers and relevant to non-IntersectionObserver use cases. Including caveats such as "for a given well-defined font" allows this change to be focused on the behavior of the glyph rendering subsystem without involving other parts of the spec or the browser.

[noamr]

For view transitions, the relevant issue is #8282. We want elements to count as "off screen" if they're actually offscreen, including their drop-shadows etc.

Perhaps we can reach some middle ground, where we define something that's ink-overflow-ish, and includes glyphs of the known font, drop-shadows, a fixed number/function for blur etc, and ignores user overrides, and can be used for Intersection observer v2, view transitions, and any other place that requires this kind of insight. Though these quirks should be measured against an option for the author to supply their own margin with reasonable defaults.

[szager-chromium]

@noamr For the view transition case, similar to IntersectionObserver, it appears that "maximum possible extent of ink overflow" is a useful concept.

[astearns]

Given that @fantasai and @jfkthame do not yet appear to be satisfied with this discussion, I think it’s premature to put this on the agenda.

[szager-chromium]

@fantasai @jfkthame Can I ask you to respond to my most recent comments? There is a genuine need here -- View Transitions is a CR, and IntersectionObserver V2 is used on 6% of page loads in Chrome. Even if I can't overcome your objections to the PR's I put up, I would like to find a way forward.

Thanks,

Stefan

[szager-chromium]

@jfkthame -- you make a good point that the actual font description used by the glyph rendering subsystem might be different from the one requested by the developer, in which case any visual inspection of ink overflow they did during development will not apply. I meant to acknowledge this with the preface “for a given well-defined font” , but the spec text could be more explicit, something like:

“note that there are many factors that may cause the glyph rendering subsystem to use a font other than the one specified in stylesheets when drawing text, including but not limited to UA settings, accessibility features, and the availability of web fonts. The extent of ink overflow derives from the actual font used for rendering text, and developers should allow for the possibility that the actual rendered extent of ink overflow from glyph overhang does not match what they measured during development."

... and then we could add some language to the IntersectionObserver spec along these lines, with links to the css-text spec:

“Note that the extent of ink overflow from text and text decorations cannot always be predicted or measured in advance. In all cases, IntersectionObserver will determine occlusion based on the actual rendered extent of ink overflow.”

@fantasai -- Although the concept of ink overflow is well-defined, the extent of it is not, and that’s what I’d like to fix. I hope my previous comment about the utility of “maximum possible extent of ink overflow” shows why the difference between painted extent and occlusion is not a problem – in fact in almost all cases they are the same, and even for blur filters there is an easy practical solution that also agrees with browsers in practice. If we encounter a particular special case where there is a divergence we can note it explicitly in the spec.

[chrishtr]

Given that @fantasai and @jfkthame do not yet appear to be satisfied with this discussion, I think it’s premature to put this on the agenda.

I didn't see any more comments, but in any case @szager-chromium added more clarifications yesterday. I think this issue would benefit from more live discussion next week.

[fantasai]

Some points from reviewing the issue and the proposed PRs:

• What @szager-chromium describes as a "hanging glyph" in his proposed appendix is not what css-text considers a "hanging glyph". What he's describing (see his very nice illustration) is a glyph whose glyph bounds extend beyond its bounding box. This is already defined as being ink overflow in the definition of ink overflow. I don't mind adding clarifying statements about that elsewhere (e.g. where we define the sizing of an inline box), but it definitely has nothing to do with the definitions in hanging-punctuation.
• For border-image, this is already defined as ink overflow, but the terminology didn't exist when css-backgrounds-3 was written; added in as an editorial clarification to the spec in ee0d718. I also made the same clarification for box-shadow in b03d123.
• Text decoration is already defined as ink overflow, and I don't think the proposed example adds very much to the spec.

As @jfkthame explains, and as several of your PRs admit, ink overflow is not going to be a 100% interoperable measurement, only an interoperable concept, because of the differences in painting and font rendering across implementations.

Given we've defined what is and is not ink overflow already, what point of confusion are you trying to clarify here in this issue?

[szager-chromium]

@fantasai -- jfkthame made the same point about "hanging glyph" in a code review comment; I updated the PR to correct the language.

For box-shadow, I deliberately placed the new text just after the illustrated example that shows the effect of spread and blur; I felt that it added clarity to the specifics of the geometry. I agree that border-image-outset doesn't need any clarification.

More generally, although the language added to text and text-decoration is squishy and non-normative, it might also be useful for developers who want to reason about the extent of ink overflow. Specifically, the de facto (within 2px) consistency of ink overflow extent between different implementations for a given well-defined font, and the effect of using the from-font keyword where supported for text decorations, might be useful guidance. Maybe they belong in the IntersectionObserver spec rather than in css-text or css-text-decoration; or maybe they belong in a blog post or a tweet instead of a spec. That's the question I'm hoping to resolve.

[frivoal]

maybe they belong in a blog post or a tweet instead of a spec.

That would be my take.

[css-meeting-bot]

The CSS Working Group just discussed [css-overflow-3] Specify extent of ink overflow, and agreed to the following:

• RESOLVED: Close the issue, adding a note to the ink overflow definition about platform and rendering differences

The full IRC log of that discussion

<emilio> szager: trying to summarize the discussion
<emilio> ... IntersectionObserver v2 uses ink overflow rects to determine visibility
<emilio> ... view transitions use it too
<emilio> ... it was brought up to me that ink overflow is a bit under-specified
<emilio> ... which can cause interop issues
<emilio> ... my personal agenda was to enable IOv2
<emilio> ... for everything except text, the extent of ink overflow is pretty straight-forward
<emilio> ... a couple places didn't call out it in the spec
<emilio> ... so I wrote a few PR
<florian> q+ to speak (once szager is done)
<emilio> ... but text is the really different one
<emilio> ... when laying out text we're subject to font selection / glyph shaping, web devs understand that different platforms give different results
<emilio> ... you can access the layout geometry via APIs and getBoundingClientRect()
<emilio> ... on inlines etc
<emilio> ... ink overflow is different, it's magic and not observable
<emilio> ... I wonder if we can help developers get a hand on this
<emilio> ... given we now have APIs which kind of expose ink overflow extents directly, it seems weird not to expose them
<astearns> ack florian
<Zakim> florian, you wanted to speak (once szager is done)
<chrishtr> q+
<emilio> ... I know that some of my PRs are not up to the standard in the spec. I'm not super-engaged on getting them landed but I think we should do the best possible for developers
<emilio> florian: you submitted 3 prs, for text, text-decoration, and for backgrounds
<emilio> ... it seems you want (1) define ink overflow better
<emilio> ... which seems useful
<khush> q+
<emilio> ... I haven't reviewed all the PRs, but the text PR isn't about defining it
<emilio> ... it's author guidance about how to reason about it
<emilio> ... so that's less obvious to me that it belongs in the spec
<emilio> ... you made some arguments which I didn't understand before
<emilio> ... but this looks more like devrel etc
<emilio> ... if there's agreement that this should be in the spec then we should work more of it
<astearns> ack chrishtr
<emilio> ... but as is the PR is not definition of ink overflow, more like author guidance
<emilio> chrishtr: just to bring it back to what I think should be the main resolution
<emilio> ... is ink overflow well-specified enough for those use cases?
<emilio> florian: not sure that's relevant because the PRs doesn't define anything
<emilio> chrishtr: I agree with it being more dev guidance
<emilio> chrishtr: if it's well defined let's close the issue and we can put developer guidance elsewhere
<fantasai> scribe+
<fantasai> emilio: I think the differences in text ink overflow are unlikely to cause substantial web compat problems for the APIs we're exposing right now
<fantasai> emilio: I don't think defining ink overflow for text is a blocker for the occlusion stuff
<fantasai> emilio: nor for View Transitions
<fantasai> emilio: The differences are relatively minor, and also happen even within the same browser across different platforms.
<fantasai> emilio: If you only test Chrome on Windows, Chrome on Mac is different
<fantasai> emilio: I don't think the ink overflow extents for text are likely to break use cases for intersectionObserver v2
<fantasai> emilio: same for View Transitions
<fantasai> emilio: I don't think it can cause realistic compat issues
<fantasai> emilio: Not necessarily well defined
<fantasai> emilio: but it doesn't matter much
<fantasai> emilio: One thing Stephen was asking was about exposing the ink overflow rects more
<fantasai> emilio: I think authors should generally not care about ink overflow
<fantasai> emilio: devtools could point out if needed
<fantasai> chrishtr: Sounds like you think we could close the issue as "good enough"
<florian> q+
<fantasai> emilio: Depends on the goal of the issue
<fantasai> chrishtr: for existing use cases that need it
<fantasai> emilio: text ink overflow maybe needs better definition
<fantasai> chrishtr: open a new issue, add some examples
<emilio> scribe+
<astearns> ack khush
<fantasai> khush: We're using this definition in one spot for VT
<fantasai> khush: but not Web-observable
<emilio> khush: want to clarify that v-t we are using it at one spot but it hasn't been a compat issue because it's not observable at all
<emilio> khush: because the area is managed by the browser
<emilio> ... but we want to use it for a feature where visibility with the viewport is handled with ink overflow instead of border box
<emilio> ... that technically would expose it
<emilio> ... and you could measure it moving a div 1px at a time
<astearns> ack florian
<emilio> ... but for the use cases we want I think it's ok
<emilio> florian: no strong view on whether what we have is good enough
<emilio> ... for v-t or intersection-observer
<emilio> ... predictable for authors is one thing, easy to compute for browsers is another
<chrishtr> q?
<emilio> ... glyphs are not the kind of infinite thing you need to approximate
<emilio> ... it could be very expensive to compute where a glyph ends
<fantasai> Definition of ink overflow https://www.w3.org/TR/css-overflow-3/#ink
<emilio> ... it's where the text ends, which you can compute, even though it might not be easy
<emilio> ... so I don't think it's badly defined
<emilio> szager: All browsers need to compute it already
<astearns> ack fantasai
<emilio> [missed]
<emilio> fantasai: I was on the queue to say what florian said, I don't think text ink overflow is ambiguous
<emilio> ... if there's an actual ambiguity then we should fix the spec
<emilio> ... otherwise it's not ambiguous
<emilio> ... I don't think an API to expose it would be a good idea
<emilio> ... because of that issue
<emilio> q+
<emilio> astearns: I think what's currently in the spec is good enough for current use cases
<emilio> ... so propose to close unless spec is ambiguous
<fantasai> emilio: ambiguity is not so much the spec definition, but the fact that the same text in the same font may generate different ink overflow on different platforms due to shaping and rasterization
<fantasai> emilio: might be worth pointing out in the spec
<fantasai> emilio: so that people defining APIs that depend on ink overflow can take that into account
<fantasai> emilio: doesn't have to be long, just a paragraph, that says it'll differ across platforms and rendering engines
<astearns> s/think what's/think what I’m hearing is that what’s/
<fantasai> emilio: for Khush's proposal about exposing more directly, file a different issue
<astearns> ack emilio
<fantasai> emilio: idk to what extent that can increase fingerprinting surface
<fantasai> emilio: That's a good reason why someone might measure a div pixel by pixel
<fantasai> emilio: not something you really want to expose, maybe it's exposed on canvas already
<fantasai> emilio: worth more thinking
<astearns> ack fantasai
<emilio> fantasai: If you want a note saying this can differ due to different factors then happy to do that
<emilio> chrishtr: sounds great, ty for volunteering
<emilio> RESOLVED: Close the issue, adding a note to the ink overflow definition about platform and rendering differences