Enhancement: write PNG tEXt chunks

[vqt123]

With the MASSIVE generative AI movement, automatic1111 has become a staple tool for image generation. This tool creates images that include lots of data regarding the configuration of any image in the 'tEXT' chunk of an image's metadata. It does not appear that sharp has a way to read or write to this section of the images data.

[vqt123]

here is an image that includes the 'tEXT' data section

https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/60d49930-c321-4012-e4d9-aee47bc49c00/width=1280

the decoded chunk is as follows:

keyword: paramaters
text:
highly detailed robot head, clockwork, transparent, filled with colorful liquid, expressive, abstract, trending on artstation, by ryohei hase Negative prompt: (bad-image-v2:0.8) Steps: 20, Sampler: DPM++ 2M, CFG scale: 7, Seed: 756648331, Size: 512x512, Model hash: c35782bad8, Model: general_realisticVisionV13, Denoising strength: 0.4, ENSD: 31337, Hires upscale: 2.5, Hires steps: 8, Hires upscaler: 4x-UltraSharp, Discard penultimate sigma: True

[lovell]

"Artificial Intelligence" hyperbole aside, there are existing metadata standards such as EXIF that might be more suitable for such image generators to use, however tEXt chunk handling is a useful feature for PNG images in its own right.

The tEXt chunks are currently parsed by libspng/libvips into metadata properties with names of the form png-comment-{index}-{keyword}, for example png-comment-0-parameters. The index exists only within libvips to create unique names, as the PNG spec allows multiple tEXt chunks with the same keyword.

I'd suggest something like the following data structure to expose this, both when reading via metadata() or writing via a new property of png().

// PROPOSED API, NOT YET AVAILABLE
comments: [
  { keyword: 'parameters', text: 'highly detailed...' },
  { keyword: 'Copyright', text: 'legal minefield' },
  ...
]

A PR to expose this in sharp would be welcome, if you're able.

[lovell]

v0.33.5 added support for reading PNG tEXt chunks from an input via #4157. This issue remains open to track the future possible enhancement to control setting/updating PNG tEXt chunks in the output.

[huming2207]

v0.33.5 added support for reading PNG tEXt chunks from an input via #4157. This issue remains open to track the future possible enhancement to control setting/updating PNG tEXt chunks in the output.

Hi @lovell Just out of curiosity, if I want to implement the write tEXt feature, should I use libvips's vips_image_set_string() API?

I might need this later for embedding a set of JSON string for putting on server-rendered UIs on to some e-paper HMI devices. If I need it I will try implementing it and send you guys a pull request.

[lovell]

@huming2207 Thank you for the offer, a PR would be welcome if you're able. You could use either vips_image_set_string or the slightly nicer C++ VImage.set.

As an aside, to anyone implementing this, we'll need to think carefully about how setting one or more comments might interact with existing comments, especially given the use of an (internal-only) index as part of the name.

[huming2207]

Hi @lovell thanks for the info. I made another separate Rust based binding library in the past, that does dithering for 4-bit grayscale to "emulate" 8-bit grayscale which makes the output probably looks nicer: https://github.com/huming2207/epaperify

Anyway I'm not sure yet. I may also work on that one instead. I will investigate further and see what should I do next.