papplDevice Improvements for Default Schemes

[zdohnal]

Is your feature request related to a problem? Please describe.
The CUPS 2.x USB backend provides quirk support - quirks can help in some cases when the device has a known issue which we can workaround, or it can reject the device in case its USB support is utterly broken.

Describe the solution you'd like
Provide a quirk support where files can be read from /usr/share/pappl/quirks/* and from /etc/pappl/quirks/* with higher priority, where printer application maintainers and admins can put quirk files regarding their devices.

Describe alternatives you've considered
This might be solved on printer application level as well, but a generic support in PAPPL would be great.

[michaelrsweet]

OK, so I have reservations about this... By far the most common usage of quirks is to force unidirectional mode, which is unnecessary for a real printer application since the driver only reads when it needs to. I'm not sure "delay-close" matters, either. I'm not sure what to do for for "blacklist" and "no-reattach".

But the bigger issue is that I don't want PAPPL to have to track/support quirks forever like CUPS has, and in the age of snaps and flatpaks and other containers, even having a common PAPPL "repository" of quirks won't work because the instances are all isolated and might not even have access to a common /etc directory for configuration.

[tillkamppeter]

I think PAPPL's USB backend should support quirks, the same types as CUPS supports and ideally the same syntax of quirks files as CUPS uses. Due to the fact that we use sandboxed packaging (Snap or OCI containers, Flatpak does not work with daemons) more and more often we will not have a central quirk database, but the typical native Printer Application in the future will only support a handful of printers or the printers of one manufacturer. Here the developer of the Printer Application could easily set up a table of the few quirky printers which are supported by this Printer Application.

The big database of all known quirks is only needed for the big retro-fitting Printer Applications (like the Ghostscript Printer Application) which will use CUPS-2.x- and/or driver-supplied backends (contained and hidden in the Snap or OCI container) and so have all the quirks on board (pappl-retrofit supports using CUPS 2.x backends).

[michaelrsweet]

@tillkamppeter Looking at the current quirks file, there are 117 unique entries; here is the breakdown:

Quirk	Count
unidir	54
no-reattach	32
blacklist	27
delay-close	9
soft-reset	2
vendor-class	1
whitelist	1
usb-init	0

unidir is unnecessary because with PAPPL the driver only reads if it needs to (vs. the USB backend reading in a background thread).

no-reattach is unnecessary because PAPPL does not re-load the usblp kernel module.

blacklist currently only applies to a subset of Canon inkjet printers, all of which are in a specific range of product IDs (0x3NNN) so if I really needed to I could explicitly mask printers with the Canon vendor ID (0x04a9) and product IDs. Haven't had any bug reports on that...

'soft-reset' applies to the old Prolific USB to parallel converter and to certain Samsung printers (and it looks like at some point we gave up trying to isolate to specific models and always do a soft reset...) If this turns out to be an issue I guess I could add a soft reset on device close, but so far I've had 0 reports of this issue and doing a soft reset typically resets the printer as well which isn't what we want...

delay-close is spread around but is unlikely to apply since the issue had to do with reading back-channel data at the end of a job - again, the PAPPL architecture does things differently and won't have to worry about delaying things outside of the driver.

vendor-class applies to a single receipt printer from EPSON - you'd need to provide your own custom scheme to support this printer with PAPPL (it won't show up otherwise...)

whitelist is used by a single Samsung printer to turn off soft-reset behavior - not needed for PAPPL.

Short of blacklisting the subset of Canon inkjet printers that won't work with PAPPL anyways, I don't see a need to support USB quirks in PAPPL.

[zdohnal]

@michaelrsweet At the start I meant quirk support by providing PAPPL API, which would read files from specified directories and return a list of quirks defined in directories plus the quirk behavior implementation in pappl/printer-usb.c - the database itself would be in a specific printer application (it does not make sense to carry it with PAPPL). But I guess you would not like to have the quirky behavior in PAPPL USB implementation (which I understand), correct?

However that complicates backward support of old printers via retrofitting printer applications - we have the proper drivers, setup and IPP system via printer applications, but some older printers won't work properly as it does with CUPS 2.x.

Does PAPPL support a plug-in backends, so retrofitting printer apps could f.e. provide its own USB backend implementation and still use PAPPL, or is it like "Use PAPPL USB backend or implement the printer application without PAPPL at all"?

[michaelrsweet]

@zdohnal Correct, I would prefer to not encumber PAPPL with USB quirks if we can avoid it.

PAPPL does support adding custom schemes, which Till uses in the pappl-retrofit library to support/use the original CUPS backends. But there isn't a way to suppress the built-in "usb", "socket", and "file" schemes so that users of these applications are unable to specify them.

Conceptually I could update the papplDeviceAddScheme APIs to allow a program to replace the standard handers with its own - then Till could register each of the CUPS backends as their own scheme.

[zdohnal]

Thank you for looking into it!

@tillkamppeter Looking at the current quirks file, there are 117 unique entries; here is the breakdown:
Quirk Count
unidir 54
no-reattach 32
blacklist 27
delay-close 9
soft-reset 2
vendor-class 1
whitelist 1
usb-init 0

unidir is unnecessary because with PAPPL the driver only reads if it needs to (vs. the USB backend reading in a background thread).

no-reattach is unnecessary because PAPPL does not re-load the usblp kernel module.

Aha, we only detach it - hopefully nobody is using it till now.

blacklist currently only applies to a subset of Canon inkjet printers, all of which are in a specific range of product IDs (0x3NNN) so if I really needed to I could explicitly mask printers with the Canon vendor ID (0x04a9) and product IDs. Haven't had any bug reports on that...

'soft-reset' applies to the old Prolific USB to parallel converter and to certain Samsung printers (and it looks like at some point we gave up trying to isolate to specific models and always do a soft reset...) If this turns out to be an issue I guess I could add a soft reset on device close, but so far I've had 0 reports of this issue and doing a soft reset typically resets the printer as well which isn't what we want...

delay-close is spread around but is unlikely to apply since the issue had to do with reading back-channel data at the end of a job - again, the PAPPL architecture does things differently and won't have to worry about delaying things outside of the driver.

vendor-class applies to a single receipt printer from EPSON - you'd need to provide your own custom scheme to support this printer with PAPPL (it won't show up otherwise...)

whitelist is used by a single Samsung printer to turn off soft-reset behavior - not needed for PAPPL.

Short of blacklisting the subset of Canon inkjet printers that won't work with PAPPL anyways, I don't see a need to support USB quirks in PAPPL.

I'm planning to package all printer application into Fedora 39 since cups-filters 2.0b and friends are in Fedora 38 already and announce testing round for printer applications during Fedora 39, so hopefully we will get useful feedback on which quirks are still needed. CentOS Stream and RHEL are another stories - this I will handle later unfortunately.

[tillkamppeter]

But there isn't a way to suppress the built-in "usb", "socket", and "file" schemes so that users of these applications are unable to specify them.

@michaelrsweet For me this does not look correct. When I create a custom backend (scheme) without having touched (for example listed) the backends/devices before, the built-in schemes of PAPPL are not available. If I want both my own scheme and the built-in schemes I have to list the standard schemes via papplDeviceList(() before I add my own scheme:

https://github.com/OpenPrinting/pappl-retrofit/blob/master/pappl-retrofit/pappl-retrofit.c#L4354

[tillkamppeter]

What we need is direct control about what schemes are used. If one just adds a custom scheme, the built-in scheme and the custom scheme should be active. Then there should be an API function to enable and disable schemes, so that one can easily disable unwished schemes, or even, depending on the situation enable the desired set of schemes. The possibility of overloading built-in schemes with ones own, equally-named schemes could also be interesting.

Even if one does not create a custom scheme disabling built-in ones would be useful, as the class of printers where the Printer Application is designed for could be for example only USB printers.

[michaelrsweet]

[Changing the scope/title of this bug]

@tillkamppeter OK, so looking at the code it isn't adding the default schemes.

Specific changes I'm proposing:

• papplDeviceAddScheme* needs to populate the standard URI scheme handlers before adding.
• papplDeviceAddScheme* needs to allow an existing URI scheme to be replaced by a new handler
• Add papplDeviceRemoveScheme to remove the specified scheme (e.g. you don't want file: URIs in your printer application)
• Add papplDeviceRemoveTypes to remove schemes of the specified type (e.g. you don't want network URIs in your printer application)

This should allow pappl-retrofit to replace the existing USB scheme handlers with the wrapper around the CUPS USB backend, for example.

Thoughts?

[tillkamppeter]

This would mean the following:

• If I have a native (not using pappl-retrofit) Printer Application, supporting only printers with Ethernet interface I suppress the USB backend via papplDeviceRemoveScheme("usb:");
• If I have a native Printer Application, supporting only printers with USB interface I suppress all the network backends at once via papplDeviceRemoveType(NETWORK);
• If I have a retro-fitting (using pappl-retrofit) Printer Application supporting printers with USB quirks I overwrite PAPPL's USB backend by creating my own USB backend with papplDeviceAddScheme(), giving it the scheme name usb:
• In general, if I retro-fit CUPS (2.x) backends I create a custom scheme for each individual one, named with the backend's name, giving exactly the same URIs as we had with CUPS (2.x). I do not need to papplDeviceRemoveScheme() the original backend first.
• If I only want retro-fitted CUPS backends and independent which ones are actually available, no original PAPPL backends dangling around, I can papplDeviceRemoveScheme() all the original backends, then papplDeviceAddScheme() my CUPS backend retro-fits, independent whether their names have existed as original PAPPL scheme or not.

Am I right with all this?

[michaelrsweet]

@tillkamppeter Yes, correct. You could also remove all schemes with papplDeviceRemoveTypes(PAPPL_DEVTYPE_ALL)...

I think most printer applications won't use these APIs, but clearly it would be helpful for the retrofitting applications...

[michaelrsweet]

[master 66e6871] Add new papplDeviceRemove APIs and support replacement of device URI schemes (Issue #259)