Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove print filters and printer driver support #4

Open
michaelrsweet opened this issue Feb 17, 2021 · 112 comments
Open

Remove print filters and printer driver support #4

michaelrsweet opened this issue Feb 17, 2021 · 112 comments
Assignees
Labels
enhancement New feature or request priority-high
Milestone

Comments

@michaelrsweet
Copy link
Member

michaelrsweet commented Feb 17, 2021

Note: The goal of this issue is to have a place to discuss the eventual removal of printer driver support and support for raw queues in OpenPrinting CUPS v3.0. The original Apple CUPS bug report has some additional history, and what exists here is an updated summary of that historical discussion.

Why do we want to do this?

  • Raw queues used for special-use printers require custom applications that know about printer capabilities and how to produce printer-ready (document) data. Using CUPS is a convenience, but plenty of applications talk directly to printers (think Point of Sale systems)
  • Raw queues used for shared printers require client software to talk directly to the server to get the printer capabilities, which breaks when sandboxing/AppArmor/SELinux is used
  • Almost every printer manufactured since 2010 supports IPP/2.0 with standard file formats
    • Holdouts are industrial label printers and certain vertical market printers
  • PPDs and drivers have been holding us back from offering better user experience (ready media, localization, full range of printer options/values), improved document processing, and improved accounting
  • PPDs and drivers are a security and distribution nightmare

We already have a replacement for raw queues for shared printers (local/temporary queues managed by cupsd), and raw queues for special-use printers already largely bypass CUPS and can use existing commands or character device files to communicate with those printers.

As for printer drivers, those few printers that "need" them can migrate to standalone applications/services using the CUPS API to provide an IPP Everywhere-compatible Printer instance. PAPPL provides a convenient framework for easily creating these applications and porting existing CUPS raster drivers, and the following printer applications are already available or (in the case of Gutenprint) under development:

  • hp-printer-app: PAPPL-based PCL printer application based on the CUPS rastertohp driver.
  • LPrint: PAPPL-based label printer application, currently supporting Zebra and Dymo label printers with plans to support more, based on the CUPS rastertolabel driver. "Raw" printing is supported as well if you have an application that produces the native print data format.
  • ps-printer-app: PAPPL-based PostScript printer application that supports all CUPS/PostScript printers via PPDs and includes all of the Foomatic and HPLIP drivers.
  • Ghostscript Printer Application: PAPPL-based printer application for Ghostscript-based printer drivers.
  • Gutenprint: A PAPPL-based printer application for all Gutenprint printer drivers.
@tresf
Copy link

tresf commented Mar 15, 2021

@michaelrsweet thanks for the work on these sub-projects which will help fill the gap for the industries relying on this technology. I assume this thread is an open discussion like the previous and I hope my questions and thoughts are well received.

"vertical market printers"

Although I think I understand why this wording is chosen, the shipping industry largely caters to PC owners that have the same (or perhaps similar) plugin-and-play expectations when printing, which is why this issue (as well as the previous discussions over at apple/cups#5271 are of great importance to companies affected by this change. Setting aside the industry's (Point of Sale, Shipping, Tickets, etc) perceived reluctance to adopt IPP, I find it important to still focus on the end-user frustration of "How do I print a label" (or "receipt" or "ticket") to begin ringing help desk lines as soon as these changes are adopted into an operating system.

Some questions... I'll use LPrint as an example, but I assume any IPP-compatible printer application would be relevant instead...

  • Is the expectation for an average user to configure LPrint? If so, is there an expectation that LPrint will be readily available for end-user consumption (via precompiled binary?).
  • If not, is LPrint intended to be bunded with an application? If so, is the expectation that each app will use some form of LPrint discovery? The reason I ask is because configuring a printer is often done prior and separate from the application install. Coupling the two I believe to become a support nightmare and for this reason, I think the term "vertical" is a bit misleading, as if only industries choosing to use this hardware are affected. Often app developers want to support all printer types and the industry which requires certain hardware has historically been the onus of the end-user (person or company) to manage. If LPrint starts to become bundled, printer discovery and installation starts to become shifted onto the application market.
  • Will there be a non-CLI technique for LPrint? It depends where LPrint lives, but Apple's sandboxing rules can make these utilities difficult or impossible to interact with (Linux may suffer a similar fate for snaps).
  • Will CUPS lose the ability to print to a non-IPP printer completely? Let's say a developer doesn't want the ZPL2 driver to begin with, can we still add a raw queue? If not, is there no chance CUPS would be willing to keep this single, raw functionality around? I assume the resistance lies in the fact that the raw queue can't really function on a sane, printer + job lifecycle and therefore would break the workflow of CUPS moving forward, but simply offering the ability to fall-back onto a raw queue would really allow some of these (soon to be driverless) printers work without LPrint.

Last, I'd love to hear a success story about an app that's already successfully leveraged LPrint in a user-friendly way (or a place such as a chat channel to talk with these people).

I hope my questions are well received.

@michaelrsweet
Copy link
Member Author

@tresf Yes, please do continue to use this as an open discussion area.

WRT label printers, there are certainly those that are targeted for SOHO users where the primary focus has been on smaller mailing/identification label uses where the vendor also provides a selection of labels - think Brother, DYMO, and the like. And then there are the "industrial" label printers where you buy labels in bulk from a convenient local/national supplier, often to print thousands of shipping labels per day. The expectations from the former group are very different from the latter, but both are best served by printers that implement IPP/IPP-USB and standard formats.

WRT LPrint (and these responses apply equally to other Printer Applications):

  • LPrint is expected to be readily available to end-users and I'd expect it to be installed by default or offered to be installed when you connect/discover one of the printers that LPrint supports. Till Kamppeter has been working on doing this with the Snapcraft store for Ubuntu, and we've had an interesting discussion on the OpenPrinting printing-architecture list on how we can do this in general.
  • While LPrint can be bundled with an application, that isn't the intended focus since LPrint isn't limited to just one application or use case. Again, the purpose is to get away from applications that need to know how to talk to a particular printer and instead have applications that can provide standard formats that can be printed on any printer through the printing system.
  • LPrint provides a standard IPP service (TCP/IP socket), and if an application (sandboxed or otherwise) is using the standard print APIs the LPrint-configured printers will just show up in the list of available destinations. LPrint also provides a web interface, and (soon via PAPPL) will have a simple GUI status "widget" for desktop users that provides current status info and easy access to the web interface.

WRT CUPS, yes it is going to lose the ability to host a raw queue. Tools like "ippeveprinter" could still be used to setup individual "raw" printers that CUPS could then print to, but honestly by the time we are ready to release CUPS 3.0 there shouldn't be any printers that don't have a corresponding printer application. It's not like we are charging forward on a whim, this has been in the works for years (I first started talking about it with printer vendors in 2010) and we want to make sure we have everything in place before we "throw the switch".

@tresf
Copy link

tresf commented Mar 15, 2021

The expectations from the former group are very different from the latter, but both are best served by printers that implement IPP/IPP-USB and standard formats.

I'd like to share my experience...

Although this is largely true from 10,000 feet, it's not true in practice. For example, Amazon fulfillment is moving label printing from warehouses to homes. There's also a large use-case for label printers for non-shipping (e.g. inventory management). UPS worldship on Windows is another very popular SMB/home use application and it's perfectly normal to get a non-IPP capable printer for this purpose. As ISVs find creative ways to leverage CUPS to perform similar (e.g. UPS worldship-style) tasks, the hardware is slowly becoming commonplace on Mac, Linux and other CUPS environments. At least from my corner/soap box of tech -- SOHO (Small Office/Home Office) to SMB (Small/Medium Business) to completely integrated enterprise environments, the printing support is mostly analogous. With the speed and setup of cloud services, it's completely normal for an ISV to start off coding to ZPL but then add proper support for a wider range of hardware and the end-user rarely knows or cares about this, regardless of their business size.

I'd expect it to be installed by default or offered to be installed when you connect/discover one of the printers that LPrint supports.

Thanks, I'm hopeful this becomes true, this is great news.

LPrint provides a standard IPP service (TCP/IP socket)

Great, the existing CUPS IPP sockets seem to work well in a sandbox, but as you've stated, if the "driver" install binds to some form of LPrint wizard, this will already be done by the time the ISV need to talk to the printer.

by the time we are ready to release CUPS 3.0 there shouldn't be any printers that don't have a corresponding printer application.

Right, I think a source of concern would then be LPrint adoption at large (e.g. Apple). Thanks for sharing your feedback on all of this, it's greatly appreciated.

@michaelrsweet
Copy link
Member Author

@tresf FWIW, I'm well aware of the "work from home" warehousing. Existing SOHO label printers are, quite frankly, not up to the task. The mid-range industrial printers (Zebra, etc.) are OK, but usability is on par with the high-volume models and requires a fair amount of manual configuration and, of course, a driver/Printer Application...

I've been working with one vendor on a new 4" medium volume (~5000 4x6 labels/day max) SOHO label printer with AirPrint/Mopria/IPP Everywhere/IPP-USB support. This printer works, out of the box, with Android, Chrome OS, iOS, Linux, macOS, and Windows 10 using IPP - no drivers or Printer Applications needed, all using IPP and Apple Raster, PWG Raster, PNG files, or PDF (yes, PDF!)

@tillkamppeter
Copy link
Member

tillkamppeter commented Oct 25, 2021

An update on available Printer Applications:

Most existing free software CUPS drivers are already available in Printer Applications. They are in the 5 Printer Applications listed below. You can get them all in the Snap Store: LPrint, the other 4

These are the Printer Applications (links to their source repositories):

  • PostScript Printer Application: Supports all PostScript printers. If your printer model is not listed/recognized, you can simply load the PPD file from the manufacturer into the Printer Application using the web interface.
  • HPLIP Printer Application: Supports all HP printers, also the ones for which HPLIP needs the proprietary plugin, you can load it into the Printer Application via the web interface. Scanning support for the multi-function devices will get added later.
  • Gutenprint Printer Application: Supports Epson and Canon inkjets, dye sublimation printers and several other printers. All adjustable parameters of the driver are available via the web interface.
  • Ghostscript Printer Application: Supports all printers which work with GhostScript's built-in drivers, Foomatic PPD files and many other drivers. This should cover practically all regular printers which work under Linux but are not supported by the other three Printer Applications. For example non-HP PCL laser printers, non-driverless PDF printers, dot-matrix printers, label printers not supported by LPrint, many laser and inkjet printers with prioprietary protocols, ...
  • LPrint: Supports many label printers.

Driver Listings on OpenPrinting have links to the Printer Applications now.

If you are using a laser printer from Ricoh and OEM (Gestetner, InfoPrint, Infotec, Lanier, NRG, Ricoh, Savin) they are supported by the PostScript Printer Application (PostScript models/mode) and the GhostScript Printer Application (PDF and PCL models/modes). Ricoh is supplying me with support for the newest models regularly and I am updating the Printer Applications appropriately.

The Printer Applications support the CUPS drivers with their full functionality, including back- and side channel functionality. For this the Printer Applications use the CUPS backends and not the PAPPL ones, especially also CUPS backends which come with the drivers.

Note also that there are two errors in the initial posting of this issue:

  • The PostScript Printer Application does NOT contain the Foomatic drivers, they are in the Ghostscript Printer Application
  • On the home page of Gutenprint you do not find a Printer Application (yet). For the time being until they publish their own native Printer Application, use my Gutenprint Printer Application mentioned above, which retro-fits the Gutenprint CUPS driver.

@RokeJulianLockhart
Copy link

RokeJulianLockhart commented Sep 1, 2022

As a mere user, is this important? I care solely whether iscan-firmware, epson-inkjet-printer-escpr, and epson-inkjet-printer-escpr2 shall continue to operate.

@zdohnal
Copy link
Member

zdohnal commented Sep 1, 2022

@BEEDELLROKEJULIANLOCKHART this change is not about scanning, so no impact on iscan package. And regarding printing functionality, in case your device supports a driverless standard (Airprint/IPP Everywhere for network, or IPP-over-USB for USB) and its options suit you, you can switch to driverless and abandon the printer driver package.

In case you have old device or driverless doesn't work for you, you will have to use a printer application - they are currently available as SNAPs.

@tresf
Copy link

tresf commented Sep 1, 2022

Is there a quick way to test this new functionality (e.g. a Linux distro that makes this pretty easy to setup and test without breaking a live machine)?

you can switch to driverless and abandon the printer driver package.

I thought raw was being removed too per apple/cups#5271.

@zdohnal
Copy link
Member

zdohnal commented Sep 2, 2022

Is there a quick way to test this new functionality (e.g. a Linux distro that makes this pretty easy to setup and test without breaking a live machine)?

IMHO all distros don't break a live machine if you set this up, since CUPS supports all ways (printer driver, driverless and raw) for long time - although there is one tweak, which does not break your machine, but prevents you from testing temporary queue functionality - you just have to make sure to don't use the same queue name for your old permanent queue as the temporary has, otherwise CUPS will not create a new temporary destination (to protect your current configuration).
The temporary queue name is created from:

$ avahi-browse -avrt
...
= enp0s25 IPv4 HP LaserJet M1536dnf MFP (42307C)             _ipp._tcp            local
...
$ lpstat -e
HP_LaserJet_M1536dnf_MFP_42307C

as you can see, libcups strips or replace not-alphnum characters in mDNS entry and use it as queue name for temp queue.

About being pretty easy to setup, it's subjective - IMHO driverless printer (which is a printer supporting AirPrint/IPP Everywhere/IPP-over-USB or any other driverless standard) can be setup pretty easily on all distros, just plug them/set them up on your local network and your good to go - at least from CUPS perspective, how your favorite application handles a temporary queue is a quite different story... As for non-driverless printer, you have to install a correct printer application from SNAP repository, configure it via its web interface/CLI and then CUPS sees it via mDNS. There is work in progress on implementing this via GNOME Control Center, so it might be easier in the future.

As for distros where you can test - any distro with recent CUPS (in general approx 2.2.7 or 2.2.8 and up is good start (or an older if fixes are backported as I did in RHEL 8/CentOS Stream 8) - since there was an important fix regarding IPP Everywhere there, but the API is in place since 2.1 or 2.0) and snapd will do - differences can lie in whether you have to install and start Avahi and nss-mdns plugin, which are needed for mDNS support and .local hostname resolution. But from my experience Fedora and Ubuntu work for this.

you can switch to driverless and abandon the printer driver package.

I thought raw was being removed too per apple/cups#5271.

driverless != raw

Driverless means the printer driver functionality is substituted by IPP protocol (printer's options and settings are communicated via IPP instead of being taken from PPD file) - driverless queue is a queue which communicates with printer by IPP 2.0+ and has IPP Everywhere model or it is a temporary queue.
Regarding raw, we should make a difference between raw queues and raw printing:

  • Raw printing means sending a file, which printer supports, to a queue, which is driverless or with a printer driver, so no filters are applied - options can be applied depending on which communication protocol you use with printer (f.e. printer supports Postscript and communicates via IPP, so if you send PostScript file via IPP with options, they are applied. However if you do the same with socket or usb backends, the options are not applied, because they don't send options to the printer and count on filters or application to apply them). However if you send a different file format to this queue, it is converted to a document format acceptable by the printer. This feature is not going away, actually IIUC this will be the only way of printing after print filters are gone from CUPS (moved to printer applications for printers which need them)
  • Raw queue is a queue installed with raw model, so CUPS is not able to get a printer information and options either from IPP or printer driver, so you don't see any options via lpoptions and no filtering happens during printing, regardless of input document format. For raw queues CUPS counts on the application knows what options printer has and sends a print-ready file to cupsd itself, so cupsd here acts only as a transfer layer. This feature is going away in CUPS 3.0.

@zdohnal
Copy link
Member

zdohnal commented Sep 2, 2022

@michaelrsweet ad removing print filters - do we expect applications doing document format conversion to a document format acceptable by a printer? Viewers and tools seem to pass a certain document format regardless of what the destination accepts (evince, firefox, libreoffice send PDFs, okular, xpdf send PostScript, lp just copies in the exact file from CLI), so IIUC printer will print garbage if the file is not converted to a printable format.

F.e. my printer which works with IPP Everywhere (printer from 2010) shows only the following document formats:

document-format-supported (1setOf mimeMediaType) = image/urf,application/pdf,application/postscript,application/vnd.hp-PCL,application/vnd.hp-PCLXL

so without filtering I wouldn't be able to print text files (IIUC).

@tillkamppeter
Copy link
Member

@zdohnal in CUPS 3.x not all filters will get removed, only the ones which are classic printer drivers (like rastertohp) or only serve for obsolete data formats (like pstops) will go away. CUPS 3.x will still have filters which turn typical input formats (PDF as it is sent by nearly every desktop application, Apple Raster to allow iOS devices print on a shared CUPS printer, PWG Raster, and perhaps some others) into the 4 output formats needed by driverless iPP printers, PDF, Apple Raster, PWG Raster, and PCLm.

So desktop app developers should simply send PDF as they are already doing for a longer time. For Okular (or KDE/Qt ingeneral, please test) it is a bug to send PostScript. please report it, these apps have to send PDF, as all the other apps do.

@michaelrsweet
Copy link
Member Author

@zdonal We expect Printers to support standard file formats (Apple Raster, PWG Raster, JPEG, and PDF are specifically called out in the specs), and Clients (CUPS) to convert incoming print jobs to one of those standard formats. Today's applications already support PDF, and we have several choices (with licenses of varying openness) for converting PDF to Apple Raster and PWG Raster. The remaining issue is command-line printing of other formats - plain text, PNG, etc. - but as Till notes we have code for handling that already outside of the traditional CUPS filter chain.

@tresf
Copy link

tresf commented Sep 4, 2022

driverless != raw
Raw queue is a queue installed with raw model, so CUPS is not able to get a printer information and options either from IPP or printer driver, so you don't see any options via lpoptions and no filtering happens during printing, regardless of input document format. For raw queues CUPS counts on the application knows what options printer has and sends a print-ready file to cupsd itself, so cupsd here acts only as a transfer layer. This feature is going away in CUPS 3.0.

This is the part of this change that's the most concerning to me. A lot of printers as well as a lot of applications use the locally installed printers to start raw communication. It could be argued that the printer application should be written to handle this, but currently the respective drivers generally don't. For example, the drivers for Zebra and Epson printers have to be bypassed to talk to them raw, so I don't expect the replacement application to handle this any more gracefully (conversely, the Windows equivalents generally do).

I've seen arguments that developers should just allow raw USB, serial, socket comms themselves, but when it comes to printing, this breaks user-space as users have to know a lot more about how a printer is attached.

The use-case I'm still gravely concerned about, which I'm still unsure how to handle, is the lpr -o raw replacement. Users will continue to buy these Epson and Zebra (Citizen, Boca, Honeywell, Star, etc) printers, but they'll be forced to move to a Windows platform if they can't use it with applications that send raw commands. Although it could be argued that it's not CUPS' (or more generically the printing subsystem) responsibility to provide this communication channel, the industry can't recover from this without a path forward. The push to IPP Everywhere has a lot of benefits, but the removal of a raw communication channel makes CUPS-based platforms dead in these industries the day the change goes live.

Arguments to migrate to the applications that replace this support ignores the fact that in its current form, CUPS can use the PPD or bypass it. The application only seems to replace the PPD, not the ability to bypass it, which is tremendously important to supporting these industry printers moving forward. Even if these printers were to all get thrown into the garbage and new IPP capable hardware replacement were to hit the market, the missing raw commands would have to be added to the IPP specification, which just won't scale with device-spcific raw commands such as "eject cash drawer" or "download barcode font".

Raw printing means sending a file, which printer supports, to a queue, which is driverless or with a printer driver, so no filters are applied - options can be applied depending on which communication protocol you use with printer (f.e. printer supports Postscript and communicates via IPP, so if you send PostScript file via IPP with options, they are applied. However if you do the same with socket or usb backends, the options are not applied, because they don't send options to the printer and count on filters or application to apply them). However if you send a different file format to this queue, it is converted to a document format acceptable by the printer. This feature is not going away, actually IIUC this will be the only way of printing after print filters are gone from CUPS (moved to printer applications for printers which need them)

This sounds so much like a Raw queue that I'm not even sure what the functional difference is. Are you stating that Raw printing should fulfill the (my own) above use-case?

IMHO all distros don't break a live machine if you set this up, since CUPS supports all ways (printer driver, driverless and raw)

Yes, sorry, you're right, apps like LPrint will fill this gap with existing functionality. I'm more concerned about testing the removal of raw queues. Perhaps this is more of a development question, but a quick way to get a system working with PPDs gone and raw queues gone so those relying on CUPS can better understand the impact this change will make.

@michaelrsweet
Copy link
Member Author

@tresf

For example, the drivers for Zebra and Epson printers have to be bypassed to talk to them raw, so I don't expect the replacement application to handle this any more gracefully (conversely, the Windows equivalents generally do).

LPrint implements support for most of the label printers supported by the rastertolabel driver and it supports raw (ZPL/EPL/DYMO) printing.

The key thing is that in the old driver-based architecture you can use the driver (PPD) or tell the printing system you are printing raw (preformatted) data. The same is true for IPP Everywhere, the difference is that you don't have a printer-specific driver, just a generic one for the standard formats required by IPP Everywhere.

@tresf
Copy link

tresf commented Sep 4, 2022

LPrint implements support for most of the label printers supported by the rastertolabel driver and it supports raw (ZPL/EPL/DYMO) printing.

This is fantastic news, thanks.

The same is true for IPP Everywhere, the difference is that you don't have a printer-specific driver, just a generic one for the standard formats required by IPP Everywhere.

This is the part I'd like to know more about. Many of the aforementioned environments don't really need the PPD today (depends whether or not they use the printer from other applications, really), so if there's still a way to add these printers without LPrint (or equivalent) and use them as raw queues (raw printing?), that would be nice to know (even while knowing most standard OS print operations would fail -- as they would today).

There's also the gap of the receipt and line printers (e.g. ESC/POS, ESC/P), so knowing that users will have a path forward is critical.

@michaelrsweet
Copy link
Member Author

@tresf WRT a standard printing system providing standard interfaces to allow any application to print to any printer, there is no place for a printer that can't at least do basic raster printing - less than that and they are basically not printers.

Case in point - those receipt/line printers can do raster printing as well as their basic text printing with hardcoded fonts/character sets (which would be your "raw" printing), so they'll work just fine.

@tillkamppeter
Copy link
Member

@michaelrsweet if the label or POS printers print with the same speed and quality in raster mode (do they?), getting everything rendered on the CUPS server, then one principally would not need raw printing, and one can even better predict what the printer will print. More a problem is to handle commands like opening a cash drawer (really strange that such a thing falls into the responsability of the printing stack).

@michaelrsweet
Copy link
Member Author

@tillkamppeter There are sometimes differences in quality and speed (particularly for old dot matrix printers), but they can absolutely provide a basic level of printing, in addition to "raw" printing when it makes sense to do so.

WRT things like the cash drawer commands (really "pulse pin N"), there has been some discussion of adding an IPP operation for this which can then be discovered and used by a client. However, the push back on that has been on several fronts:

  • Integrated retail POS systems aren't spooling up print jobs - print a line of text, pulse the drawer pin, maybe engage a cutter but most use a tear bar. This isn't what CUPS or IPP is used for.
  • Only a subset of receipt printers provide this functionality, and those that do are bare modules meant for integration.
  • Cash transactions are increasingly rare.

Personally, I feel that if you are going to write a POS application that runs on some (embedded/semi-embedded) Linux distro which interfaces with specific receipt printers, writing code that opens a USB serial or usblp character device to spit out some printer commands using the vendor's SDK is not a difficult task. You don't need (and shouldn't use) CUPS for that, and the lack of raw printing support is not an issue.

@tresf
Copy link

tresf commented Sep 5, 2022

@michaelrsweet if the label or POS printers print with the same speed and quality in raster mode (do they?)

Not even close.

  • Speed is tremendously slower when printing raster
  • Quality can be argued either way, but these low-dpi devices generally have their own optimized fonts that they use, so divergence from these fonts can result in some aliasing or interpolation
  • Some rasterization is usually needed (e.g. company logo), but applications may provide this in the raw data, so that the rest of the content still benefits from the speed and quality of the raw commands

getting everything rendered on the CUPS server

There are many advantages to rendering content, the largest is support for international characters (rendering right to left languages like Arabic on a raw device is extremely difficult) but the two shouldn't be mutually exclusive, I don't understand this philosophy and the industry really hasn't treated the printers this way either.

there is no place for a printer that can't at least do basic raster printing

The lack of raster printing is happening because of the removal of PPD, so this is a bit of cart-before-horse problem. I'm asking for a lifeline for when this removal occurs, not to argue the definition of printing or even the "should CUPS even be used for this purpose" fundamental question.

On Windows, the drivers handle any passthru/raw commands. On CUPS today, this is generally a bit more of a manual workaround (e.g. shell commands or CUPS APIs), but I don't think it's fair to tell the industry they shouldn't be doing this.

Personally, I feel that if you are going to write a POS application that runs on some (embedded/semi-embedded) Linux distro which interfaces with specific receipt printers, writing code that opens a USB serial or usblp character device to spit out some printer commands using the vendor's SDK is not a difficult task. You don't need (and shouldn't use) CUPS for that, and the lack of raw printing support is not an issue.

POS can, will and should run on non-embedded platforms, so I'm not sure why this is even part of the discussion. Yes, there was a time that POS terminals were largely closed, proprietary locked down system, but as businesses need cloud services, this is becoming less and less true. Most POS rus on Windows today but it doesn't need to be this way. Being able to take advantage of a printer as a raster device AND a raw device in CUPS has coexisted just fine, but when the raster drivers are removed, it would be nice to keep these systems at least on life support using raw and not ask them to switch away from a CUPS system for these purposes.

writing code that opens a USB serial or usblp character device to spit out some printer commands using the vendor's SDK is not a difficult task.

I feel minimizing the impact doesn't really help those looking to keep things working. It's really not trivial for apps looking to communicate with many varying devices via USB, Socket and Serial to take over this communication. What's a bit more trivial is for these companies to drop support for CUPS version x.x in general for these purposes such as "Sorry, we no longer support ESC/POS on Mac or Linux".

In regards to the SDKs, some are unavoidable such as Brazil's government-mandated, fiscally-aware Bematech models, but for the vast majority of printers, the SDK isn't needed. This is important for many use-cases, as often the SDK is only offered for Windows anyways.

I don't want to get too off-topic about how these printers should be used but rather understand if there will be viable workaround to keep them working, even in a degraded fashion until a proper printer application is created.

@michaelrsweet
Copy link
Member Author

@tresf WRT print speed, my experience has been that raster printing on thermal printers is just as fast, if not faster, than the "high level" drawing commands for these devices. When it comes to impact printers the reverse is true.

There are many advantages to rendering content, the largest is support for international characters qzind/tray#339 (comment) but the two shouldn't be mutually exclusive, I don't understand this philosophy and the industry really hasn't treated the printers this way either.

WRT not mixing graphics and native text, the problem is simple: the PDF/PostScript imaging model allows for color/grayscale, transparency, rotation, scaling, etc. of text, making the cases where you could actually use native text support very limited. Plus you need to provide desktop fonts that correspond to printer fonts, and provide some hints as to what sizes are supported, etc. Classic Windows GDI printer drivers would use this method to support mixed output, but GDI rendering is much more limited than PDF/PostScript.

The lack of raster printing is happening because of the removal of PPD, so this is a bit of cart-before-horse problem. I'm asking for a lifeline for when this removal occurs, not to argue the definition of printing or even the "should CUPS even be used for this purpose" fundamental question.

PPD files in CUPS serve two purposes: to describe the capabilities and commands needed for PostScript printers (the original Adobe purpose), and to describe the capabilities and filters needed for raster printers (the CUPS extension). Till also wrote the Foomatic wrapper around Ghostscript that creates a sort-of hybrid PPD containing PostScript commands (for Ghostscript) and Ghostscript options/settings that enabled printing via Ghostscript-based drivers, some of which support vector/text drawing (but not for ESC/P devices...) But none of this connects raw printing to PPDs or excludes raw printing because of the IPP Everywhere/raster requirements.

On Windows, the drivers handle any passthru/raw commands. On CUPS today, this is generally a bit more of a manual workaround (e.g. shell commands or CUPS APIs), but I don't think it's fair to tell the industry they shouldn't be doing this.

At no time have we ever said this. What we have said is that we will not support raw-only ("dumb", "proprietary", etc.) queues - we can't manage jobs or monitor printer state for raw queues, and applications cannot print to them without explicit support for the printer's PDL. By requiring basic printing support and allowing raw printing, you get the advantages of both without saddling CUPS with supporting arbitrary crappy raw printers.

POS can, will and should run on non-embedded platforms, so I'm not sure why this is even part of the discussion. Yes, there was a time that POS terminals were largely closed, proprietary locked down system, but as businesses need cloud services, this is becoming less and less true. Most POS rus on Windows today but it doesn't need to be this way.

Traditional POS systems are vertically integrated and largely use embedded/closed software. Cloud POS systems (Square, etc.) are less so, but they also discard things like cash drawers and use network or Bluetooth printers and not bare hardware POS printers.

Being able to take advantage of a printer as a raster device AND a raw device in CUPS has coexisted just fine, but when the raster drivers are removed, it would be nice to keep these systems at least on life support using raw and not ask them to switch away from a CUPS system for these purposes.

I think you are completely missing the point. Raster driver support is being removed from CUPS, but they can still be used with Till's wrapper printer application. And of course CUPS 2.x source code will be available for a long time to come if you are really stuck in a world that cannot adapt to new technologies.

@tresf
Copy link

tresf commented Sep 6, 2022

Thanks for the pointer to Till's wrapper...

So to take two use-cases, please correct me if any of these assumptions are wrong:

  1. Sending ZPL to a Zebra printer (moving forward will use LPrint)
    • ✅ Support for 2D/raster content will work; sending raw ZPL should also work.
  2. Sending ESC/POS to a receipt printer (moving forward will use ghostscript-printer-app
    • ❓ This is partially what the above conversation is about, because the PPD files are maintained by a 3rd party... I seems like ghostscript-printer-app is being proposed... Since ghostscript-printer-app is being distributed as a snap, I assume it can't locate the old PPD files on the machine, is there a migration procedure for the old raster apps that shipped by the manufacturers? If so, will the old PPD location stick around for a while so that the installers can continue to place them there for IT technicians to find (and then copy to the wrapper)? Or instead will technicians be expected to extract these on an old CUPS 2.x system and copy the PPD files to the wrapper?

Some notes on comments above:

Cloud POS systems (Square, etc.) are less so, but they also discard things like cash drawers and use network or Bluetooth printers and not bare hardware POS printers.

I can assure you, this is simply not true. Cloud apps leverage the raw stuff too. I'd rather not get into an argument here about it, but there are hundreds of cloud-first applications that I work with directly that leverage these types of printers and use the printer's raw language as well as some other 1, 2 products that do the same.

At no time have we ever said this. [...] By requiring basic printing support and allowing raw printing, you get the advantages of both without saddling CUPS with supporting arbitrary crappy raw printers.

My questions may be mal-informed, but there's no mal-intent. I'm trying to determine if CUPS will continue working with these devices or not and even raw comms are enough for many users.

if you are really stuck in a world that cannot adapt to new technologies.

I'll try to not take this as a personal attack. The raw stuff isn't going away (and based on all of the talk about "raw" above, it sounds like this is understood). What I'm trying to figure out (and others will too, but possibly AFTER this change has occurred) is what things will look like when support has been removed. If tomorrow the printer manufacturers wrote their own printer applications to bridge the gap that would be a solution, but historically, the manufacturers' support for CUPS has been slow and lacking (when compared to Windows), so it's important to understand what will work and what will not and what new steps users will need to take.

Although CUPS has a fundamental "no raw queues" philosophy, it doesn't mean people that are still looking to use it are unwilling to adapt to change. For example, I worked with a French printer company "Evolis" to get some raw commands working with their hardware and its important to know what is the path forward (if any) when we can't setup a raw queue for these.

supporting arbitrary crappy raw printers.

"Support" is the operative word here. I'm only asking for a communication channel as is available in CUPS today (albeit deprecated) and Windows.

@tresf
Copy link

tresf commented Sep 6, 2022

Is there a quick way to test this new functionality (e.g. a Linux distro that makes this pretty easy to setup and test without breaking a live machine)?

In short (please correct if any of this is wrong):

  1. Use CUPS 2.2.8+, e.g:
    • MacOS 13.0 Beta 6 ships with 2.3.4
    • Ubuntu 22.04 ships with 2.4.1
  2. Move any affected printers to a printer application

I'll provide any feedback on this process to the appropriate projects.

@OdinVex
Copy link

OdinVex commented Sep 19, 2022

...? I don't know much about CUPS or anything, but without PPDs, no printer I've ever used will ever show levels of ink to CUPS. Not to mention I never get the options to "Print Self-Test Page" or "Clean Print Heads” (last option broken since my Mint 21 update). Not sure why PPDs should be removed, given the trade-off of losing ink-level monitoring. Edit: I just want my IPP/IPPS and ink-levels. shrug

@tillkamppeter
Copy link
Member

The issue with printer applications is that some manufactures just haven't adopted it yet.

Has it ever occurred to you that some people might not want to run binary blobs direct from the manufacturer? Or even off-distro "open source" software direct from the manufacturer? I have no desire to have hundreds of MB of bloated spyware drivers, control centres, and whatnot on my system. No, not even in some sort of container.

We (especially the Printer Working Group (PWG) but also OpenPrinting) have done everything that this is not needed, asthere are standards for driverless printing, IPP Everywhere, AirPrint, Mopria, and many manufacturers follow them (in their hardware):

https://openprinting.github.io/printers

There are near 10000 know, driverless models.

Unfortunately, there are always manufacturers who think that this is not needed, or do not want it, to force their users to install their software (bloatware, spyware, ...). And printer manufacturers are big companies, one department building a nice driverless printer and another disguising it as non-driverless, to get the user install their software, which make the users pay a lot for original ink cartridges:

https://openprinting.github.io/OpenPrinting-News-June-2024/#most-hated-gets-loved-under-linux

Printer Applications, as a software emulation of a driverless printer to make a non-driverless printer work, replace printer drivers. But as most modern printers are driverless, so you only need to choose the right printer ...

@fallenguru
Copy link

We (especially the Printer Working Group (PWG) but also OpenPrinting) have done everything that this is not needed

No, on the contrary. By pushing (self-contained) "printer applications" and citing manufacturer adoption as a roadblock (therefore putting the ball in the manufacturer's lap) you're legitimising the practice of shipping such abominations instead of adhering to open standards. I realise that CUPS is multi-platform, but I'm a Linux user. And one of the neat things about Linux is that manufacturer's drivers are easily avoided, because everything is geared towards having devices either adhere to a standard or get the drivers included somewhere upstream. This change is promoting a more Windows-like printer driver paradigm.

[most modern printers are driverless]

Not really. Maybe many have some half-hearted IPP support, just enough that they can put it on the spec sheet without getting sued too much, but driverless IPP as a first class (all features accessible, on all ports, no bugs) method to access a printer is not ubiquitous.

you only need to choose the right printer ...

Brilliant. Problem is, nobody advertises driverless IPP, let alone as a prominent feature. Nor IPP Everywhere, for that matter. https://www.pwg.org/printers/ lists a grand total 1324 printers for me, currently. (AirPrint may be a de facto standard, but IDK how open it is, really, and how well it interoperates with other implementations.) So choosing the right printer has become hard, when it used to be as simple as getting something with support for PS (or later PDF).

@michaelrsweet
Copy link
Member Author

[most modern printers are driverless]

Not really. Maybe many have some half-hearted IPP support, just enough that they can put it on the spec sheet without getting sued too much, but driverless IPP as a first class (all features accessible, on all ports, no bugs) method to access a printer is not ubiquitous.

OK, so any printer with the AirPrint logo on the box supports IPP and all key printing functionality via IPP. The certification tests are quite involved...

...
Brilliant. Problem is, nobody advertises driverless IPP, let alone as a prominent feature. Nor IPP Everywhere, for that matter. https://www.pwg.org/printers/ lists a grand total 1324 printers for me, currently. (AirPrint may be a de facto standard, but IDK how open it is, really, and how well it interoperates with other implementations.)

AirPrint is IPP + Bonjour (mDNS and DNS-SD) + Apple Raster (a variant of CUPS/PWG Raster), JPEG, and (optional) PDF. So if the box has an AirPrint logo, it will work as a driverless printer with full functionality.

@amarihw
Copy link

amarihw commented Jul 13, 2024 via email

@michaelrsweet
Copy link
Member Author

What is going to be the replacement functionality for interface files?

There is no official replacement.

Is it going to be possible to apply programming to each and every print job per printer queue with bash/shell scripting? And if not, why?

The short answer is security - allowing arbitrary software to be loaded (potentially from a remote client) onto a server with few (if any) restrictions on the run-time environment is a bad idea.

That said, it is possible you will be able to make use of the accounting/managed printing interfaces (still to be defined - we don't have a lot of resources/developers to do this stuff) to extend things at your sites. But the focus is 100% on standard formats and not on legacy/proprietary formats like PostScript and ZPL.

@tresf
Copy link

tresf commented Jul 13, 2024

What is going to be the replacement functionality for interface files? We produce over 18million postscript and ZPL print jobs a year to over 50sites. We use interface files fairly substantially to apply logic, controls, filters and additional features that our ERP systems don't handle (printing wise) out of the box (like on the fly changing the word 'foo' to 'bar'). Is it going to be possible to apply programming to each and every print job per printer queue with bash/shell scripting? And if not, why?

I asked a similar question here: #4 (comment)

This was @michaelrsweet's response:

WRT CUPS, yes it is going to lose the ability to host a raw queue. Tools like "ippeveprinter" could still be used to setup individual "raw" printers that CUPS could then print to, but honestly by the time we are ready to release CUPS 3.0 there shouldn't be any printers that don't have a corresponding printer application. It's not like we are charging forward on a whim, this has been in the works for years (I first started talking about it with printer vendors in 2010) and we want to make sure we have everything in place before we "throw the switch".

I too am concerned about this. What's particularly off-putting are terms like "vertical market printers", which suggests that these are "special needs" customers, quoting [source: investopedia]:

"A vertical market is a business niche where vendors provide goods and services to a specific group of customers or industry with specialized needs. [...]"

This buzzword sounds very reasonable at a glance, but just to focus on ZPL alone, shipping services such as DHL, FedEX, UPS, USPS, Royal Mail still offer ZPL as a label format because it's smaller, faster and more reliable than an PDF. Where the term "vertical market" starts to go sour is when we look at services such as Amazon Vendor Services, where SOHO users on Mac, Linux and Windows are printing their own labels. There was once a time where shipping was largely a "vertical market", but now raw printing is in the hands of millions of end users.

The way that Windows handles this problem -- in the scope of ZPL -- is that the Zebra driver -- written and maintained by Zebra Technologies -- detects the raw data and sends the ZPL directly, however the ZPL driver in CUPS doesn't have this feature, requiring some form of raw bypass -- either on the application level -- or on the OS level.

My fear, and I've stated this before, is that removal of raw communication will force these users to switch to Windows.

Note, although ZPL is used as an example, this raw functionality is still very popular for other formats (CPCL, EPL, ESC/POS, FGL, JSCRIPT, StarPRNT).

With regards to the bloatware/spyware claims, this is largely untrue for the industry printers and drivers and -- in my experience -- is more of a problem with gigantic software bundles provided with MFPs. I think it's important to quantify any arguments for or against the adopt of IPP when calling out vendors because not all drivers are intended to force the purchase of ink cartridges -- especially those printers (e.g. label, receipt, ticket) that don't use ink.

Also -- unrelated, but great news -- HP has FINALLY backpedaled on this: https://www.tomshardware.com/peripherals/printers/hp-discontinues-online-only-laserjet-printers-in-response-to-backlash

Lastly, as an advocate for keeping RAW, I want to be careful to NOT sound minimizing and NOT sound unappreciative to AirPrint/IPP Everywhere. Everyone I encounter really enjoys using AirPrint and avoids the printer driver whenever possible. For example, I've been a Windows ARM user for several years and this provides a way to use hardware that may never receive an official driver from the manufacturer. This is revolutionizing and simplifying the way we print. That said, I just really, strongly feel that RAW printing will be needed side-by-side this functionality.

@michaelrsweet
Copy link
Member Author

@tresf

I too am concerned about this. What's particularly off-putting are terms like "vertical market printers", which suggests that these are "special needs" customers, quoting [source: investopedia]:

"A vertical market is a business niche where vendors provide goods and services to a specific group of customers or industry with specialized needs. [...]"

This buzzword sounds very reasonable at a glance, but just to focus on ZPL alone, shipping services such as DHL, FedEX, UPS, USPS, Royal Mail still offer ZPL as a label format because it's smaller, faster and more reliable than an PDF. Where the term "vertical market" starts to go sour is when we look at services such as Amazon Vendor Services, where SOHO users on Mac, Linux and Windows are printing their own labels. There was once a time where shipping was largely a "vertical market", but now raw printing is in the hands of millions of end users.

So really, relying on a vendor PDL for a mission-critical printing application (shipping) has never seemed a safe result to me. ZPL compatibility between printers/vendors is not 100%, and I've done enough development and support over the years to know that "raw printing" causes as many problems as it solves.

All of the major shipping companies support PNG label images (and honestly most PDF output consists of an image embedded in the PDF with some extra text added around the label), which is why LPrint is so keen on supporting it as well as ZPL, EPL, etc.

The way that Windows handles this problem -- in the scope of ZPL -- is that the Zebra driver -- written and maintained by Zebra Technologies -- detects the raw data and sends the ZPL directly, however the ZPL driver in CUPS doesn't have this feature, requiring some form of raw bypass -- either on the application level -- or on the OS level.

CUPS absolutely supports this, it is just that auto-detecting ZPL as a general thing is not particularly accurate - you can add MIME typing rules to your CUPS system to get it to auto-detect your particular files, and LPrint also has specific detection rules for each type of label printer it supports.

My fear, and I've stated this before, is that removal of raw communication will force these users to switch to Windows.

I don't see this myself, as Windows is not particularly reliable and is different enough to make porting whole systems from Linux/Unix. Also, LPrint exists and supports the most common label printers already.

Note, although ZPL is used as an example, this raw functionality is still very popular for other formats (CPCL, EPL, ESC/POS, FGL, JSCRIPT, StarPRNT).

CPCL, EPL, and ZPL are already covered by LPrint.

ESC/POS (and its replacement) is coming in a future LPrint release.

It isn't clear whether adding FGL, JSCRIPT, or StarPRNT support is worthwhile since I've had no requests for them.

But as I keep stating, "raw printing" isn't something we will support going forward - it simply doesn't scale and isn't at all secure for a printing system that has to support a wide variety of printers. Printer Applications are intended to bridge the gap to safely allow support for vendor PDLs while also providing support for standard PDLs so that all applications can make use of a printer, not just an esoteric site/vendor application that is unable to properly support the printers it uses.

@tresf
Copy link

tresf commented Jul 13, 2024

So really, relying on a vendor PDL for a mission-critical printing application (shipping) has never seemed a safe result to me.

Well, you're entitled to your opinion, but I think the term "safe" is quite misleading here. What I believe is being inferred is that the contract between the application and the printer is broken when you largely bypass the queue status information. In that regard, I wholeheartedly agree. Application developers need to put special code in place to handle queue events and this experience is sub-optimal.

However, this "safety" is so very misleading of a word because despite the hardware variances in PDL handling (not to mention DPI variances), this must be weighed in an organization against the performance and reliability costs of formats such as PDF or raster and in my experience, this decision is best left up to the organization to decide.

I'm admittedly biased in this dialog -- and for that reason, I apologize for repeating several points over and over -- but I work with 800 companies across millions of PCs to help with these decisions and after testing and analysis, ZPL is often preferred. While I do provide these pros/cons to customers, I do not make the decision for them. As long as there's a choice, I allow the users to make an informed decision.

ZPL compatibility between printers/vendors is not 100%, and I've done enough development and support over the years to know that "raw printing" causes as many problems as it solves.

Some will say IPP Everywhere/AirPrint suffers an identical problem! I'm trying to be civil here, but it's taking a long time for these vendors to have feature-parity even when it DOES work. Out of politeness, I mention the times it DOES work, but in many cases, we've reverted from AirPrint/IPP Everywhere to the vendor drivers because of a botched implementation -- hundreds of gibberish printouts for a single PDF because the AirPrint support mysteriously breaks on certain desktops. This also ignores the fact that the mappings just stop working under certain scenarios (removing and re-adding the printer often fixes this). There's also an auto-magic problem users are dealing with since the tech can work without IP Addresses as all. But I digress, the point of this conversion is NOT to talk about the shortcomings of AirPrint and friends -- that will get better with time -- but rather to talk about the impact of removing filters.

CUPS absolutely supports this, it is just that auto-detecting ZPL as a general thing is not particularly accurate - you can add MIME typing rules to your CUPS system to get it to auto-detect your particular files, and LPrint also has specific detection rules for each type of label printer it supports.

Thank you for sharing this information. I've yet to test LPrint, but if it handles this scenario rather gracefully, it would be a huge improvement over the existing driver.

I don't see this myself, as Windows is not particularly reliable

Anecdotal, but most organizations use Windows for it's reliability in printing. This is often due to the fact that device drivers are more prominent there, but I think "reliable" is misleading. In my experience, Windows is a very reliable printing experience, but often the stacking of 20 years worth of device drivers causes system and printer instabilities, but I believe this to be orthogonal to the OS and rather simply a reason for adopting AirPrint/IPP Everywhere, NOT a reason to drop support for raw queues and not related to the usefulness of raw queues as they continue to exist on other OSs (rather manually added raw queues or via driver detection).

If CUPS continues to support the raw queues -- albeit through a different pipeline -- I agree, this won't be a reason to switch. I owe it to you and the OpenPrinting team to test the feasibility of this approach.

raw printing" isn't something we will support going forward
LPrint also has specific detection rules for each type of label printer it supports.

It's these types of statements that add confusion. Is the intent to ONLY allow raw printing when a specific printer application has been coded to accept it? It sounds like support will be maintained -- in some form -- through these applications. What's not obvious is what options are available when these applications aren't available.

Regarding FGL, Ticketmaster and Livenation use this format, so it's still a rather "vertical industry" so they'll probably just keep using Windows, but there also a lot of smaller ticketing companies that use it that would be more inclined to use a CUPS solution if one were available. Still "vertical" -- since it's just much less likely for an individual (e.g. SOHO) to own FGL-capable of hardware, whereas label printers are becoming more and more common.

@michaelrsweet
Copy link
Member Author

@tresf

However, this "safety" is so very misleading of a word because despite the hardware variances in PDL handling (not to mention DPI variances), this must be weighed in an organization against the performance and reliability costs of formats such as PDF or raster and in my experience, this decision is best left up to the organization to decide.

You do realize that malicious ZPL can brick your printers, right?

But in any case, I don't think we disagree on who should choose the format to use - LPrint was explicitly designed to support both vendor PDLs and standard PDLs for this very reason, while providing standardized status monitoring, etc. "Raw" support cannot come at the expense of standard printing, and since the raw applications are unwilling or unable to develop their own printing infrastructure, this is the best compromise I can provide.

raw printing" isn't something we will support going forward
LPrint also has specific detection rules for each type of label printer it supports.
It's these types of statements that add confusion. Is the intent to ONLY allow raw printing when a specific printer application has been coded to accept it? It sounds like support will be maintained -- in some form -- through these applications. What's not obvious is what options are available when these applications aren't available.

So to be explicit about this:

  1. Raw queues (no PPD/driver, no interface script) are going away forever in CUPS 3.0.
  2. The "-oraw" option is going away forever in CUPS 3.0 (printers often need to know what you are sending them and auto-detection is unreliable...)
  3. Support for specific vendor PDLs (as supported by the printer) will continue.
  4. Auto-detection of vendor PDLs is unreliable but both CUPS and the various printer applications will (continue to) try.
  5. Manual specification of the vendor PDL ("-o document-format=application/vnd.FOO-BAR") works today and will continue to work in the future.

The goal is simple: all CUPS printers can be used by any application. If you have an application that supports a/the native PDL of a printer, it can use it, but otherwise it uses a standard PDL to still be able to print.

Regarding FGL, Ticketmaster and Livenation use this format, so it's still a rather "vertical industry" so they'll probably just keep using Windows, but there also a lot of smaller ticketing companies that use it that would be more inclined to use a CUPS solution if one were available. Still "vertical" -- since it's just much less likely for an individual (e.g. SOHO) to own FGL-capable of hardware, whereas label printers are becoming more and more common.

Last time I checked there are only two vendors left supporting FGL - Boca Systems and Microcom Corporation. If someone provided a printer or other support then I could probably add it to LPrint pretty quickly. But like I said I've had nobody ask or offer... :/

@tresf
Copy link

tresf commented Jul 14, 2024

I'll reach out to BOCA today with this feedback. I've been in communication with them over the years... here's a quote from 2023:

Click to expand

My name is [John Doe] I am a senior software engineer with Boca Systems [...] It appears that you offer a library that allows for sending raw print data but do you also offer a product that uses the installed print driver to format the ticket data as well? We have customers who use both methods depending on how much control they want to exercise over the printing process but struggle when trying to do this from a web app. [...]

I'll link them to this bug report to see if they have resources to help get support added.

@poscat0x04
Copy link

Raw queues used for shared printers require client software to talk directly to the server to get the printer capabilities, which breaks when sandboxing/AppArmor/SELinux is used

btw I'm using epson-inkjet-printer-escpr just fine on fedora (which has selinux enabled).

PPDs and drivers have been holding us back from offering better user experience (ready media, localization, full range of printer options/values), improved document processing, and improved accounting

Are there any reasons CUPs can't support printer filters/drivers with degraded functionality? Surely degraded functionality is better than not supporting it at all?

@michaelrsweet
Copy link
Member Author

@poscat0x04

Raw queues used for shared printers require client software to talk directly to the server to get the printer capabilities, which breaks when sandboxing/AppArmor/SELinux is used

btw I'm using epson-inkjet-printer-escpr just fine on fedora (which has selinux enabled).

Not applicable - a SHARED raw printer requires the client application to directly communicate with a remote server, but some environments restrict outgoing network connections for security reasons (depending on the application, etc.) The local spooler, however, has networking enabled and can indirectly provide access to such things. In the current CUPS 2.x architecture, that requires a local PPD and queue.

PPDs and drivers have been holding us back from offering better user experience (ready media, localization, full range of printer options/values), improved document processing, and improved accounting

Are there any reasons CUPs can't support printer filters/drivers with degraded functionality? Surely degraded functionality is better than not supporting it at all?

There is a history of bugs, regressions, and security issues around raw queues. And it is not just “degraded” functionality but outright broken functionality when an application sends a file that the printer cannot understand and spews out a whole ream of paper with random characters on it.

Is there a reason why you need a “raw queue” and need CUPS to support that functionality? I understand the need (in some cases) to support sending a proprietary file format to a printer (and that functionality will remain), but why do you need a queue for a printer that otherwise cannot be used?

I am reminded of a hack an Apple engineer did very early in the development of macOS - that engineer wrote an interface script that played music files and had CUPS function as a primitive jukebox. It was certainly an interested trick but was not a very useful or efficient solution that could (and did) break with a change to the printing system or OS security model. Using CUPS to spool files to be sent to a random hardware device with no safety checks, processing, or status monitoring is just asking for trouble. It has never worked well, and has frequently resulted in unexpected behaviour even for the people setting the queues up.

@tresf
Copy link

tresf commented Jul 16, 2024

I'll reach out to BOCA today with this feedback. I've been in communication with them over the years...

@michaelrsweet Re: FGL drivers and raw support: BOCA replied with gratitude, and asked for me to share their engineer's email address with you. I've forwarded the information to you via email (gmail) and will reply to them with the same.

@amarihw
Copy link

amarihw commented Jul 17, 2024 via email

@michaelrsweet
Copy link
Member Author

@amarihw

Thanks Michael / Tres / poscat. Is there any scope for the insane idea of simply having the "new CUPS 3.0" code along-side the current code (but being the default code), but then 'cupsd.conf' having a flag in it like: useOldCode = yes/no and then those of us that want to continue using the 'insecure code' can do so whilst not either having to move to a port of CUPS or having to manually maintain an older version of CUPS to run outside of package managers etc?

No.

It isn't like the new code will contain the old code. If you want to continue getting the CUPS 2.x experience, you need to continue running CUPS 2.x.

@amarihw
Copy link

amarihw commented Jul 18, 2024 via email

@fallenguru
Copy link

fallenguru commented Jul 24, 2024

OK, so any printer with the AirPrint logo on the box supports IPP and all key printing functionality via IPP. [...] AirPrint is IPP + Bonjour (mDNS and DNS-SD) + Apple Raster (a variant of CUPS/PWG Raster), JPEG, and (optional) PDF. So if the box has an AirPrint logo, it will work as a driverless printer with full functionality.

It that case, OpenPrinting/CUPS has a marketing problem. Because I didn't know that, and I've been working with computers and printers for over 30 years and I consider myself fairly technical. I also even did some reading recently because I'm in the market for a new printer.

AirPrint, to me, meant "proprietary Apple tech" (read: irrelevant for me as someone who doesn't use Apple products) and "printing tech, primarily wireless, for (Apple) phones and tablets" (read: irrelevant for me as a user of Ethernet-connected PCs). Even if printing from a phone/tablet were a significant use case, my smartphone runs Android ... To add to that, the manufacturers' documentation suggests that AirPrint is for mobile devices and functionality is restricted vs a full driver.

How is anyone supposed to know that you're supposed to check for AirPrint, that it is preferred, if you want good Linux compatibility?

Someone linked https://openprinting.github.io/printers above. Great. Except the main OpenPrinting site (https://www.openprinting.org/printers) doesn't have that. The printer listing there doesn't have much in the way of recent models, and that's that. I realise now that that is because that only lists printers that aren't driverless, but, again, how is anyone supposed to know?

@michaelrsweet
Copy link
Member Author

@fallenguru

OK, so any printer with the AirPrint logo on the box supports IPP and all key printing functionality via IPP. [...] AirPrint is IPP + Bonjour (mDNS and DNS-SD) + Apple Raster (a variant of CUPS/PWG Raster), JPEG, and (optional) PDF. So if the box has an AirPrint logo, it will work as a driverless printer with full functionality.

It that case, OpenPrinting/CUPS has a marketing problem. Because I didn't know that, and I've been working with computers and printers for over 30 years and I consider myself fairly technical. I also even did some reading recently because I'm in the market for a new printer.

OpenPrinting has nobody to do marketing, and the current participants/contributors are primarily technical people/geeks. We do try to point out that any IPP Everywhere or AirPrint printer will work in differently places, but I'm sure we can do a better job of this.

AirPrint, to me, meant "proprietary Apple tech" (read: irrelevant for me as someone who doesn't use Apple products) and "printing tech, primarily wireless, for (Apple) phones and tablets" (read: irrelevant for me as a user of Ethernet-connected PCs). Even if printing from a phone/tablet were a significant use case, my smartphone runs Android ... To add to that, the manufacturers' documentation suggests that AirPrint is for mobile devices and functionality is restricted vs a full driver.

That's something I wish Apple would push harder to change - AirPrint certification requires driver parity, both for (printer) features and performance, so either those vendors are trying to push their own solutions or are lying to Apple. Certainly within the Printer Working Group we worked for over a decade (and continue to work) to ensure that IPP offers driver parity and support for all printer features.

...
Someone linked https://openprinting.github.io/printers above. Great. Except the main OpenPrinting site (https://www.openprinting.org/printers) doesn't have that. The printer listing there doesn't have much in the way of recent models, and that's that. I realise now that that is because that only lists printers that aren't driverless, but, again, how is anyone supposed to know?

It doesn't help that we have the old OpenPrinting site (that's the openprinting.org URL) and the new OpenPrinting site (hosted by Github) - the old site supports Foomatic. @tillkamppeter can we get a link on the old OpenPrinting printer driver site to the new site/supported printers page to make it clear that most printers sold since 2010 don't need a driver?

@fallenguru
Copy link

Case in point, Epson ET-8550. Judging by this in-depth review at least—scroll down a bit—you do not want to be using AirPrint with it, not even on a Mac. But I don't actually have this printer yet, I'm just interested enough to have done a little research, with the result that to use it on Linux, you want TurboPrint ...


I do actually have a Kyocera PA4500cx as of this afternoon. It's a 2023 colour laser advertised as having AirPrint, Mopria, and IPP 1.0. It pops up immediately on my desktop (Ubuntu 22.04 LTS) and it does print, but toner levels aren't reported (I'm not sure anything is), and the options are beyond barebones: I can select colour/B/W and duplex, but not the output tray, and nothing related to print quality. No KIR, EcoPrint, not even the resolution, ... There's a "print quality" setting, but that has only "normal". And the options that are there are a mix of English and localised.

Obviously this doesn't belong here—should I file another GitHub issue on this repo?—this is just meant to illustrate that "all new printers can do driverless" simply isn't true, at least not plug & play fashion.

@tillkamppeter
Copy link
Member

The Kyocera PA4500cx seems to have a rather crippled reporting of its capabilities and user-settable options via IPP. @michaelrsweet does the IPP-Everywhere certification has some facility to make printers like this fail?

@michaelrsweet
Copy link
Member Author

@fallenguru Please reach out to EPSON and Kyocera, respectively - the issues you'e identified require a firmware update. Both EPSON and Kyocera are members of the Printer Working Group and Mopria, and are AirPrint licensees. The lack of resolution and tray options is a hard fail for the AirPrint certification process/tools (and probably Mopria, but I don't have access to those tools). The current IPP Everywhere self-certification tools only test for what is reported via IPP - the 2.0 tools will do a little better (asking what features should be present before running the tests) but ultimately that certification program depends on the honesty of the printer vendors (which honestly is normally pretty good...)

WRT the EPSON printer and its driver, it isn't clear from the review which options are missing. AirPrint has a hard requirement for "driver parity", at least for printer-based functionality (i.e. not some special EPSON color management/image enhancement/etc. garbage). Based on the screenshots it looks like the driver provides the usual EPSON color management/enhancement controls which actually make it harder to get a consistent print...

EcoPrint and KIR are vendor-specific options that haven't been standardized so while they could be exposed via IPP it isn't likely that the current generation of client UIs will expose them. (CUPS itself can handle them but the options will only appear if the UI looks them up...)

@fallenguru
Copy link

fallenguru commented Aug 13, 2024

Please reach out to EPSON and Kyocera, respectively

😂 @michaelrsweet, I'm a mere end user who owns 0 Epson printers and 2 Kyocera ones. I haven't a snowball's chance in hell of getting anywhere via the support channels available to me. Strictly speaking, my use case (driverless printing on Linux via AirPrint) isn't even supported, and I assume the full-fat software suite (which I'd very much prefer not to have to use) has access to all features.

Mind you, if anyone has a contact in the right department at Kyocera I'll happily go bug them.

P.S. Can't even find a support inquiry form, let alone an e-mail address; I'll try their phone support in the morning, but ...

EcoPrint and KIR are vendor-specific options that haven't been standardized so while they could be exposed [...]

My 20-years-old Kyocera B/W laser has both, and I can toggle them just fine. Granted, it doesn't work driverless, but once you feed CPUS a PPD everything works.

Besides, nearly all printers have some vendor-specific features—as long as those can't be expected to be supported, "driverless" will never be a first-class driver.

the issues you'e identified require a firmware update.

Realistically, these issues require that I take apart the various driver packages until I find a PPD file. Or is there an easier way?

@fallenguru
Copy link

[...] vendor-specific options that haven't been standardized [...] could be exposed via IPP [but] it isn't likely that the current generation of client UIs will expose them. (CUPS itself can handle them but the options will only appear if the UI looks them up...)

So correct me if I'm wrong, but I read this as "driverless IPP, even when implemented correctly by the printer manufacturer, will only present a standardised set of options to the user"? If so I don't get how anyone thought that this was a good idea, that this could ever work beyond basic printing, i.e. as a first-party driver. Every printer I've ever seen has some vendor, or even model-specific options. And no, not just fluff.

We've had driverless printing for decades. It's called buying a postscript printer and using it with a suitable PPD file. If there's an option in the PPD, the UI will expose it, no matter how arcane. Customisation possible as needed by editing the PPD.

I get wanting to get rid of that step as well, host a default PPD [or equivalent information] on the printer, brilliant idea, but firstly "default" is the operative word here: Don't take away our ability to tweak/fix that default. And secondly don't restrict what features a printer can advertise, or which of these features will be exposed. Else all this is a giant step back.


Re. the Kyocera PA4500cx situation: The PPD included with the Linux package doesn't work (it defines custom binary filters for whatever reason, and removing that breaks printing, even though the printer takes PostScript 3 just fine as-is), but the PPD included with the KPDL mini driver does. Yay, usable printer!

To add insult to injury the only access method that consistently works is socket:// ...

I'd appreciate pointers re. where to take this, both as a support question (workarounds, how to get the most out of this printer using CUPS, etc.) and regarding further steps. Is there a way to alert Apple to the fact that the AirPrint implementation is lacking, for example? Looks like it affects the entire PA/MA series, too.

@michaelrsweet
Copy link
Member Author

@fallenguru

So correct me if I'm wrong, but I read this as "driverless IPP, even when implemented correctly by the printer manufacturer, will only present a standardised set of options to the user"? If so I don't get how anyone thought that this was a good idea, that this could ever work beyond basic printing, i.e. as a first-party driver. Every printer I've ever seen has some vendor, or even model-specific options. And no, not just fluff.

First, 99.9% of all printer features have been standardized in the Printer Working Group - we've been working on that for over two decades and the last little bits are just really hard to standardize because that last 0.1% of features are not well defined and/or are covered by patents and trademarks that those companies don't want to standardize. Not all print clients can expose all of the attributes/values, because honestly there is a LOT more complexity in IPP than most users will ever need, but the capability is there.

Second, IMHO those last few features are just fluff - "resolution enhancement" was important when we were sending 300dpi bitmaps to laser printers but not anymore. "EcoPrint" aka "ink/toner saving" could potentially see some standardization in the future but right now we recommend vendors activate this mode for "draft" printing since all IPP clients support the "print-quality" attribute and "draft printing" semantically reflects the intent that you want a legible print that uses less ink/toner. The remaining color effect and imposition/scaling options are better handled by the Client, which has many times the resources of the printer - you don't need the printer to remove red-eye, make the sky bluer, or lay out your photos in a thumbnail grid.

We've had driverless printing for decades. It's called buying a postscript printer and using it with a suitable PPD file. If there's an option in the PPD, the UI will expose it, no matter how arcane. Customisation possible as needed by editing the PPD.

There is a reason everyone has moved past PostScript and PPDs - the language doesn't support modern graphics or typography, and the description files don't support modern printer features/capabilities.

PostScript had its day, but if you look under the covers in CUPS you'll notice that there are a LOT of workarounds in there for PostScript interpreter bugs and limitations in the language itself. PostScript was barely enough when I adopted PPDs for what eventually became CUPS in the early 90's. And if you think PPDs can express every driver option, you haven't used a Mac where vendors famously pass hundreds of arcane non-PPD options from their PDEs - that fact required extensive engineering by the Apple printing team to make vendor UI work in the print dialog over the years, where the driver might even be old PPC or Intel code that had to run via emulation...

Is there a way to alert Apple to the fact that the AirPrint implementation is lacking, for example? Looks like it affects the entire PA/MA series, too.

Send email to "airprint@apple.com".

@fallenguru
Copy link

fallenguru commented Aug 16, 2024

First, 99.9% of all printer features have been standardized in the Printer Working Group

You wrote above that EcoPrint isn't standardised. That's just Kyocera's marketing name for their toner saving mode. What general purpose printer doesn't have a toner (ink) saving mode? You write below that draft mode is supported, is that defined as "trade quality for speed", then, as opposed to "trade quality for lower consumables use"? I mean, lump them together if you must, or map them to print quality profiles, but this isn't .1 % functionality, this is basic functionality.

Not all print clients can expose all of the attributes/values, because honestly there is a LOT more complexity [...]

Why? What specifically is complex about attribute/value pairs? Ok, for ones that aren't standardised everything needs a label, too, and for that i18n would be nice, but other than that? "It's complicated" just sounds like hand-waving.

Those last few features are just fluff.

So here's the major (for me) features my Kyocera's AirPrint implementation is lacking: resolution, toner saver mode, gloss mode, colour reproduction profile, print greys with black toner, a slew of colour adjustments. How many of those are Kyocera's fault and how many are "not standardised"?

Look, you're pushing for supporting only driverless IPP going forward, arguing that every printer fully supports that (if only in the guise of AirPrint) these days. That's clearly wishful thinking. I've given two counter examples: The Epson EcoTank line are very popular home/SOHO inkjets, and my Kyocera qualifies as a business printer. Spend ten minutes googling and you'll find that AirPrint not having feature parity is a very common complaint.

Reading your responses it's not just the vendors' fault, not just an issue of a few black sheep. They like to differentiate their offerings via the driver, and I don't mean branding them (that, too, of course), but via the print options they offer. Going by the reviews people like Epson's colour management "garbage" and consider it a selling point.
In a driverless printing model where everything is standardised, printers can't have unique (driver) features any more, or unforeseen ones. No wonder vendors stick to their driver packages and relegate driverless to "basic functionality so you can print an invoice from an iPhone in a pinch". I can't blame them.

you don't need the printer to remove red-eye, make the sky bluer, or lay out your photos in a thumbnail grid.

I wholeheartedly agree. But most applications don't have colour adjustment support. So being able to tweak the colours at the printer/driver level, if, say, blue comes out with a violet tinge, is valuable. I'm also finding the text+graphics / text+photo / ... / line art settings that the PPD has useful.

There is a reason everyone has moved past PostScript and PPDs

They have? Canon aside every laser I looked at had PS support still.

The language doesn't support modern graphics or typography, and the description files don't support modern printer features/capabilities.

Now that's just disingenuous. Postscript is Touring complete, it can do Everything™. I even wrote a game in it at uni decades ago. And, yes, it ran fine on actual printers. The only argument against postscript that I can see is not wanting a Touring complete print language in the first place for security reasons. Though I suspect PDF isn't much better and the idea of sending everything rasterised doesn't appeal at all.

if you think PPDs can express every driver option, you haven't used a Mac [...]

True. But I'm not saying that client-side (binary) drivers should stay, or that we should necessarily stick to PS; but that force-replacing it with something that is over a decade in the making and still doesn't have anywhere near feature parity in practice is a bad idea. And calling features "fluff" or even "garbage" really doesn't help. Experience has taught me to avoid dismissing others' use cases; never ended well.

tl;dr: Buy any AirPrint printer, they said. It'll just work, they said. Wish that were true ...

@amarihw
Copy link

amarihw commented Aug 16, 2024 via email

@michaelrsweet
Copy link
Member Author

@fallenguru

You wrote above that EcoPrint isn't standardised. That's just Kyocera's marketing name for their toner saving mode. What general purpose printer doesn't have a toner (ink) saving mode?

Lots of printers, actually. For inkjet printers the general guidance from printer manufacturers is to choose "draft" print quality (at least for Canon, EPSON, and HP, which I have in my office and just checked).

HP laser printers do offer an Economode setting (both on the embedded web interface and in their drivers), but they also say the following:

"This printer has an EconoMode option for printing drafts of documents. Using EconoMode can use less toner. However, using EconoMode can also reduce print quality. HP does not recommend the full-time use of EconoMode. If EconoMode is used full-time, the toner supply might outlast the mechanical parts in the toner cartridge. If print quality begins to degrade and is no longer acceptable, consider replacing the toner cartridge."

Based on my own testing with a recent HP Laser printer in my office, choosing "draft" quality activates economode.

Lexmark laser printers have "eco-mode" settings on the embedded web interface that control energy usage (mainly when the motors engage), default duplex printing, and toner density (1 to 5). Choosing draft (one of my Lexmark laser printers supports that) seems to lighten the print somewhat but I don't know whether it is activating an "eco" mode or just printing at a lower resolution.

Why? What specifically is complex about attribute/value pairs? Ok, for ones that aren't standardised everything needs a label, too, and for that i18n would be nice, but other than that? "It's complicated" just sounds like hand-waving.

The "overrides" and finishings-col attributes are particularly complex. "overrides" allows for per-page and per-document option overrides when printing, and "finishings-col" allows you to specify the location, orientation, type, etc. of staples, folds, cuts, coatings (front and/or back), creases, bindings, covers, bales, punched holes, etc. Right now the best we have for general print UI there is support for the printer-listed finishing templates and doing basic overrides (first page/sheet from letterhead, remainder from the regular tray, etc.)

So here's the major (for me) features my Kyocera's AirPrint implementation is lacking: resolution, toner saver mode, gloss mode, colour reproduction profile, print greys with black toner, a slew of colour adjustments. How many of those are Kyocera's fault and how many are "not standardised"?

Toner save mode and possibly gloss mode (although that might be covered by finishings) and the color adjustments (depending on what they are). Everything else has been standardized.

Now that's just disingenuous. Postscript is Touring complete, it can do Everything™. I even wrote a game in it at uni decades ago. And, yes, it ran fine on actual printers. The only argument against postscript that I can see is not wanting a Touring complete print language in the first place for security reasons. Though I suspect PDF isn't much better and the idea of sending everything rasterised doesn't appeal at all.

PostScript can't do transparency, and PostScript can't do OpenType. And yes, having a Turing complete document format is a major security issue.

PDF isn't Turing complete and supports transparency and OpenType, in addition to the old PostScript imaging model stuff. PDF isn't perfect, but from a security and printing/graphics standpoint it is far better than PostScript.

@amarihw
Copy link

amarihw commented Aug 18, 2024 via email

@michaelrsweet michaelrsweet self-assigned this Aug 18, 2024
@OpenPrinting OpenPrinting locked as too heated and limited conversation to collaborators Aug 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request priority-high
Projects
None yet
Development

No branches or pull requests