PDF renderer rounds coordinates to two decimal places

[mrd0ll4r]

Description

When using PDF as the renderer, coordinates (at least for bezier curves, didn't test anything else) are rounded to two decimal places.

Expected Behavior

I didn't expect any rounding to happen. The PDF specification does not impose a limit on the precision of real numbers (see section 7.3.3 of the Adobe spec).

Current Behavior

Coordinates are rounded to two places after the decimal dot.

Steps to Reproduce

Using this program:

import processing.pdf.*;

void setup() {
  size(400, 400, PDF, "test.pdf");
}

void draw() {
  background(250);
  noFill();
  stroke(0);

  //a few marker lines so we find our way around the PDF later
  line(100, 101, 300, 101);
  line(100, 102, 300, 102);
  line(100, 103, 300, 103);
  line(100, 104, 300, 104);
  line(100, 105, 300, 105);

  beginShape();
  vertex(100, 100);
  // Test rounding behaviour.
  // Note that we use X coordinates to observe what's happening, because PDF coordinate spaces is flipped along the X-axis in comparison to Processing's.
  bezierVertex(
    150.1234, 100, 
    120.127, 210, // observe rounding behaviour
    223.123, 323); 
  endShape();

  // more marker lines
  line(100, 101, 300, 101);
  line(100, 102, 300, 102);
  line(100, 103, 300, 103);
  line(100, 104, 300, 104);
  line(100, 105, 300, 105);

  exit();
}

will produce a PDF file test.pdf. We can decompress the file using mutool, specifically the mutool clean command (https://mupdf.com/docs/manual-mutool-clean.html):

mutool clean -dif test.pdf decompressed.pdf

The resulting PDF file, without all the preamble and trailer stuff, looks like this:

100 299 m
300 299 l
S
100 298 m
300 298 l
S
100 297 m
300 297 l
S
100 296 m
300 296 l
S
100 295 m
300 295 l
S
100 300 m
150.12 300 120.13 190 223.12 77 c
S
100 299 m
300 299 l
S
100 298 m
300 298 l
S
100 297 m
300 297 l
S
100 296 m
300 296 l
S
100 295 m
300 295 l
S

We can see that the coordinates of the bezier curve are rounded.

Of course, I see that this is not the biggest of issues, some precision is still there. For my use case, I could scale everything up by some factor, then rasterize the PDF at a lower resolution, and probably get the same result. But this behavior still came as a surprise to me.

Your Environment

• Processing version: 3.5.3
• Operating System and OS version: Debian 9

Possible Causes / Solutions

Does iText enforce this? Can we configure it? Are there hints for this?

[mrd0ll4r]

I want to add that this entire behaviour is weird. For example, adding a stroke weight of 0.9876543 will result in a 0.98765 w in the PDF, whereas setting the stroke weight to 2.9876543 results in 2.99 w. I don't blame float precision for this, personally...

[mrd0ll4r]

Actually, this has become an issue now - because of the rounding behaviour, a bunch of the curves I draw fall together on the same coordinates, and that's really not nice...

[benfry]

We're at the mercy of the iText library that handles converting the drawing commands to PDF format. So this can't be fixed without rewriting PDF support entirely.

To deal with your specific issue, however, I'd recommend using a larger area with the size() command, and drawing to that.

[mrd0ll4r]

We're at the mercy of the iText library that handles converting the drawing commands to PDF format. So this can't be fixed without rewriting PDF support entirely.

While it's true that you are at the mercy of iText, this can be improved massively without rewriting PDF support. com.lowagie.text.pdf.ByteBuffer has a public static (yikes) boolean that controls precision.

For future reference, if anyone should run into this, you can try this:

import processing.pdf.*;
import com.lowagie.text.pdf.ByteBuffer;

void setup() {
  ByteBuffer.HIGH_PRECISION = true;
  size(400, 400, PDF, "test.pdf");
}

This will get you up to six places after the decimal dot. It's still rounded, and in some arcane way, but it's much better than without. For this input:

  bezierVertex(
    150.12345678, 100, 
    120.123227, 210, // observe rounding behaviour
    223.123223, 323); 

we now get

100 300 m
150.123459 300 120.12323 190 223.12323 77 c
S

I don't have the sources of that iText version at hand and don't know when that boolean might be reset. I also don't have enough overview over the processing sources to figure out if there would be a good place to put this. Maybe a hint for the renderer? No clue, maybe someone else can figure this out...

[benfry]

Awesome, thanks for tracking that down! Now incorporated for 4.0 alpha 2.

[mrd0ll4r]

Wow, cool! Thanks :)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.