WIP: Add option to not create a temporary EPS file but instead edit the original PS.

[joa-quim]

It has always annoyed me (e., g. #1435) that for running psconvert (and before, ps2raster) we must create a temporary EPS file that is the one we sent through the Ghostscript guts to get our raster file (I hate solutions based on temporary disk files). No big deal if the PS file is small, but far from ideal if it's a hundreds MB file.

This PR adds an option to psconvert that avoids that by editing the original PS turning it into what before we obtained with EPS temporary file. Due to shortage of letters, excluding the global options we have no more alphabet letters available, that option was made -!, which inspired in the Julia convention that bang (!) symbol is used for data modifying functions.

I have also added run time reports via the -Vt option and, a bit disappointingly, it takes big PS files to show small run times differences between master and this PR (I do have a fast SSD disk, is it it?).

Run the tests with an hard-coded -! so that also the modern mode scripts use it and saw no obvious failures, but certainly not all psconvert paths were tested. So I would like to ask for more testing on this. If it proves reliable, the idea is to make this PR the default behavior and add some flag as a modifier of an existing option to keep the ancient behavior.

PLEASE TEST

[anbj]

May I ask why things are done this way? (Doing the EPS-stuff, instead of using the PS, as is the goal here)

[joa-quim]

There are things I don't know. Namely, why GMT PS have a setpagedevice (is it needed?) and why the EPS should not have those (but that's what Artifex people say). But one thing is simple to explain. Given that PMT PSs are made with layers, there is no way that GMT can know the final figure BoundingBox, and without that knowledge we cannot do a correct -A. So we have to find it by asking the Ghost to calculate it and after insert the right numbers plus offsets to put the figure origin at (0,0). And there is the GeoPDF stuff, transparencies etc. Not all of this is possible to do by editing the original file and not changing its size by a single byte. In those cases the new -! option is deactivated.

[anbj]

Thanks @joa-quim.

[seisman]

Not sure if I'm using it correctly, but it crashes for the following command:

$ gmt --version
6.6.0_f571a71_2024.03.27
$ gmt psbasemap -R0/10/0/10 -JM10c -Baf > map.ps
$ gmt psconvert -A -P -Tj -! map.ps
ERROR: Caught signal number 11 (Segmentation fault) at
/lib64/libc.so.6(+0x61b21)[0x7fd9ce87fb21]
[0xc0]
Stack backtrace:
/home/seisman/opt/GMT-master/lib64/libgmt.so.6(gmt_sig_handler_unix+0xbf)[0x7fd9d2f582ef]
/lib64/libc.so.6(+0x3e9a0)[0x7fd9ce85c9a0]
/lib64/libc.so.6(+0x61b21)[0x7fd9ce87fb21]
/lib64/libc.so.6(_IO_fprintf+0x9c)[0x7fd9ce87437c]
/home/seisman/opt/GMT-master/lib64/libgmt.so.6(GMT_psconvert+0x21c5)[0x7fd9d3199885]
/home/seisman/opt/GMT-master/lib64/libgmt.so.6(GMT_Call_Module+0x335)[0x7fd9d2e59595]
gmt[0x401734]
/lib64/libc.so.6(+0x2814a)[0x7fd9ce84614a]
/lib64/libc.so.6(__libc_start_main+0x8b)[0x7fd9ce84620b]
gmt[0x4024a5]

[joa-quim]

Oh ghrrr. That works fine for me on Windows. Can you try it with a debug build. Maybe the stack trace will be more informative.

[seisman]

Can you try it with a debug build. Maybe the stack trace will be more informative.

Just tried the debug build. The stack trace is the same.

One question: even if it doesn't crash, does it mean that the original PS file on the disk will be broken since it's edited when calling psconvert? Or the editing is done to the PS file in memory?

[joa-quim]

The -! should not break the original PS file but changes it to become equal to what we get with -Te. No in-memory processing though that is already possible when called from Matlab & Julia but I have not tried this in long time.

GMT_LOCAL int psconvert_pipe_HR_BB(struct GMTAPI_CTRL *API, struct PSCONVERT_CTRL *Ctrl, char *gs_BB, double margin, double *w, double *h) {
	/* Do what we do in the main code for the -A (if used here) option but on an in-memory PS 'file' */

One idea is that would be useful to the wrappers that don't really care about the original PS.

[joa-quim]

Can you check the beginning of the map.ps file and see if any of the BoungingBoxes have been changed?

Original

%%BoundingBox: 0 0 595 842
%%HiResBoundingBox: 0 0 595.0000 842.0000  

After change

%%BoundingBox: 0 0 321 338
%%%HiResBoundingBox: 0 0 337.9680 320.7420  

[seisman]

There are no changes in the PS file.

[joa-quim]

So it must have crashed on lines 2366 or 2368

if (Ctrl->O.active) {
	fseek(fp, (off_t)-strlen(line), SEEK_CUR);	/* Seek back to start of line */
	if (got_BB && !Ctrl->A.round)
		sprintf(line, "%%%%BoundingBox: 0 0 %ld %ld", lrint (psconvert_smart_ceil(w_t)), lrint (psconvert_smart_ceil(h_t)));

I'm opening the PS file with fp = fopen (ps_file, "r+"). Could it be the problem for unix?

[seisman]

So it must have crashed on lines 2366 or 2368

if (Ctrl->O.active) {
	fseek(fp, (off_t)-strlen(line), SEEK_CUR);	/* Seek back to start of line */
	if (got_BB && !Ctrl->A.round)
		sprintf(line, "%%%%BoundingBox: 0 0 %ld %ld", lrint (psconvert_smart_ceil(w_t)), lrint (psconvert_smart_ceil(h_t)));

I'm opening the PS file with fp = fopen (ps_file, "r+"). Could it be the problem for unix?

r+ should work on Unix (https://man7.org/linux/man-pages/man3/fopen.3.html).

I tried to add a print statement in the while (psconvert_file_line_reader (GMT, &line, &line_size, fp) != EOF && n_read_PS_lines < max_PS_lines) loop. It seems only the first line %!PS-Adobe-3.0 is read and then it crashes.

[joa-quim]

Please try again. It was a dumb error in the null file name.

[seisman]

Please try again. It was a dumb error in the null file name.

Yes, now it works on Linux, but as I asked before, now I can confirm that the original PS file is actually a EPS file after editing.

I'm wondering can we just read the PS file and send it to Ghostscript via stdin, like https://stackoverflow.com/questions/70706765/passing-stdin-to-ghostscript-in-php-proc-open.

[anbj]

How to test this? Just running a bunch of scripts?

[joa-quim]

How to test this? Just running a bunch of scripts?

Well, yes but what I did was to hardwire the -! option by adding a line (~line 1735 of psconvert.c)

Ctrl->O.active = true;

and run all the tests. But even all the tests do not probably test everything.

[joa-quim]

I'm wondering can we just read the PS file and send it to Ghostscript via stdin

The psconvert_pipe_ghost() function on psconvert tries to do that but for some reason it only works on Windows and is slow. Note that the interest here is to be able to return the raster image directly in memory. That was made for the Matlab wrapper (an later for GMT.jl) using the fact that we can return the PS data in a memory address and not writing it on disk. On CLI that is not possible so sending in by stdin doesn't look better than what we do (a system call).

The still-in-the-air idea is to one day be able to call the Ghostscript API directly to do these operations. This would free us from the system calls and popen/popen2 troubles.

[anbj]

The still-in-the-air idea is to one day be able to call the Ghostscript API directly to do these operations. This would free us from the system calls and popen/popen2 troubles.

And not using popen/popen2 have what benefits?

[joa-quim]

The popen/popen2 functions we have for some mysterious reasons do not work on unix and are slow on Windows.

[seisman]

Instead of adding -!, can we just add a modifier to the -Z option?

• -Z: Remove the input PS file
• -Z+r: Same as -Z
• -Z+e: Edit the original PS file when creating the temporary EPS
• -Z+e+r: Edit the original PS file and delete.

[seisman]

Suggested change

	GMT_Usage (API, 1, "\n-! Modify the input PS file instead of creating a temp EPS. Some options like -Ng will disable it.");
	GMT_Usage (API, 1, "\n-! Modify the input PS file instead of creating a temp EPS. Some options like -Wg will disable it.");

I think you mean -Wg, right?

I've tried this branch in PyGMT. Without adding the -! option, most tests still work, but with -Wg, I see errors like:

Error: rt [ERROR]: Could not find the 'PROJ' tag in the PS file. No world file created.
Error: rt [ERROR]: This situation occurs when one of the two next cases is true:
Error: rt [ERROR]: 1) the PS file was created with a pre-GMT v4.5.0 version
Error: rt [ERROR]: 2) the PS file was not created by GMT

Is the -! option enabled by default? I didn't add -!, so it should not affect any existing tests, right?

[joa-quim]

No, it's -Ng. This and other have to disable the -! (not possible given the particular option requirements). But we need it better explained and to create a dedicated option to specifically ignore what -! does.

[joa-quim]

But the error is real, thanks. Have to see what to do about it.

[joa-quim]

It's been a while since I did this. Details are no longer fresh. I'll revisit it later today.

[joa-quim]

Is the -! option enabled by default?

If I was not wrong at the time I wrote it, yes.

How to test this? Just running a bunch of scripts?

Well, yes but what I did was to hardwire the -! option by adding a line (~line 1735 of psconvert.c)

[joa-quim]

Instead of adding -!, can we just add a modifier to the -Z option?

The point is that I was thinking in making this the future behavior, thus independent of any option.

[joa-quim]

Is the -! option enabled by default?

I thought it was, but it turns out that it wasn't. I set it on by default temporarily to run the test and must have unset it after.

But now, with my last commit, we can set this the default behavior by adding /DPS_NO_DUP or use add_definitions(/DPS_NO_DUP) in ConfigUser.cmake. Then the -! option has the reverse meaning. That is, with /DPS_NO_DUP the -! unsets the no-dup behavior, whilst without /DPS_NO_DUP it is the contrary. It sets it.

Run the tests locally and got a couple more failures but I'm confused because I don't even see why they fail.

Another not investigated thing is the effect that this may have on what we talked the other day about the fig1 & fig2 on modern mode. Please be my guest to test it.

[joa-quim]

I've tried this branch in PyGMT. Without adding the -! option, most tests still work, but with -Wg, I see errors like:

This is totally unrelated to this branch as you saw it when -! was not active. Please open an issue.

[joa-quim]

@seisman Do you understand what is going on with these failures? This branch builds fine for me.