displaying the progress status and total time of near2far calculations in output

[oskooi]

In certain situations involving a large number of far-field points, the near2far routines get_farfields and output_farfields can comprise the majority of the wall-clock time (e.g., the tutorial example in #845). Currently, the progress status of the near2far calculation is not included in the output. Unlike the progress status for the time-stepping (i.e., on time step 4467 (time=55.8375), 0.0549067 s/step) which provides an approximate insight into its expected completion, no such information is available for near2far calculations. In fact, there is no mention at all in the output that the far-fields are being computed. It would be useful to include some indication of the near2far progress in the output. This information could also facilitate with debugging to improve performance.

[stevengj]

i.e. we'd add something like

meep/src/step.cpp

Lines 43 to 54 in a06587c

	if (!t) {
	last_step_output_wall_time = wall_time();
	last_step_output_t = t;
	}
	if (!quiet && wall_time() > last_step_output_wall_time + MIN_OUTPUT_TIME) {
	master_printf("on time step %d (time=%g), %g s/step\n", t, time(),
	(wall_time() - last_step_output_wall_time) / (t - last_step_output_t));
	if (save_synchronized_magnetic_fields)
	master_printf(" (doing expensive timestepping of synched fields)\n");
	last_step_output_wall_time = wall_time();
	last_step_output_t = t;
	}

after every call to farfield_lowlevel here

(We can use the same MIN_OUTPUT_TIME constant, which is currently 4 seconds. It doesn't seem necessary to make this a user parameter — outputting every few seconds has been satisfactory for a long time now.)

(farfield_lowlevel is expensive enough that doing a check of time > walltime + min_time should be negligible.)

[stevengj]

One thing we might think about is having a callback function for progress instead of a hard-coded printf. That is, you would call something like progress_update(label_string, fraction) and it would default to a printf.

This would allow us to potentially call something like the progressbar2 package in Python (or tqdm or similar).

[oskooi]

The total time spent computing the far fields during the call to get_farfields or output_farfields is also missing from the "Field time usage" statistics info appearing at the end of the time stepping when Simulation.run has completed.

For example, python/examples/binary_grating_n2f.py has three separate calls to run and get_farfields but the near-to-far field transformation portion of the runtime (which comprises the majority) is not included in any of its three output statistics. This is the output from the final run:

Field time usage:
        connecting chunks: 0.0613327 s
            time stepping: 37.194 s
            communicating: 1.59942 s
     Fourier transforming: 8.40363 s
          everything else: 0.60348 s