[demo] interactive debugging

[mmd-osm]

This is not a feature request or anything, I just wanted to share a few screenshots illustrating how Lua scripts can be debugged while running osm2pgsql. Debugging in this case means, integrating this Lua debugger, and run flex processing as usual.

Whenever there's an error in the script, or the debugger is triggered explicitly, processing stops and you end up in debug mode.

You can require other Lua files from there, play around with osm2pgsql.* functions, or change any values while processing an object.
This might be interesting for people who struggle a bit with Lua, and want learn a bit more about the details in an interactive way.

generic.lua:

local dbg = require 'debugger'

[...]

function osm2pgsql.process_node(object)
    if clean_tags(object.tags) then
        return
    end
    
    if object.id == 500661 then
      dbg()
    end

  <return> => re-run last command
  c(ontinue) => continue execution
  s(tep) => step forward by one line (into functions)
  n(ext) => step forward by one line (skipping over functions)
  f(inish) => step forward until exiting the current function
  u(p) => move up the stack by one frame
  d(own) => move down the stack by one frame
  w(here) [line count] => print source code around the current line
  e(val) [statement] => execute the statement
  p(rint) [expression] => execute the expression and print the result
  t(race) => print the stack trace
  l(ocals) => print the function arguments, locals and upvalues.
  h(elp) => print this message
  q(uit) => halt execution

I have uploaded this strictly proof-of-concept demo here, if anyone wants to play with it: https://github.com/mmd-osm/osm2pgsql/tree/debugger

[lonvia]

That's neat. Is it even necessary to embed the debugger in osm2pgsql? Would it be possible to just require the lua version?

[mmd-osm]

Yes, that's also possible, although not all of the features are available. The embedded version can be used to wrap/intercept calls to lua_pcall, and register the embedded debugger as a message handler. Rather than calling pcall_error_traceback_handler, we would end up in the debugger.

For explicit calls to the debugger using dbg() we wouldn't need the embedded version.

require 'debugger'

debugger.lua: Loaded for Lua 5.2
2022-10-22 11:29:24  ERROR: Failed to execute Lua function 'osm2pgsql.process_node': generic.lua:234: assertion failed!
stack traceback:
	[C]: in function 'assert'
	generic.lua:234: in function <generic.lua:227>.

Embedded debugger

debugger.lua: Loaded for Lua 5.2
ERROR: "generic.lua:234: assertion failed!"
break via dbg.msgh() => generic.lua:234 in chunk at generic.lua:227
debugger.lua> w
 229            return
 230        end
 231        
 232        if object.id == 500661 then
 233          --dbg()
 234 =>       assert(1 == 0)
 235        end
 236    
 237        tables.points:insert({
 238            tags = object.tags,
 239            geom = object:as_point()
debugger.lua>

It wouldn't even be necessary to include debugger.lua file contents as part of the osm2pgsql binary, it could as well be loaded on the fly from a debugger.lua file: https://github.com/mmd-osm/osm2pgsql/blob/debugger/src/debugger_lua.c#L43-L44