async check all urls

[C-Loftus]

Can't get this working for some reason. Always runs out of memory no matter how I configure it. I think it might be because of the fact the XML files are too big but I am manually freeing the memory after parsing it so not really sure why

Multiprocess executor: child process for step gleaner_config_gleaner_links_are_valid was terminated by signal 9 (SIGKILL). This usually indicates that the process was killed by the operating system due to running out of memory. Possible solutions include increasing the amount of memory available to the run, reducing the amount of memory used by the ops in the run, or configuring the executor to run fewer ops concurrently.
dagster._core.executor.child_process_executor.ChildProcessCrashException

Stack Trace:
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/executor/multiprocess.py", line 256, in execute
    event_or_none = next(step_iter)
,  File "/usr/local/lib/python3.10/site-packages/dagster/_core/executor/multiprocess.py", line 388, in execute_step_out_of_process
    for ret in execute_child_process_command(multiproc_ctx, command):
,  File "/usr/local/lib/python3.10/site-packages/dagster/_core/executor/child_process_executor.py", line 173, in execute_child_process_command
    raise ChildProcessCrashException(pid=process.pid, exit_code=process.exitcode)