[TTAHUB-3097] Implement transactions for worker processes

[GarrettEHill]

Description of change

This update introduces transaction handling for worker processes in the Head Start TTA Data Platform. The main changes include:

1. Implementing transaction management in worker scripts to ensure data consistency.
2. Enhancing error logging for detailed information about errors and database connection states.
3. Updating tests to align with the new transaction management and error handling.

How to test

1. Run Unit Tests: Execute all unit tests to ensure they pass with the new transaction handling logic.
2. Simulate Worker Processes: Trigger worker processes and monitor logs for detailed error messages.
3. Check Database Consistency: Verify that data remains consistent and correctly rolled back in case of errors during worker processes.

Issue(s)

• https://jira.acf.gov/browse/TTAHUB-3097

Checklists

Every PR

• Meets issue criteria
• JIRA ticket status updated
• Code is meaningfully tested
• Meets accessibility standards (WCAG 2.1 Levels A, AA)
• API Documentation updated
• Boundary diagram updated
• Logical Data Model updated
• Architectural Decision Records written for major infrastructure decisions
• UI review complete

Before merge to main

• OHS demo complete
• Ready to create production PR

Production Deploy

• Staging smoke test completed

After merge/deploy

• Update JIRA ticket status

[GarrettEHill]

@thewatermethod @nvms @AdamAdHocTeam @kcirtapfromspace @kryswisnaskas @hardwarehuman
Its all passing now, ready for review... again

[nvms]

Let's simplify transactionWrapper a little bit:

export default function transactionWrapper(originalFunction, context = '') {
  return async function wrapper(req, res, next) {
    const startTime = Date.now();
    try {
      // eslint-disable-next-line @typescript-eslint/return-await
      return await sequelize.transaction(async () => {
        try {
          await addAuditTransactionSettings(sequelize, null, null, 'transaction', originalFunction.name);
          const result = await originalFunction(req, res, next);
          const duration = Date.now() - startTime;
          auditLogger.info(`${originalFunction.name} ${context} execution time: ${duration}ms`);
          removeFromAuditedTransactions();
          return result;
        } catch (err) {
          auditLogger.error(`Error executing ${originalFunction.name} ${context}: ${err.message}`);
          throw err;
        }
      });
    } catch (err) {
      return handleErrors(req, res, err, logContext);
    }
  };
}

So we don't have to do that error = err thing, which isn't needed. Also the result can be returned inside the try, so it doesn't need to be in the upper scope. Just a bit easier to read IMO.

[thewatermethod]

All functionality looks to still be there/work 👍

[thewatermethod]

I'd grab Jon's changes to the sequelizeConnectionError stuff, or wait until that's merged into main, so we don't merge anything that throws errors in the error handler

[AdamAdHocTeam]

Wouldn't ...error also contain error.stack. Why do we need to put it in the property errorStack?

[GarrettEHill]

just matching what the original code had

[AdamAdHocTeam]

why do we check instanceof Sequelize.Error here but above we do operation !== 'SequelizeError'.

[GarrettEHill]

just matching original logic

[AdamAdHocTeam]

Will call stack be visible here?

[GarrettEHill]

i think so

[AdamAdHocTeam]

Do we need to pass a second param here to the transactionQueueWrapper like in S3 and Resource queue's?

[GarrettEHill]

the scan queue never passed a key, so there is no associated value to use as a context