Use columns defined for COPY to Redshift so we never rely on order

[jamesmcm]

Description

See issue #2244

Motivation and Context

Right now COPYs to Redshift are dependent upon the column ordering which can break imports.

Have you tested this? If so, how?

I've tested it locally with our columns defined.

[dlstadther]

Thanks for submitting!

I have listed one code refactor request.

[dlstadther]

Perhaps you could do something more like below to reduce duplicate code

colnames = ''
if len(self.columns) > 0:
    colnames = ",".join(x[0] for x in self.columns])
cursor.execute("""\
    copy {table} {cols} from '{source}'
    credentials '{creds}'
    {options}
    ;""".format(
        table=self.table,
        colnames='({})'.format(colnames),
        source=f,
        creds=self._credentials(),
        options=self.copy_options
    )

[jamesmcm]

Fixed :)

[dlstadther]

I'm going to pull this branch down and verify its compatibility with my ETLs (as i'm not familiar with the data mapping options with COPY). Will report back soon

[jamesmcm]

Thanks, I don't think it should break anything, unless you have defined the columns there but are reliant on it importing them out of order, and I can't really think of any use case where that is desirable.

[dlstadther]

Thanks @jamesmcm !