Remove Avro Schema description copying from bigquery_avro

[idreeskhan]

Description

Google BigQuery now has support by default for Schema description copying. Removing the existing description copying from bigquery_avro.py since this is now redundant and may cause issues/conflicts in the future.

The logic currently will overwrite Google-side logic to copy over descriptions.

Have you tested this? If so, how?

I ran my jobs with this code and it works for me.

[idreeskhan]

Any advice on how to handle the last few flake8 errors? Don't seem to be related to anything I have done and I tried to clean most of them up.

./test/decorator_test.py:136:7: E742 ambiguous class definition 'I'
./test/parameter_test.py:876:13: E741 ambiguous variable name 'l'
./test/parameter_test.py:880:25: E741 ambiguous variable name 'l'

[idreeskhan]

@ulzha FYI

[ulzha]

Yeah. Hm. Seems like flake8 has arbitrarily started to apply new rules.

[ulzha]

Thanks for cleaning up. Does the bigquery_gcloud_test succeed unchanged? (We seem to not be running the gcloud tests on Travis.)

I think avro dependencies are becoming unneeded in

luigi/tox.ini

Line 24 in 4801fec

py27-gcloud: avro

and perhaps somewhere else as well...

[idreeskhan]

Good catch on the test. Seems like google doesn't propagate the table level description at the moment so I'll have to pull that back in 😢

[ulzha]

@idreeskhan or just file a feature request to propagate description as well, and not pull it back in here? :]

[idreeskhan]

@ulzha Yes, for sure! We are working on that but the timeline may not match ours since this has a short timeline for us. Will know more and update later in the week.

[idreeskhan]

@ulzha This will not be fixed soon enough by Google. I have re-added the table description copying logic and verified that the current tests still pass. I kept the field level description copying out as that is properly handled by BQ.

[ulzha]

Merging despite codecov diff complaints... Removing code is awesome.

[Tarrasch]

Removing code is awesome

I cannot agree more. Thanks @idreeskhan!