Athena Database Connector #352
Conversation
|
@n-riesco Just as an FYI I am based out of Montreal. So if the Plot.ly folks want I can swing by their office in the mile end and walk them through creating the table in Athena if this will help move this along. Just a thought. |
|
@shannonlal I'm forwarding your message. Again, sorry about the delay. |
|
Hey @shannonlal - thanks for the offer. I just invited the hotmail address in your bio to Plotly's slack if you want to coordinate there. You're welcome by the office anytime! |
|
@n-riesco I just pulled in the latest changes from the master branch into this PR. I am looking at the comments we discussed about this morning |
|
@shannonlal OK, I'll be online here and on slack. Please, ping me, if anything needs clarification. |
|
@n-riesco . I just wanted to summarize your PR comments into one location. Can you let me know if the following changes to this PR make sense:
Does this make sense? |
* All tests share the Athena connection. * Added connect() test and ensured it runs first, so that connection.athenaClient is defined for the remaining tests. * Defined helper function mockAthenaResponses(query, columnNames, rows). * Fixed tables() test (tables() shouldn't return the column name `table_names`).
* Fixed tables() so that it doesn't return the column name table_names. * Avoided the creation of unnecessary promises. * Used slice() and map() to generate the response rows.
Athena/pr 352 debugging
|
💃 |
|
I'm getting the following error.
What is the solution for this? |
|
Can you provide a little bit more information on how you got the error and someone else can reproduce. Also, are you able to submit any queries?
Shannon
…Sent from my iPhone
On Feb 2, 2020, at 8:22 AM, Srikanth Gadicherla <notifications@github.com> wrote:
I'm getting the following error.
HIVE_METASTORE_ERROR: com.facebook.presto.spi.PrestoException: varchar type is specified without length: varchar (Service: null; Status Code: 0; Error Code: null; Request ID: null)
What is the solution for this?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub<#352?email_source=notifications&email_token=AASYB23YGICE2TRSSX3JTU3RA3CITA5CNFSM4ENDY66KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRWWAI#issuecomment-581135105>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AASYB25PSA24XRAZZJCO5BTRA3CITANCNFSM4ENDY66A>.
|
|
Hi,
I get this error when I try to connect the Athena database. I have attached
the screenshot. Kindly help me out.
<img width="1033" alt="Screen Shot 2020-02-02 at 19 05 23" src="https://user-images.githubusercontent.com/16464365/73611899-9820ad00-45ef-11ea-8db8-7097af09318a.png">
|
|
Hi, i have the same problem: HIVE_METASTORE_ERROR: com.facebook.presto.spi.PrestoException: varchar type is specified without length: varchar (Service: null; Status Code: 0; Error Code: null; Request ID: null) from glue i created a crawler for a table in Aurora MySQL with a column varchar(50), into the crawler has been set like string and an error appear running an athena query: so i changed the data type varchar into the crawler but i can't specify the length. There are any news? |
|
I tried with a simple Aurora table within a single field of type int, the error is the previous first: running the crawler it found the new table and from Athena it appear running show tables. Thanks |
|
link to resolve the error "unable to create input format" in Athena: https://aws.amazon.com/premiumsupport/knowledge-center/athena-unable-to-create-input-format/ |
This is the first pull request for the Athena Database Connector #261. A couple of notes:
I broke the Athena database connector into two files. The first is the Athena Connector similar to the other Falcon DB Connectors which defines the standard interfaces (connect, query, tables, schema, etc). The second file just handles the connection to Athena using the AWS SDK. I broke this into a separate file as it made unit testing easier and also if at some point a standard NodeJS library for Athena becomes available it would be easier to replaces this.
I was not able to create an integration test for this connector as I don't have access to a Plot.ly instance of Athena.
This is my first major PR on this project so I welcome any and all comments or suggestions.