Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] name of table not set in DeltaTable.detail() (Delta 2.1.0) #1549

Closed
1 of 3 tasks
keen85 opened this issue Jan 9, 2023 · 4 comments
Closed
1 of 3 tasks

[BUG] name of table not set in DeltaTable.detail() (Delta 2.1.0) #1549

keen85 opened this issue Jan 9, 2023 · 4 comments
Assignees
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@keen85
Copy link

keen85 commented Jan 9, 2023

Bug

Describe the problem

Delta Lake Python API features a method DeltaTable.detail() for retrieving information on a given Delta table

The documentation claims that this will also return the name of the Delta table.
My tests showed that this is not the case; null is returned.

Steps to reproduce

from delta.tables import *

spark.sql("""
DROP TABLE IF EXISTS test_delta_210
""")
spark.sql("""
CREATE TABLE test_delta_210 (
    id LONG,
    desc STRING
)
USING delta
""")

deltaTable = DeltaTable.forName(spark, "test_delta_210")

deltaTable.detail().show(truncate=False, vertical=True)

Observed results

 format           | delta                                                                         
 id               | 4cb0168d-f070-4c56-a17c-a41f303d2354                                          
 name             | null                                                                          
 description      | null                                                                          
 location         | abfss://xxx/test_delta_210 
 createdAt        | 2023-01-09 12:30:37.285                                                       
 lastModified     | 2023-01-09 12:30:37                                                           
 partitionColumns | []                                                                            
 numFiles         | 0                                                                             
 sizeInBytes      | 0                                                                             
 properties       | {}                                                                            
 minReaderVersion | 1                                                                             
 minWriterVersion | 2   

Expected results

name should be default.test_delta_210

Environment information

Azure Synapse Analytics

  • Delta Lake version: 2.1.0
  • Spark version: 3.3.1
  • Scala version: 2.12.15

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

  • Yes. I can contribute a fix for this bug independently.
  • Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
  • No. I cannot contribute a bug fix at this time.
@keen85 keen85 added the bug Something isn't working label Jan 9, 2023
@scottsand-db scottsand-db self-assigned this Jan 12, 2023
@scottsand-db scottsand-db added the good first issue Good for newcomers label Jan 26, 2023
@scottsand-db
Copy link
Collaborator

Thanks for making this! Could you confirm if this issue exists in the latest 2.2.0 release as well?

@keen85
Copy link
Author

keen85 commented Jan 27, 2023

Hi @scottsand-db,
I tried it on Databricks-Runtime 12.1.x with Delta Lake 2.2.0 and the issue exists there as well.

@scottsand-db
Copy link
Collaborator

Thanks for clarifying! Will investigate ...

@zsxwing
Copy link
Member

zsxwing commented Feb 9, 2023

Currently, we use the table path to run DESC DETAIL command. We can simply store the table name when creating DeltaTable and pass it to DESC DETAIL command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants