Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added globalization unit tests for various charset including GB18030 #293

Merged
merged 4 commits into from
Sep 16, 2019

Conversation

geleems
Copy link

@geleems geleems commented Sep 12, 2019

Added globalization unit tests for various charset including GB18030

@geleems geleems self-assigned this Sep 12, 2019
from mssqltestutils import create_mssql_cli_client, shutdown


# All tests require a live connection to an AdventureWorks2014 database with a hardcoded test server.
Copy link
Member

@pensivebrian pensivebrian Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests don't depend on the AdventureWorks2014 schema, so we can remove that reference. #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed. #Closed

shutdown(client)


def get_unit_str(self, chars, unit_len = 50):
Copy link
Member

@pensivebrian pensivebrian Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename this to get_object_name or something more descriptive? Plus add some comments noting this is used to create localized objects. #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_unit_str() is removed. #Closed

def get_unit_str(self, chars, unit_len = 50):
curr_index = 0
while curr_index < len(chars):
next_index = min(curr_index + unit_len, len(chars))
Copy link
Member

@pensivebrian pensivebrian Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be more simple to use the substring operator if the character is greater than unit_len. #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With that approach, it seems we do not even need generator. I removed get_unit_str(). Fixed. #Closed

shutdown(client)


def get_unit_str(self, chars, unit_len = 50):
Copy link
Member

@pensivebrian pensivebrian Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename unit_len to max_string_length? #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed. #Closed

assert is_error == False

select_query = u"SELECT * FROM {0};".format(table_name)
for rows, columns, status, statement, is_error in client.execute_query(select_query):
Copy link
Member

@pensivebrian pensivebrian Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we assert execute_query returned the expected number of tuples? If it returns 0, it seems like it would pass. #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you meant 'if it returns 2, it is pass'? Because we inserts 2 tuples in the tests. #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if it returns 2 rows like this?:

('??', 1)
('??', 2)

'??' indicates broken encoding. #Closed

Copy link
Member

@pensivebrian pensivebrian Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should assert that too. #Closed

Copy link
Author

@geleems geleems Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added back the assert statements. #Closed

# All tests require a live connection to an AdventureWorks2014 database with a hardcoded test server.
# Make modifications to mssqlutils.create_mssql_cli_client() to use a different server and database.
# Please Note: These tests cannot be run offline.
class GlobalizationTests(unittest.TestCase):
Copy link
Member

@pensivebrian pensivebrian Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename to GlobalizationResultSetTests? #Closed

Copy link
Author

@geleems geleems Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed. #Closed

assert is_error == False

select_query = u"SELECT * FROM {0};".format(table_name)
for rows, columns, status, statement, is_error in client.execute_query(select_query):
Copy link
Member

@pensivebrian pensivebrian Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should assert that too. #Closed

for rows, columns, status, statement, is_error in client.execute_query(setup_query):
assert is_error == False

select_query = u"SELECT * FROM {0};".format(table_name)
Copy link
Member

@pensivebrian pensivebrian Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using a select *, can you specify the column name? This way, we make sure we encode the correctly sent over json/rpc as input. #Closed

Copy link
Author

@geleems geleems Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added column names to select statement #Closed

max_string_length = 50
for chars in chars_list:
while len(chars) > 0:
next_index = min(len(chars), max_string_length)
Copy link
Member

@pensivebrian pensivebrian Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: next_index isn't that descriptive. How test_string_length? #Closed

Copy link
Author

@geleems geleems Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to test_string_length as requested. #Closed

select_query = u"SELECT * FROM {0};".format(table_name)
for rows, columns, status, statement, is_error in client.execute_query(select_query):
assert is_error == False
assert len(rows) == 2
Copy link
Member

@pensivebrian pensivebrian Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we assert the value in the result set matches the input string? #Closed

Copy link
Author

@geleems geleems Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added back the input value assert statements. #Closed

setup_query = u"CREATE TABLE {0} ({1} nvarchar(MAX), {2} int);"\
u"INSERT INTO {0} VALUES (N'value_{3}1', 1);"\
u"INSERT INTO {0} VALUES (N'value_{3}2', 2);"\
.format(table_name, col1_name, col2_name, test_str)
Copy link
Member

@pensivebrian pensivebrian Sep 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

format [](start = 25, length = 6)

Can we insert the entire chars string? #Closed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And validate the chars string.


In reply to: 324836186 [](ancestors = 324836186)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synced offline.

try:
client = create_mssql_cli_client()
max_string_length = 50
for chars in chars_list:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chars [](start = 16, length = 5)

nit, but important: chars and test_str are vague. Can you rename these to be more descriptive?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synced offline. Changed to a little more descriptive words.

Copy link
Member

@pensivebrian pensivebrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just one last comment above.

max_string_length = 50
for chars in chars_list:
while len(chars) > 0:
test_string_length = min(len(chars), max_string_length)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some comments here describing this loop would be good.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments added.

@geleems geleems merged commit a43215b into master Sep 16, 2019
@geleems geleems deleted the gelee/glob branch September 18, 2019 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants