[vlan] Revise sleep time for pytest case "test_addMaxVlan".

[linemma]

Signed-off-by: Emma Lin emma_lin@edge-core.com

What I did
Revise sleep time for pytest case "test_AddMaxVlan".

Why I did it
Because the pytest case "test_AddMaxVlan" is taking too long time to finish.

How I verified it

Details if related
=================================== test session starts ===================================
platform linux2 -- Python 2.7.15rc1, pytest-3.3.0, py-1.8.0, pluggy-0.6.0 -- /usr/bin/python
cachedir: .cache
rootdir: /home/ubuntu/sonic-buildimage/src/sonic-swss/tests, inifile:
collected 11 items

test_vlan.py::TestVlan::test_VlanAddRemove PASSED [ 9%]
test_vlan.py::TestVlan::test_MultipleVlan PASSED [ 18%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input0-0] PASSED [ 27%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input1-0] PASSED [ 36%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input2-0] PASSED [ 45%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input3-1] PASSED [ 54%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input0-0] PASSED [ 63%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input1-0] PASSED [ 72%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input2-0] PASSED [ 81%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input3-1] PASSED [ 90%]
test_vlan.py::TestVlan::test_AddMaxVlan PASSED [100%]

=============================== 11 passed in 599.15 seconds ===============================

[stcheng]

240 seconds seem to be quite long time; is a smaller number causing time to fail?

[linemma]

I check the test result:

        time.sleep(240)
    
        # check asic database
        tbl = swsscommon.Table(self.adb, "ASIC_STATE:SAI_OBJECT_TYPE_VLAN")
        vlan_entries = [k for k in tbl.getKeys() if k != dvs.asicdb.default_vlan_id]
>       assert len(vlan_entries) == (4094-1)
E       AssertionError: assert 3105 == (4094 - 1)

test_vlan.py:270: AssertionError

It seems 240 seconds does not enough... it need more time

[lguohan]

why do we need to sleep for 10 minutes?

[linemma]

I skip the sleep on create_vlan and remove_vlan function to shorten the test time. And call time.sleep to wait for the system to become stable after create/remove vlan.

I am waiting the test results on Jenkins. It seems 240 seconds does not enough to finish the test on 1st commit, so I extend the sleep time (10 minutes) to retest to check if the sleep time is enough or not

[linemma]

@lguohan The test is passed, it seems 10 minutes can finish the test. I will try other sleep time to find the most short time for the test.

test_vlan.py::TestVlan::test_AddMaxVlan PASSED [ 83%]

[linemma]

@lguohan, @stcheng the final test time is :

• take 6 minutes to add 4k vlan
• take 7 minutes to remove 4k vlan

The test result of all VLAN cases are PASSED.
test_vlan.py::TestVlan::test_VlanAddRemove PASSED [ 75%]
test_vlan.py::TestVlan::test_MultipleVlan PASSED [ 76%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input0-0] PASSED [ 77%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input1-0] PASSED [ 78%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input2-0] PASSED [ 79%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectKeyPrefix[test_input3-1] PASSED [ 79%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input0-0] PASSED [ 80%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input1-0] PASSED [ 81%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input2-0] PASSED [ 82%]
test_vlan.py::TestVlan::test_AddVlanWithIncorrectValueType[test_input3-1] PASSED [ 83%]
test_vlan.py::TestVlan::test_AddMaxVlan PASSED [ 83%]

[linemma]

retest this please

[stcheng]

thanks i'm still concerned about this test. it failed due to not fully syncing all the routes. could you investigate what's the most time consuming part in this test?