Refactored ResultSet to lazy loading

[yafimvo]

Describe your changes

Lazy loading added to ResultSet.

1. Fetch logic moved from the constructor to def fetch_results
2. Returns the min number of rows to display
3. list(result_set) returns all results
4. Display limit message modified

Issue number

Closes #470

Checklist before requesting a review

• Performed a self-review of my code
• Formatted my code with pkgmt format
• Added tests (when necessary).
• Added docstring documentation and update the changelog (when needed)

---

📚 Documentation preview 📚: https://jupysql--624.org.readthedocs.build/en/624/

[mehtamohit013]

At first glance, seems to be working well. When loading 10 entries, it is 0.036 sec while loading all it is 0.040 sec while using duckdb

Looking at the testcases...

[tonykploomber]

We can do some benchmarks, I did this before

Collecting

def test_execution_time(chinook_db, ip_empty):

    ip_empty.run_cell("%sql duckdb://")
    ip_empty.run_cell("%sql INSTALL 'sqlite_scanner';")
    ip_empty.run_cell("%sql LOAD 'sqlite_scanner';")
    ip_empty.run_cell(f"%sql CALL sqlite_attach({chinook_db!r});")

    res = ip_empty.run_cell("%sql SELECT * FROM Invoice").result
    X = res.DataFrame().Total
    results = []
    for i in range(0, 100):
        start_time = time.time()
        plot._boxplot_stats(Connection.current.session, "Invoice", "Total")
        end_time = time.time()
        results.append(end_time - start_time)
    print(results)

Benchmark graphing

# importing matplotlib module
from matplotlib import pyplot as plt
 
# After values
result1 = [0.007344961166381836, 0.0055768489837646484, 0.0051631927490234375, 0.005031108856201172, 0.004762887954711914, 0.004705190658569336, 0.004629850387573242, 0.004528999328613281, 0.004563093185424805, 0.004524946212768555, 0.004357099533081055, 0.004371166229248047, 0.00431370735168457, 0.0042302608489990234, 0.004227876663208008, 0.004274129867553711, 0.00427699089050293, 0.004266023635864258, 0.0044651031494140625, 0.004297018051147461, 0.004207134246826172, 0.004314899444580078, 0.0041959285736083984, 0.004226207733154297, 0.004321098327636719, 0.0042111873626708984, 0.004172801971435547, 0.004288911819458008, 0.0041599273681640625, 0.004218101501464844, 0.004271745681762695, 0.0041468143463134766, 0.004213094711303711, 0.004291057586669922, 0.004229068756103516, 0.004282951354980469, 0.004322052001953125, 0.004169940948486328, 0.004214048385620117, 0.004355907440185547, 0.00419926643371582, 0.004180908203125, 0.00429844856262207, 0.004190921783447266, 0.004258871078491211, 0.004286050796508789, 0.004172801971435547, 0.004253864288330078, 0.004264116287231445, 0.004148960113525391, 0.004186868667602539, 0.004228115081787109, 0.004124879837036133, 0.0041179656982421875, 0.004217863082885742, 0.004322052001953125, 0.004211902618408203, 0.03515219688415527, 0.0047609806060791016, 0.004333972930908203, 0.004361867904663086, 0.004261970520019531, 0.004225969314575195, 0.004415988922119141, 0.004232168197631836, 0.0042421817779541016, 0.00431370735168457, 0.0042569637298583984, 0.004189014434814453, 0.004328012466430664, 0.004199028015136719, 0.004136085510253906, 0.004189014434814453, 0.0041141510009765625, 0.004189252853393555, 0.004303932189941406, 0.005089998245239258, 0.0046520233154296875, 0.00444483757019043, 0.0042989253997802734, 0.004374980926513672, 0.0042040348052978516, 0.004191160202026367, 0.0043528079986572266, 0.004607200622558594, 0.00436091423034668, 0.00426173210144043, 0.0043430328369140625, 0.004238128662109375, 0.004242897033691406, 0.004355192184448242, 0.004319906234741211, 0.004235029220581055, 0.004358053207397461, 0.0042798519134521484, 0.004220247268676758, 0.004347801208496094, 0.004212141036987305, 0.004242897033691406, 0.00434422492980957]
 
# Before values
result2 = [0.006510019302368164, 0.0051631927490234375, 0.004922151565551758, 0.004791736602783203, 0.004918098449707031, 0.004452228546142578, 0.004400730133056641, 0.0043141841888427734, 0.004185914993286133, 0.004270076751708984, 0.0040929317474365234, 0.004024028778076172, 0.003973722457885742, 0.004038333892822266, 0.003862619400024414, 0.003898143768310547, 0.0038909912109375, 0.003983020782470703, 0.0038847923278808594, 0.003862142562866211, 0.0038628578186035156, 0.003941059112548828, 0.003871917724609375, 0.003854990005493164, 0.003843069076538086, 0.004060983657836914, 0.003895998001098633, 0.003863811492919922, 0.00386810302734375, 0.0039479732513427734, 0.0038459300994873047, 0.0038721561431884766, 0.0038509368896484375, 0.003952980041503906, 0.003924846649169922, 0.003851175308227539, 0.003865957260131836, 0.0039348602294921875, 0.0038061141967773438, 0.003844022750854492, 0.003822803497314453, 0.004008054733276367, 0.003911018371582031, 0.003906965255737305, 0.003913164138793945, 0.004021167755126953, 0.003965854644775391, 0.003923892974853516, 0.003919839859008789, 0.003979206085205078, 0.0038950443267822266, 0.0039370059967041016, 0.003918170928955078, 0.003964900970458984, 0.003912925720214844, 0.003895998001098633, 0.0038766860961914062, 0.003973245620727539, 0.00392603874206543, 0.00391387939453125, 0.003908872604370117, 0.0039708614349365234, 0.0038907527923583984, 0.0039000511169433594, 0.003895998001098633, 0.003947019577026367, 0.0038709640502929688, 0.0038771629333496094, 0.003911018371582031, 0.004011869430541992, 0.003889799118041992, 0.003910064697265625, 0.003912925720214844, 0.0039730072021484375, 0.003862142562866211, 0.003901958465576172, 0.029926061630249023, 0.0043108463287353516, 0.004015207290649414, 0.004102945327758789, 0.0039899349212646484, 0.003971099853515625, 0.0039708614349365234, 0.004026174545288086, 0.0039119720458984375, 0.00392913818359375, 0.003875732421875, 0.004054069519042969, 0.003957033157348633, 0.003923177719116211, 0.003943920135498047, 0.003952741622924805, 0.004002094268798828, 0.003921985626220703, 0.003918170928955078, 0.003947019577026367, 0.004024982452392578, 0.003910064697265625, 0.003969907760620117, 0.004126071929931641]# Function to plot
plt.plot(list(range(0, 100)), result1, 'g--', list(range(0, 100)), result2, 'r--', )
 
# function to show the plot
plt.show()

[edublancas]

I tested it and it works well. Just a minor comment