Add job to run the sample and check for memory leak #2071

sirknightj · 2024-10-25T22:54:31Z

Issue #, if available:
N/A

What was changed?

GitHub actions CI. Run the master and viewer samples both with valgrind to check for memory leaks. Instead of kill with signal 9, it interrupts the process (signal 2) and also checks for graceful exit.

Why was it changed?

Reduce the number of times I need to manually run this check. Automating this check will also give confidence to future developers that their change did not introduce a memory leak.

How was it changed?

Script overview:
- Disable logs by setting log level to 7 (silent).
- Run the master and viewer samples in the background with valgrind. The valgrind output is captured into two different files, one for each.
- After 30 seconds, interrupt the viewer (disconnect)
- After 30 more seconds, interrupt the master.
- Wait for processes to exit & capture their exit status.
- After 10 seconds, kill the master and viewer processes. Killing the processes will make them exit with non-zero exit status.
- Check if both exited with status 0 (whether they exit gracefully on their own after interrupt).
- Check the valgrind output files and verify that no memory leaks are detected.

What testing was done for the changes?

Run it on the GitHub actions runner. Take a look at the image:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sirknightj · 2024-10-26T01:37:48Z

Output when I added a memory leak for verification:

.github/workflows/ci.yml

* Reduce hash table size for SDP transceivers (#2067) * Reduce hash table size for SDP transceivers (#1914) * Modify hash table sizes * Add logs * Fix hash table size to be set according to number of m-lines and unique supported codecs * CI branch * Update actions to use node20 * Unit test for fake transceiver * Cleanup * Fix build failure * Fix unit test * Initialize to 0 * PR description check: Reduce minimum character count and print the current character count * Use PR number instead of manually constructing link, add fetch error logging * Print out the PR link * Debug the CI * Fix the extraction logic * Clang-format --------- Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Add job to run the sample and check for memory leak (#2071) * Add job to run the sample and check for memory leak * Simulate a memory leak * MEMALLOC instead of MALLOC * Remove simulated memory leak * Print the exit statuses in case of failure * Move offer from stack to heap (#2070) * Ensure remote description does not live in memory for viewer duration (#1915) * Ensure remote description does not live in memory for viewer duration * Fix memory leak * Free the allocated memory upon failure, also add missing enter, leave and chk_log_error in peerconnection * Working reduced stack sizes. Adjusted default stack size to 64kb, moved Offer/Answer messages to be on heap instead of stack * Revert setting the stack sizes by default; Samples: revert disableSenderSideBandwidthEstimation to TRUE by default * Add missing malloc out of memory checks * Undo samples/Common.c changes * Add chk_log_err to freeJitterBuffer and freeKvsRtpTransceiver * Address comments: add additional null checks and adjust comments * Clang-format --------- Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> Co-authored-by: James Delaplane <delaplan@amazon.com> * Memory Optimization - ICE Agent Stats (#1947) (#2074) * Ice memory reduction - enable ice stats flag (#1947) * Change params size * Use dyanmic allocation and flag for ice stats * Debug 1 * Revert "Debug 1" This reverts commit ad7d02f. * Revert "Use dyanmic allocation and flag for ice stats" This reverts commit bf9a2ee. * Working version * enable flag in samples * Add unit test * Fix bug * README for the flag * Update readme * Fix readme typos * clang format * Add ENABLE_STATS_CALCULATION_CONTROL CMake option and compiler flag. Update ReadMe. * Add IceStatsControlOn tests * Add CI test forENABLE_STATS_CALCULATION_CONTROL=TRUE * Clang formatting * Remove unused iceAgentAddConfig * Remove unintended readme merge changes "Controlling RTP rolling buffer capacity" * Address comments, add null check * Cleanup unused variable * Rename and negate variable * Move setting of enableIceStats to fix segfault * Revert member size savings changes (to be applied in separate PR) * Move other instance setting of enableIceStats, clang format. * Fix setting of enableIceStats in tests * Address comment --------- Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Memory Optimization - Refactor TWCC (#1934) (#2075) * Refactor TWCC (#1934) * Remove stackqueue usage * Use hashtable instead..working logic * Cleanup, increase hash table size, fix loop bug * method 2, array of pointers * Add rolling window * rolling window with hashtable * hash table with rw * Fix bug * Fix twcc unit test * Cleanup rw logic * Cleanup * Cleanup logic * Update README * unused var fix * Use defines for hash table size * Address comments, disable TWCC by default * readme * Fix windows gst issue * Comments * Remove enableIceStats merge conflict * Address comments * Fix left over typo * Address comments, refactor updaing of TWCC hash table in onRtcpTwccPacket for testing * Clang format, fix compiler Werror * Correct typos * Fix "expression result unused" Werrors * Remove unused variable * Address comments * Fix endless loop * Address comment * Add removal of null hashTable items, add test for null item in hashTable * Address comments * Cleanup freeing of KvsPeerConnection * Clang format --------- Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Shorten Stats String Lengths (#2079) * Memory Optimization - Rolling Buffer (#2077) * Rb size config (#1921) * Reduce default size * Rolling buffer configurability * Add configurability * Allow setting parameter through trx instead of config * Add UT, fix the members to DOUBLE * Remove unnecessary fields * Rolling buffer readme update * Use enums for min values * Add usage example in samples * Fix unit test * Memset init in UTs * Modify the bitrates in the samples based on codec used in our samples * Modify unit tests, add video/audio specific defaults * Add missing null check * Readme update * Address nits * Clang fix * Logs for RB * set explicit type cast * add max values * Refactor RB logic, readme update, sample cleanup * nits * Fix unit tests * return * update description in header * fix unused var * minor changes * typo fix * review comments * sdp, stun, rtp * comments * cancelled builds * log status * sctp * Revert unrelated sample changes * Revert unrelated CMake changes * Revert unrelated test changes * Revert unrelated ReadMe changes * Fix DEFAULT_MTU_SIZE variable name * Cleanup merge changes * Fixup and comment on Rtp.c * Fixup Rtp.h * Fixup ReadMe * Clang format * Revert all sample changes * Address comments * Add back accidentally deleted line * Address comment * Update README.md --------- Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Add option to specify stack size to pthread_create (#2073) * Thread stack sizes (#1924) * Working reduced stack sizes. Adjusted default stack size to 64kb, moved Offer/Answer messages to be on heap instead of stack * Custom stack sizes in Common.c * CMake flag working and added to README * Adjust connection listener stack size for datachannels * Change stack size for media related threads and connection listener again. Add README notes to highlight this potential problem threads for other applications * Unnecessary memset * nit picks * Update CMakeLists.txt fix for ZLIB since it is not found on Windows, that way we can verify the stack side change on Windows platform, previously it was not running at all due to build fail. * Update CMake dependency back to develop since the other PRs have merged * README update --------- Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * Stack size configurablility * Revert setting specific thread stack sizes in the sample and SDK * Check develop branch * Check stack size is forwarded to PIC correctly * Set e * Remove duplicate mkdir build * Print out CMake logs in case of error * Missing fi * Remove square brackets * Dont build samples * Build samples * Pass the stack size down and dont build samples * Remove constrained device * Bump kvsCommonLws from 1.5.4RC to 1.5.4 * Add v * Update README.md --------- Co-authored-by: jdelapla <delaplan@amazon.com> Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> * SDP Attribute Fixes (#2082) * fix nack pli overwrite bug (#2035) * fix nack pli overwrite bug * create generic strings instead of hard coding * remove stale comment in tests * Check that the sdp lines are not missing, append SDP_LINE_SEPARATOR to lines checked * Remove duplicate session attribute lines, add missing attribute name * Clang format --------- Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com> * ReadMe Fixes (#2083) * Add copy code buttons to github clone commands (#1928) * Add copy code buttons to github clone commands * Add language specifier * build -> built (#1925) * Update clone command * Remove Duplicate ReadMe Lines (#2085) * Update README.md * Update make commad * Bump project version in CMake (#2086) * Upgrade to macos-13 and 15, use DRY for Mac jobs (#2090) * Upgrade to macos-13 and 15, use DRY for Mac jobs, use docker for ubuntu jobs * Revert Ubuntu docker changes * Remove Mac+GCC+Tests combo * Install GStreamer for the mac jobs to check the GStreamer sample builds * Remove unnecessary installs * Build and run the tests without AWS SDK * Add D * Samples: Fit the RTP transceiver rolling buffers to fit the set of sample frames (#2089) * Properly fit the rolling buffers to fit the set of sample frames * Clang-format * Adjust the comment for the rolling buffer configuration for the sample frames * Bump the version from 1.11.1 to 1.12.0 (#2092) * Specify configureTransceiverRollingBuffer as a Public API --------- Co-authored-by: sirknightj <jggunawa@amazon.com> Co-authored-by: Divya Sampath Kumar <disa6302@colorado.edu> Co-authored-by: James Delaplane <delaplan@amazon.com> Co-authored-by: Hassan Sahibzada <hsahibza@amazon.com>

Add job to run the sample and check for memory leak

40164e4

sirknightj marked this pull request as ready for review October 25, 2024 23:25

sirknightj added the CI Automated testing label Oct 25, 2024

sirknightj added 2 commits October 25, 2024 18:25

Simulate a memory leak

de95a69

MEMALLOC instead of MALLOC

03e6977

Remove simulated memory leak

433a730

stefankiesz reviewed Oct 31, 2024

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

stefankiesz approved these changes Oct 31, 2024

View reviewed changes

unicornss approved these changes Oct 31, 2024

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

Print the exit statuses in case of failure

6cd2198

sirknightj merged commit 499eca3 into develop Nov 1, 2024
26 checks passed

sirknightj deleted the valgrind-job branch November 1, 2024 05:05

sirknightj added the v1.12.0 label Nov 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add job to run the sample and check for memory leak #2071

Add job to run the sample and check for memory leak #2071

sirknightj commented Oct 25, 2024 •

edited

Loading

sirknightj commented Oct 26, 2024

Add job to run the sample and check for memory leak #2071

Add job to run the sample and check for memory leak #2071

Conversation

sirknightj commented Oct 25, 2024 • edited Loading

sirknightj commented Oct 26, 2024

sirknightj commented Oct 25, 2024 •

edited

Loading