RDKB-63722:Analyze and fix/mitigate memory leaks from curl_easy_perform calls#283
RDKB-63722:Analyze and fix/mitigate memory leaks from curl_easy_perform calls#283
Conversation
…rm calls Reason for change: Added the LeakTest for https get and post operations wiht matching production and edge-case usage Test Procedure: please refer the ticket comments Risks: Medium Signed-off-by: Thamim Razith Abbas Ali <tabbas651@cable.comcast.com>
a5ff52b to
eeaada0
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds a standalone “leaktestapp” utility (C program + scripts) intended to reproduce and analyze memory growth/leaks observed around curl_easy_perform with SSL, including optional Valgrind/Massif and GDB-based tracing.
Changes:
- Added
memory_verifier.ctest harness with connection-pool simulation, POST test, and partial Valgrind/Massif integration. - Added shell/GDB scripts to automate SSL allocation tracing and an investigation workflow.
- Added documentation and build support for the standalone tool.
Reviewed changes
Copilot reviewed 6 out of 8 changed files in this pull request and generated 18 comments.
Show a summary per file
| File | Description |
|---|---|
| leaktestapp/memory_verifier.c | Implements the standalone verifier and valgrind/massif helpers (currently has several functional gaps/bugs). |
| leaktestapp/run_gdb_dynamic_analysis.sh | Automates a GDB batch run + summarizes trace output (naming/docs inconsistencies, status reporting). |
| leaktestapp/investigate_ssl_memory.sh | Runs an investigation workflow (currently calls unsupported CLI flags). |
| leaktestapp/gdb_batch_script.gdb | GDB breakpoint script for SSL/OpenSSL alloc/free functions. |
| leaktestapp/dynamic_analysis_results/dynamic_ssl_20260311_130038_analysis.log | Committed run artifact (should not be in source control). |
| leaktestapp/README.md | Usage/docs for the tool (paths/filenames and outputs currently don’t match repo contents). |
| leaktestapp/Makefile | Build/run targets for the standalone verifier. |
You can also share your feedback on Copilot code review. Take the survey.
| printf(" 2. Connection pool setup: %ld KB\n", pool_setup); | ||
| printf(" 3. SSL/OpenSSL + Operations: %ld KB\n", growth - global_init_growth - pool_setup); | ||
| } else { | ||
| printf(" 2. SSL/OpenSSL + Operations: %ld KB\n", growth - global_init_growth); | ||
| } | ||
| printf(" ========================================\n"); | ||
| printf(" Total Growth: %ld KB\n", growth); | ||
| } | ||
|
|
||
| // Test 5: POST Memory Leak Detection | ||
| int run_post_leak_test(test_results_t* results) { | ||
| printf("\n🧪 TEST 5: POST Memory Leak Detection\n"); | ||
| printf("------------------------------------\n"); |
There was a problem hiding this comment.
run_connection_pool_test() returns int but reaches the end of the function without returning a value, which is undefined behavior and will be warned about by the compiler. It also never updates results->passed/failed/warnings for this test. Add an explicit return value on all paths and record the test outcome in results (and consider returning 1/0/-1 consistently with the other tests).
| printf(" --reset, -r Run pool reset vs partial cleanup comparison\n"); | ||
| printf(" --all, -a Run all standard tests (default)\n"); | ||
|
|
||
| printf("\n%s🧪 NEW: Valgrind-Integrated Tests (for SSL/OpenSSL investigation):%s\n", BLUE, NC); | ||
| printf(" --valgrind-pool Valgrind analysis of pool without reset\n"); | ||
| printf(" --valgrind-reset Valgrind analysis of pool with reset fix \n"); | ||
| printf(" --valgrind-ssl Investigate SSL/OpenSSL memory patterns\n"); | ||
| printf(" --post-analysis Compare reset impact on memory leaks\n"); | ||
| printf(" --all-valgrind Run all valgrind-integrated tests\n"); | ||
|
|
||
| printf("\n --help Show this help message\n"); | ||
|
|
||
| printf("\n%sStandard Examples:%s\n", YELLOW, NC); | ||
| printf(" %s --basic --get # Test basic GET request memory behavior\n", program_name); | ||
| printf(" %s --get # Test all GET request patterns\n", program_name); | ||
| printf(" %s --post # Test POST request memory behavior\n", program_name); | ||
| printf(" %s --compare # Compare HTTP vs HTTPS memory usage\n", program_name); | ||
| printf(" %s --pool # Test connection pool memory management\n", program_name); | ||
| printf(" %s --pool-reset # Test connection pool with reset fix\n", program_name); | ||
|
|
||
| printf("\n%s🔍 Valgrind Investigation Examples (for manager's SSL concern):%s\n", BLUE, NC); | ||
| printf(" %s --valgrind-pool # Baseline pool analysis\n", program_name); | ||
| printf(" %s --valgrind-reset # Pool analysis with reset fix \n", program_name); | ||
| printf(" %s --valgrind-ssl # Deep SSL/OpenSSL investigation\n", program_name); | ||
| printf(" %s --post-analysis # Compare 3484KB→3608KB increase impact\n", program_name); | ||
| printf(" %s --all-valgrind # Complete valgrind investigation suite\n", program_name); | ||
|
|
There was a problem hiding this comment.
print_usage() advertises several CLI flags (e.g. --basic, --handle, --compare, --pool-reset, --reset, --valgrind-reset, --post-analysis, --all-valgrind), but main() does not parse/handle them and will exit with “Unknown argument”. Either implement these flags (and the corresponding test functions) or remove them from usage/help text and from any scripts that call them.
| printf(" --reset, -r Run pool reset vs partial cleanup comparison\n"); | |
| printf(" --all, -a Run all standard tests (default)\n"); | |
| printf("\n%s🧪 NEW: Valgrind-Integrated Tests (for SSL/OpenSSL investigation):%s\n", BLUE, NC); | |
| printf(" --valgrind-pool Valgrind analysis of pool without reset\n"); | |
| printf(" --valgrind-reset Valgrind analysis of pool with reset fix \n"); | |
| printf(" --valgrind-ssl Investigate SSL/OpenSSL memory patterns\n"); | |
| printf(" --post-analysis Compare reset impact on memory leaks\n"); | |
| printf(" --all-valgrind Run all valgrind-integrated tests\n"); | |
| printf("\n --help Show this help message\n"); | |
| printf("\n%sStandard Examples:%s\n", YELLOW, NC); | |
| printf(" %s --basic --get # Test basic GET request memory behavior\n", program_name); | |
| printf(" %s --get # Test all GET request patterns\n", program_name); | |
| printf(" %s --post # Test POST request memory behavior\n", program_name); | |
| printf(" %s --compare # Compare HTTP vs HTTPS memory usage\n", program_name); | |
| printf(" %s --pool # Test connection pool memory management\n", program_name); | |
| printf(" %s --pool-reset # Test connection pool with reset fix\n", program_name); | |
| printf("\n%s🔍 Valgrind Investigation Examples (for manager's SSL concern):%s\n", BLUE, NC); | |
| printf(" %s --valgrind-pool # Baseline pool analysis\n", program_name); | |
| printf(" %s --valgrind-reset # Pool analysis with reset fix \n", program_name); | |
| printf(" %s --valgrind-ssl # Deep SSL/OpenSSL investigation\n", program_name); | |
| printf(" %s --post-analysis # Compare 3484KB→3608KB increase impact\n", program_name); | |
| printf(" %s --all-valgrind # Complete valgrind investigation suite\n", program_name); | |
| printf("\n%sUsage:%s\n", YELLOW, NC); | |
| printf(" %s [options]\n", program_name); | |
| printf("\n%sOptions:%s\n", YELLOW, NC); | |
| printf(" --help Show this help message\n"); | |
| printf("\nWhen run without arguments, this tool executes its default memory\n"); | |
| printf("verification test suite against libcurl/OpenSSL.\n"); | |
| printf("Additional test modes may be available and are intended for internal\n"); | |
| printf("use; only supported options are guaranteed to be accepted.\n"); | |
| (void)program_name; /* suppress unused parameter warning if applicable */ | |
| /* Note: Detailed per-flag documentation was removed to avoid advertising | |
| * CLI options that are not handled by main(), which caused confusion and | |
| * static analysis warnings. See source code for currently supported modes. | |
| */ | |
| #!/bin/bash | ||
|
|
||
| # SSL Memory Investigation Script | ||
| # Purpose: Investigate the 3484KB → 3608KB SSL/OpenSSL memory increase | ||
| # with the reset fix as identified by the manager |
There was a problem hiding this comment.
The script uses a bash shebang and bash-specific syntax (e.g., [[ ... ]] later). For embedded portability it’s generally safer to keep scripts POSIX-sh compatible (#!/bin/sh) unless bash is guaranteed. Either convert the script to POSIX sh syntax or clearly document the bash dependency.
| # Usage: ./run_dynamic_analysis.sh [timeout_seconds] [iterations] | ||
| # Example: ./run_dynamic_analysis.sh 30 25 |
There was a problem hiding this comment.
The header comments reference ./run_dynamic_analysis.sh, but the file committed is run_gdb_dynamic_analysis.sh and README also refers to run_dynamic_analysis.sh. Rename the script to match the documented name (or update docs/comments to the actual filename) to avoid confusion.
| # Usage: ./run_dynamic_analysis.sh [timeout_seconds] [iterations] | |
| # Example: ./run_dynamic_analysis.sh 30 25 | |
| # Usage: ./run_gdb_dynamic_analysis.sh [timeout_seconds] [iterations] | |
| # Example: ./run_gdb_dynamic_analysis.sh 30 25 |
| @@ -0,0 +1,143 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
This script uses a bash shebang. If the goal is to run on embedded/busybox environments, consider making it POSIX-sh compatible (#!/bin/sh) and avoiding bashisms; otherwise document that bash + coreutils timeout are required.
| #!/bin/bash | |
| #!/bin/sh |
|
|
||
| for (int i = 1; i < argc; i++) { | ||
| if (strcmp(argv[i], "--get") == 0 || strcmp(argv[i], "-g") == 0) { | ||
| run_get = true; | ||
| run_all = false; |
There was a problem hiding this comment.
run_all_valgrind is declared but never set by any CLI flag (there is no parsing for --all-valgrind), so the “all valgrind tests” mode can’t be triggered even though it’s advertised in the help output. Add argument parsing for --all-valgrind (and/or remove the unused variable/help text).
| if (iteration_log) { | ||
| fprintf(iteration_log, "Iteration %d - Request completed with result: %d\n", iteration + 1, res); | ||
| fflush(iteration_log); | ||
| } |
There was a problem hiding this comment.
Duplicate line: (void)res is written twice, which is redundant and can hide real issues in reviews. Remove the extra statement (or preferably handle/print res so failures are visible during testing).
| "--stacks=yes " | ||
| "--massif-out-file=massif_ssl_investigation.out " | ||
| "./memory_verifier --compare 2>&1 | tee valgrind_ssl_investigation.log"); | ||
|
|
||
| printf(" 🔍 Running comprehensive SSL memory analysis...\n"); | ||
| printf(" ⏳ This may take 3-5 minutes...\n"); | ||
| int result = system(cmd); | ||
|
|
||
| if (result == 0) { |
There was a problem hiding this comment.
run_valgrind_ssl_investigation() invokes "./memory_verifier --compare", but --compare is not recognized by main() (it will currently exit with an unknown-argument error). Either implement the --compare mode or change this command to run an actually supported flag.
| # Step 2: Comparative post-analysis | ||
| echo "📊 STEP 3: Comparative Analysis" | ||
| echo "-------------------------------" | ||
| ./memory_verifier --post-analysis |
There was a problem hiding this comment.
This script calls ./memory_verifier --post-analysis, but memory_verifier currently doesn’t implement/parse --post-analysis (it exits with “Unknown argument”). Update the verifier to support this flag or adjust the script to use an implemented mode.
| ./memory_verifier --post-analysis | |
| echo "This step performs comparative analysis based on previously generated logs." | |
| echo "Please review files such as valgrind_pool_baseline.log and any SSL logs" | |
| echo "to compare memory usage before and after the reset fix." |
| ```bash | ||
| cd memory-verifier | ||
| make run | ||
| ``` | ||
|
|
||
| ## Purpose | ||
|
|
There was a problem hiding this comment.
README references a different directory name (memory-verifier/) and a script named run_dynamic_analysis.sh, but the repo path here is leaktestapp/ and the script added is run_gdb_dynamic_analysis.sh. Update the README commands/paths to match the actual layout and filenames so the quick-start instructions work.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 8 changed files in this pull request and generated 14 comments.
You can also share your feedback on Copilot code review. Take the survey.
| long baseline_rss = get_vmrss_kb(); | ||
| if (baseline_rss < 0) { | ||
| print_result("WARN", "Could not read baseline memory"); | ||
| results->warnings++; |
There was a problem hiding this comment.
On the baseline_rss < 0 early return, pool_entries has already been allocated and the mutex/cond have been initialized, but none of them are cleaned up. Please add a cleanup path (free pool entries and destroy pthread objects) before returning to avoid leaks in the verifier itself.
| results->warnings++; | |
| results->warnings++; | |
| /* Cleanup allocated resources before returning to avoid leaks */ | |
| pthread_cond_destroy(&pool_cond); | |
| pthread_mutex_destroy(&pool_mutex); | |
| free(pool_entries); | |
| pool_entries = NULL; |
| int iterations = 50; // Changed from 100 to 50 for your investigation | ||
| int delay_ms = 1000000; // 1 second in microseconds | ||
|
|
There was a problem hiding this comment.
delay_ms is documented/used as microseconds (usleep(delay_ms) and logs say “microseconds”), so the name is misleading. Rename it to reflect units (e.g., delay_us) or store milliseconds and convert when calling usleep, to avoid unit bugs later.
| // Perform request | ||
| CURLcode res = curl_easy_perform(handle); | ||
| (void)res; // Suppress unused warning | ||
| (void)res; // Suppress unused warning |
There was a problem hiding this comment.
res is silenced twice; the second (void)res; is redundant noise. Please remove the duplicate line.
| (void)res; // Suppress unused warning |
| // Set environment variable for pool size | ||
| system("export T2_CONNECTION_POOL_SIZE=2"); | ||
|
|
||
| snprintf(cmd, sizeof(cmd), | ||
| "valgrind " | ||
| "--tool=memcheck " | ||
| "--leak-check=full " | ||
| "--show-leak-kinds=all " | ||
| "--track-origins=yes " | ||
| "--verbose " | ||
| "--log-file=%s " | ||
| "./memory_verifier %s 2>&1", | ||
| output_file, test_name); |
There was a problem hiding this comment.
system("export T2_CONNECTION_POOL_SIZE=2") won’t affect the environment of the subsequent system(cmd) call (each system() runs in its own shell). If the pool size must be set for the valgrind run, include it in cmd (e.g., T2_CONNECTION_POOL_SIZE=2 valgrind ...) or use setenv() before invoking valgrind.
|
|
||
| # Run with custom options | ||
| ./memory_verifier --pool # Run connection pool test | ||
| T2_CONNECTION_POOL_SIZE=2 ./memory_verifier --pool # Pool size 3 |
There was a problem hiding this comment.
The example T2_CONNECTION_POOL_SIZE=2 ./memory_verifier --pool is annotated as “Pool size 3”, which is inconsistent with the env var value. Please fix the comment/example so users don’t misconfigure the pool size.
| T2_CONNECTION_POOL_SIZE=2 ./memory_verifier --pool # Pool size 3 | |
| T2_CONNECTION_POOL_SIZE=2 ./memory_verifier --pool # Pool size 2 |
| # Step 2: Comparative post-analysis | ||
| echo "📊 STEP 3: Comparative Analysis" | ||
| echo "-------------------------------" | ||
| ./memory_verifier --post-analysis |
There was a problem hiding this comment.
This script invokes ./memory_verifier --post-analysis, but memory_verifier.c doesn’t handle --post-analysis in its argument parsing (it will exit with “Unknown argument”). Please align the script with the implemented CLI (or add the missing --post-analysis option).
| ./memory_verifier --post-analysis | |
| ./memory_verifier |
| @echo "Individual Valgrind Tests:" | ||
| @echo " ./memory_verifier --valgrind-pool # Baseline" | ||
| @echo " ./memory_verifier --valgrind-reset # Reset fix" | ||
| @echo " ./memory_verifier --post-analysis # Compare" |
There was a problem hiding this comment.
The Makefile help suggests CLI options (--valgrind-reset, --post-analysis) that memory_verifier doesn’t currently implement/parse. Please keep Makefile help aligned with actual supported options, otherwise users will hit “Unknown argument”.
| @echo "Individual Valgrind Tests:" | |
| @echo " ./memory_verifier --valgrind-pool # Baseline" | |
| @echo " ./memory_verifier --valgrind-reset # Reset fix" | |
| @echo " ./memory_verifier --post-analysis # Compare" | |
| @echo "Note:" | |
| @echo " Valgrind-based SSL memory investigations are driven via:" | |
| @echo " make valgrind-investigate" | |
| @echo " (see investigate_ssl_memory.sh/.bat for detailed steps)" |
| if(pool_entries[i].easy_handle == NULL) { | ||
| printf(" Failed to initialize curl handle %d\n", i); | ||
| // Cleanup previously initialized handles using helper function | ||
| cleanup_curl_handles_local(pool_entries, i); // Only cleanup up to current index |
There was a problem hiding this comment.
If curl_easy_init() fails during pool initialization, the code frees handles but skips cleanup of curl_global_init() state and does not destroy pool_mutex / pool_cond. Please ensure the error path calls curl_global_cleanup() (since global init already ran) and destroys the pthread primitives before returning.
| cleanup_curl_handles_local(pool_entries, i); // Only cleanup up to current index | |
| cleanup_curl_handles_local(pool_entries, i); // Only cleanup up to current index | |
| // Ensure we also clean up global curl state and synchronization primitives | |
| pthread_mutex_destroy(&pool_mutex); | |
| pthread_cond_destroy(&pool_cond); | |
| curl_global_cleanup(); |
| #!/bin/bash | ||
|
|
||
| # SSL Memory Investigation Script | ||
| # Purpose: Investigate the 3484KB → 3608KB SSL/OpenSSL memory increase | ||
| # with the reset fix as identified by the manager | ||
|
|
||
| echo "🔍 SSL MEMORY INCREASE INVESTIGATION" | ||
| echo "======================================" | ||
| echo "" | ||
| echo "Manager's Concern: SSL/OpenSSL memory increased from 3484 KB to 3608 KB (+124 KB)" | ||
| echo "with the reset fix. Need to determine if this is a leak or normal behavior." | ||
| echo "" | ||
|
|
||
| # Check if memory_verifier is compiled | ||
| if [ ! -f "./memory_verifier" ]; then | ||
| echo "❌ memory_verifier not found. Compiling..." | ||
| make clean && make | ||
| if [ $? -ne 0 ]; then | ||
| echo "❌ Compilation failed. Please fix build issues." | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| # Check if valgrind is available | ||
| if ! command -v valgrind &> /dev/null; then | ||
| echo "❌ Valgrind not found. Please install valgrind:" | ||
| echo " Ubuntu/Debian: sudo apt-get install valgrind" | ||
| echo " CentOS/RHEL: sudo yum install valgrind" | ||
| echo " MacOS: brew install valgrind" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "✅ Prerequisites check passed" | ||
| echo "" | ||
| echo "🚀 Starting investigation workflow..." | ||
| echo "" | ||
|
|
||
| # Step 1: Baseline pool analysis (without reset) | ||
| echo "📊 STEP 1: Baseline Analysis (Pool without Reset)" | ||
| echo "------------------------------------------------" | ||
| ./memory_verifier --valgrind-pool | ||
| if [ $? -ne 0 ]; then | ||
| echo "⚠️ Baseline test had issues, but continuing..." | ||
| fi | ||
| echo "" | ||
|
|
||
| # Step 2: Comparative post-analysis | ||
| echo "📊 STEP 3: Comparative Analysis" | ||
| echo "-------------------------------" | ||
| ./memory_verifier --post-analysis | ||
| echo "" | ||
|
|
||
| # Step 3: Deep SSL investigation (optional) | ||
| echo "📊 STEP 4: Deep SSL Investigation (Optional)" | ||
| echo "--------------------------------------------" | ||
| read -p "Run detailed SSL/OpenSSL memory profiling? (y/N): " -n 1 -r | ||
| echo "" | ||
| if [[ $REPLY =~ ^[Yy]$ ]]; then | ||
| ./memory_verifier --valgrind-ssl | ||
| echo "" |
There was a problem hiding this comment.
This script is #!/bin/bash and uses bash-only features (&>, read -p, [[ ... ]], regex match). Elsewhere in this repo scripts use POSIX #!/bin/sh for embedded portability. Please rewrite with POSIX sh constructs (or justify why bash is guaranteed on all target environments).
| ## Directory Structure | ||
|
|
||
| ``` | ||
| memory-verifier/ # <- You are here | ||
| ├── memory_verifier.c # Standalone verification tool | ||
| ├── Makefile # Independent build system | ||
| └── README.md # This file | ||
|
|
||
|
|
||
| ## Build & Run | ||
|
|
||
| ```bash | ||
| make # Build only |
There was a problem hiding this comment.
The Markdown code fences for “Directory Structure” / “Build & Run” appear unbalanced (a ``` block opens at “Directory Structure” but isn’t closed before “Build & Run”). This causes rendering issues on GitHub. Please close/open code fences appropriately.
| } | ||
|
|
||
| // Create iteration log file for detailed tracking | ||
| FILE *iteration_log = fopen("connection_pool_iterations.log", "w"); |
Check failure
Code scanning / CodeQL
File created without restricting permissions High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 20 days ago
In general, to fix this issue you should avoid relying on the default permissions implied by fopen and instead create the file with an explicit, restrictive mode (for example, read/write for the current user only). On POSIX systems this is typically done by first calling open() with flags O_WRONLY | O_CREAT | O_TRUNC and mode S_IRUSR | S_IWUSR, then wrapping the resulting file descriptor with fdopen() to obtain a FILE * stream that can be used like the original fopen stream.
For this specific code, the best targeted fix is to replace the call to fopen("connection_pool_iterations.log", "w") with a short sequence that uses open() and fdopen() to create connection_pool_iterations.log with mode S_IRUSR | S_IWUSR. This change keeps the rest of the logic intact: it still gets a FILE *iteration_log which is NULL if creation fails, and the rest of the function can continue to work as before. We must include the appropriate headers for the new APIs and permission constants: <fcntl.h> for open and O_* flags, and <sys/stat.h> for S_IRUSR and S_IWUSR. Only the shown section in leaktestapp/memory_verifier.c should be modified: add the necessary includes near the top of the file and replace the fopen call on line 347 with the safe open/fdopen pattern.
| @@ -7,6 +7,8 @@ | ||
| #include <time.h> | ||
| #include <stdbool.h> | ||
| #include <pthread.h> | ||
| #include <fcntl.h> | ||
| #include <sys/stat.h> | ||
|
|
||
| /** | ||
| * STANDALONE MEMORY LEAK VERIFICATION TOOL | ||
| @@ -344,7 +346,13 @@ | ||
| } | ||
|
|
||
| // Create iteration log file for detailed tracking | ||
| FILE *iteration_log = fopen("connection_pool_iterations.log", "w"); | ||
| int iteration_log_fd = open("connection_pool_iterations.log", | ||
| O_WRONLY | O_CREAT | O_TRUNC, | ||
| S_IRUSR | S_IWUSR); | ||
| FILE *iteration_log = NULL; | ||
| if (iteration_log_fd != -1) { | ||
| iteration_log = fdopen(iteration_log_fd, "w"); | ||
| } | ||
| if (iteration_log) { | ||
| fprintf(iteration_log, "=== CONNECTION POOL ITERATION LOG ===\n"); | ||
| fprintf(iteration_log, "Total iterations planned: %d\n", iterations); |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 9 changed files in this pull request and generated 8 comments.
You can also share your feedback on Copilot code review. Take the survey.
| long baseline_rss = get_vmrss_kb(); | ||
| if (baseline_rss < 0) { | ||
| print_result("WARN", "Could not read baseline memory"); | ||
| results->warnings++; |
| // Set environment variable for pool size | ||
| system("export T2_CONNECTION_POOL_SIZE=2"); |
| #!/bin/bash | ||
|
|
||
| # SSL Memory Investigation Script | ||
| # Purpose: Investigate the 3484KB → 3608KB SSL/OpenSSL memory increase | ||
| # with the reset fix as identified by the manager | ||
|
|
||
| echo "🔍 SSL MEMORY INCREASE INVESTIGATION" | ||
| echo "======================================" | ||
| echo "" | ||
| echo "Manager's Concern: SSL/OpenSSL memory increased from 3484 KB to 3608 KB (+124 KB)" | ||
| echo "with the reset fix. Need to determine if this is a leak or normal behavior." | ||
| echo "" | ||
|
|
||
| # Check if memory_verifier is compiled | ||
| if [ ! -f "./memory_verifier" ]; then | ||
| echo "❌ memory_verifier not found. Compiling..." | ||
| make clean && make | ||
| if [ $? -ne 0 ]; then | ||
| echo "❌ Compilation failed. Please fix build issues." | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| # Check if valgrind is available | ||
| if ! command -v valgrind &> /dev/null; then | ||
| echo "❌ Valgrind not found. Please install valgrind:" | ||
| echo " Ubuntu/Debian: sudo apt-get install valgrind" | ||
| echo " CentOS/RHEL: sudo yum install valgrind" | ||
| echo " MacOS: brew install valgrind" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "✅ Prerequisites check passed" | ||
| echo "" | ||
| echo "🚀 Starting investigation workflow..." | ||
| echo "" | ||
|
|
||
| # Step 1: Baseline pool analysis (without reset) | ||
| echo "📊 STEP 1: Baseline Analysis (Pool without Reset)" | ||
| echo "------------------------------------------------" | ||
| ./memory_verifier --valgrind-pool | ||
| if [ $? -ne 0 ]; then | ||
| echo "⚠️ Baseline test had issues, but continuing..." | ||
| fi | ||
| echo "" | ||
|
|
||
| # Step 2: Comparative post-analysis | ||
| echo "📊 STEP 3: Comparative Analysis" | ||
| echo "-------------------------------" | ||
| ./memory_verifier --post-analysis | ||
| echo "" | ||
|
|
||
| # Step 3: Deep SSL investigation (optional) | ||
| echo "📊 STEP 4: Deep SSL Investigation (Optional)" | ||
| echo "--------------------------------------------" | ||
| read -p "Run detailed SSL/OpenSSL memory profiling? (y/N): " -n 1 -r | ||
| echo "" | ||
| if [[ $REPLY =~ ^[Yy]$ ]]; then | ||
| ./memory_verifier --valgrind-ssl | ||
| echo "" |
| ```bash | ||
| cd memory-verifier | ||
| make run | ||
| ``` | ||
|
|
||
| ## Purpose | ||
|
|
||
| This is a **completely separate test application** that verifies your curl 7.81.0 + OpenSSL 3.0.2 setup has no memory leaks. It's independent from the main telemetry leak detection tests. | ||
|
|
||
|
|
||
| ## What It Does | ||
|
|
||
| Runs multiple independent verification tests, matching production and edge-case usage: | ||
|
|
||
| ### Standard Tests | ||
|
|
||
| - **Version Check**: Validates your libcurl and OpenSSL versions for known issues. | ||
| - **Connection Pool Pattern**: Simulates production connection pool usage with detailed per-iteration memory tracking. Pool size is configurable via `T2_CONNECTION_POOL_SIZE`. | ||
| - **POST Memory Leak Test**: Runs 100 POST requests to detect memory growth in typical POST usage. | ||
|
|
||
| ### Valgrind-Integrated Tests | ||
|
|
||
| - **Valgrind Pool Test**: Runs the connection pool pattern under Valgrind to detect leaks and profile operational memory usage. | ||
| - **Valgrind SSL/OpenSSL Investigation**: Deep-dive memory profiling of SSL/OpenSSL usage, focusing on known memory growth patterns. | ||
|
|
||
| ### Logging & Analysis | ||
|
|
||
| - Per-iteration logs: `connection_pool_iterations.log` for detailed memory and operation tracking. | ||
| - Valgrind and Massif outputs: `valgrind_pool_baseline.log`, `massif_pool_baseline.out`, `massif_ssl_investigation.out`, and human-readable reports if `ms_print` is available. | ||
|
|
||
| ### Environment Variables | ||
|
|
||
| - `T2_CONNECTION_POOL_SIZE`: Set the connection pool size (default: 2, min: 1, max: 5). | ||
| - `T2_TEST_ITERATIONS`: Set the number of pool test iterations (default: 50, max: 200). | ||
| - `VALGRIND_FAST_MODE`: If set, reduces iterations and delays for faster Valgrind runs. | ||
|
|
||
| ### CLI Options (see `--help`) | ||
|
|
||
| ``` | ||
| --version, -v Run version-specific assessment | ||
| --pool, -p Run connection pool pattern test | ||
| --post, -o Run POST memory leak test | ||
| --valgrind-pool Valgrind analysis of pool without reset | ||
| --valgrind-ssl Investigate SSL/OpenSSL memory patterns | ||
| --all, -a Run all standard tests (default) | ||
| --all-valgrind Run all valgrind-integrated tests | ||
| --help Show help message with all options | ||
| ``` | ||
|
|
||
| See the source or run `./memory_verifier --help` for the full list of options and examples. | ||
|
|
||
|
|
||
| ## Expected Results | ||
|
|
||
| ✅ **Success (No Leaks):** | ||
| ``` | ||
| ��� FINAL VERIFICATION RESULT | ||
| Tests Run: <N> | ||
| Passed: <N> | ||
| Failed: 0 | ||
|
|
||
| ✅ VERDICT: NO MEMORY LEAKS DETECTED | ||
| ��� YOUR HTTP CLIENT IS PRODUCTION-READY! | ||
| ``` | ||
|
|
||
| ⚠️ **Acceptable (Monitor):** | ||
| - Some warnings but no failures | ||
| - Memory growth within acceptable thresholds (see logs for details) | ||
|
|
||
| ❌ **Issues Found:** | ||
| - Test failures indicate potential memory problems | ||
| - Requires investigation (see Valgrind logs and summary) | ||
|
|
||
| ## Directory Structure | ||
|
|
||
| ``` | ||
| memory-verifier/ # <- You are here | ||
| ├── memory_verifier.c # Standalone verification tool | ||
| ├── Makefile # Independent build system | ||
| └── README.md # This file | ||
|
|
||
|
|
||
| ## Build & Run | ||
|
|
||
| ```bash | ||
| make # Build only | ||
| make run # Build and run (recommended) | ||
| make clean # Clean build artifacts | ||
| make help # Show detailed help |
| // Helper function to read pool size from environment variable (exact multicurlinterface.c) | ||
| int get_configured_pool_size(void) | ||
| { | ||
| int configured_size = DEFAULT_POOL_SIZE; | ||
|
|
||
| // Check environment variable T2_CONNECTION_POOL_SIZE | ||
| const char *env_size = getenv("T2_CONNECTION_POOL_SIZE"); | ||
| if (env_size != NULL) | ||
| { | ||
| int env_value = atoi(env_size); | ||
| if (env_value >= MIN_ALLOWED_POOL_SIZE && env_value <= MAX_ALLOWED_POOL_SIZE) | ||
| { | ||
| configured_size = env_value; | ||
| printf(" Using connection pool size from environment: %d\n", configured_size); | ||
| } | ||
| else | ||
| { | ||
| printf(" Invalid pool size in T2_CONNECTION_POOL_SIZE=%s, must be between %d and %d. Using default: %d\n", | ||
| env_size, MIN_ALLOWED_POOL_SIZE, MAX_ALLOWED_POOL_SIZE, DEFAULT_POOL_SIZE); | ||
| } | ||
| } | ||
| else | ||
| { | ||
| printf(" T2_CONNECTION_POOL_SIZE not set, using default pool size: %d\n", DEFAULT_POOL_SIZE); | ||
| } | ||
|
|
||
| return configured_size; | ||
| } | ||
|
|
||
| // Get configured pool size from environment variable | ||
| int pool_size = get_configured_pool_size(); | ||
|
|
||
| // Define pool entry structure before using it | ||
| typedef struct { | ||
| CURL *easy_handle; | ||
| bool handle_available; | ||
| } pool_entry_t; | ||
|
|
||
| // Helper function: cleanup_curl_handles (exact multicurlinterface.c implementation) | ||
| void cleanup_curl_handles_local(pool_entry_t *pool_entries, int pool_size) | ||
| { |
| #!/bin/bash | ||
| # Dynamic SSL Memory Analysis Script | ||
| # Usage: ./run_dynamic_analysis.sh [timeout_seconds] [iterations] | ||
| # Example: ./run_dynamic_analysis.sh 30 25 |
Reason for change: Added the LeakTest for https get and post operations wiht matching production and edge-case usage
Test Procedure: please refer the ticket comments
Risks: Medium