umm_calloc undetected multiply overflow/wrap

[mhightower83]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP-12]
• Core Version: [3027aca]
• Development Env: [Arduino IDE]
• Operating System: [Ubuntu]

Settings in IDE

• Module: [Adafruit Feather HUZZAH ESP8266]
• Flash Mode: [qio]
• Flash Size: [4MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: [nodemcu]
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz]
• Upload Using: [SERIAL]
• Upload Speed: [460800] (serial upload only)

Problem Description

umm_calloc(16777217u, 256u) should fail because the buffer requested is too large; however, it succeeds and returns a buffer pointer. The problem is caused by an undetected overflow/wrap using unsigned multiply.

MCVE Sketch

void setup() {
  Serial.begin(115200);
  delay(200);

  Serial.printf("\r\nCalloc Multiply Overflow Test\r\n");

  size_t a = 16777217u, b = 256u;
  void *p = calloc(a, b);
  if (p) {
    Serial.printf("  Failed: calloc(%u, %u) returned pointer: %p - Unexpected success on failure case\r\n", a, b, p);
    free(p);
  } else {
    Serial.printf("  Passed: calloc(%u, %u); - Expected failure NULL pointer returned.\r\n", a, b);
  }
}

void loop() {}

Debug Messages

Calloc Multiply Overflow Test
  Failed: calloc(16777217, 256) returned pointer: 0x3ffef3bc - Unexpected success on failure case