Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NOT an issues! just making SD card a lot faster. #529

Closed
josef2600 opened this issue Jan 4, 2025 · 14 comments
Closed

NOT an issues! just making SD card a lot faster. #529

josef2600 opened this issue Jan 4, 2025 · 14 comments

Comments

@josef2600
Copy link

thank you for your wonderful project. there is nothing wrong here! but you didn't enable discussions, so i have to make it here. sorry for that.
the esp32-s3 is able to work with 4-bit connection to sd card. this is how:
as far as i found, in the utilsFS.cpp in static bool prepSD_MMC() , lets say i made a board named Dirty_CAMERA_ESP32S3_CAM_v1 that is connected to all 4 data pins, you add this:

  //https://github.com/espressif/arduino-esp32/tree/master/libraries/SD_MMC
  //https://github.com/espressif/arduino-esp32/blob/master/libraries/SD_MMC/examples/SDMMC_Test/SDMMC_Test.ino
#if defined(Dirty_CAMERA_ESP32S3_CAM_v1)
  //setting sd card for 4-bit data transfer instead of 1-bit
  // Please note that ESP32 does not allow pin change and setPins() will always fail.
  if(!SD_MMC.setPins(SD_MMC_CLK, SD_MMC_CMD, SD_MMC_D0, SD_MMC_D1, SD_MMC_D2, SD_MMC_D3))
  {
    Serial.println("SD card Pin change failed!");
    return false;
  }
//bool SDMMCFS::begin(const char *mountpoint, bool mode1bit, bool format_if_mount_failed, int sdmmc_frequency, uint8_t maxOpenFiles)
  res = SD_MMC.begin("/sdcard", false, 1); //mode1bit==false
  
#else
  //to use 1-bit line mode (CLK, CMD, D0, 3.3V, GND) by passing the second argument mode1bit==true:
  res = SD_MMC.begin("/sdcard", true, formatIfMountFailed);
#endif

the important change is to make mode1bit==false .
i have not tested fully, just searched the driver for them. but if something goes wrong, i will edit this.

@s60sc
Copy link
Owner

s60sc commented Jan 4, 2025

Thanks.
As per readme 4 bit mode was not beneficial on an ESP32.
It might benefit an ESP32S3 so if you find it makes a significant improvement I'll look at making it a configurable option.

@josef2600
Copy link
Author

i did read that too. buy default the esp32 is 4-bit. but as i said, i was talking about esp32-s3 .
to be honest, i am going crazy on this thing! i forked your project. if i succeed, i will tell you.
and thank you again for your work.

@josef2600
Copy link
Author

josef2600 commented Jan 8, 2025

so, i found proof! BUT ONLY FOR ESP32-S3.
if you read this (espressif/esp-idf#11628) , they have extended test for the speed of sd card.
the speed of 4 line can be 4 times faster or even faster. but there are some things needs to be set.
if you use 4-line and NOT using PSRAM for use for SD card (that i think you are using PSRAM for data saving reading for SD card), then use the maximum clock of 40 MHz for SD card (by default it is only 20 MHz, and it is available in the code i wrote SDMMCFS::begin) , and the file size is more than 81K byte, then you can get the speed of over 80 Mbps . even up to 114 Mbps. but it is only for sequential read and write. but if you make them all wrong, then the speed gets under 8 Mbps. and that is a huge latency for read and writing. but i think in this code you cannot NOT use PSRAM, since the data is too large. but it still can be much faster. just by adding the choice of using the 1-line or 4-line. but the choice of maximum speed should be always enabled for esp32-S3, but with a twist. it has to check that if the memory can work with 40mhz or not. and if not, then gets back to 20mhz.
but with all that said, i personally haven't test it. but i have ordered a new board and i will get it soon. but in the meantime, i think i can still manage the test. and if successful, i will tell it. and i will test for read write speed from PSRAM too.

@s60sc
Copy link
Owner

s60sc commented Jan 8, 2025

Interested to see results.
Note that default MMC clock speed is set by Arduino, based on each board definition and is normally 40MHz.

@josef2600
Copy link
Author

funny thing! if you use the IDF , the default setting would be at 8MHz ! but arduino sets it correctly at high speed. just not sure if it does it to 20mhz or 40mhz. but all esp32-s3 can handle 40mhz. i would say it is best to set it manually just to be sure. but, then again, i did read in the datasheet or manual that SPI2 has a special pins for high speed. but then again(!) we are not working with SPI. and that is very important to tell it to sd card too. i think for that, we have to set the D3 to VCC by a 10k resistor. if we don't do that, the sd card will work as a simple SPI mode and will become in 8mhz or lower clock. this is a very long story.

@josef2600
Copy link
Author

fun fun!! i got it! after a shit ton of work, i finally have it.
first there is a big problem with sd driver in Arduino. and they don't care. just read it :
(espressif/arduino-esp32#10835)
but any way, i tested all. as i thought, it is way faster in 4-bit.
results:

res = SD_MMC.begin("/sdcard", false, 1);
SD card Pin set in 4-bit mode

SD card initialized successfull
SD card type=SDHC
Card size:   31956MB
Total bytes: 31943MB
Used bytes:  89MB
write successfull
Starting write test, please wait.
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2473,19546,34,204
2474,29586,34,205
2494,19584,34,203
2472,19638,34,205
2377,117397,34,213
Write test finished successfully.
Checking file content, please wait.
File content correct.
Starting read test, please wait.
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
5466,1192,9,92
5468,1181,9,92
5468,1179,9,92
5469,1178,9,92
5469,1178,9,92
Read test finished successfully.
Unmounting SD card...done.
Done

+++++++++++++

SD card Pin set in 4-bit mode
res = SD_MMC.begin("/sdcard", false, 1, 40000);

SD card initialized successfull
SD card type=SDHC
Card size:   31956MB
Total bytes: 31943MB
Used bytes:  89MB
write successfull
Starting write test, please wait.
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2474,19675,34,204
2416,75505,34,210
2468,19684,34,205
2492,19595,34,203
2492,19677,34,203
Write test finished successfully.
Checking file content, please wait.
File content correct.
Starting read test, please wait.
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
5488,1217,9,92
5489,1207,9,92
5489,1204,9,92
5489,1205,9,92
5489,1204,9,92
Read test finished successfully.
Unmounting SD card...done.
Done

+++++++++++++

SD card Pin set in 1-bit mode
res = SD_MMC.begin("/sdcard", true, true, 40000);

SD card type=SDHC
Card size:   31956MB
Total bytes: 31943MB
Used bytes:  89MB
write successfull
Starting write test, please wait.
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1829,20297,34,277
1760,139326,34,289
1709,211369,34,298
1842,19693,34,276
1843,19746,34,276
Write test finished successfully.
Checking file content, please wait.
File content correct.
Starting read test, please wait.
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2928,1923,9,173
2928,1912,9,173
2928,1903,9,173
2928,1902,9,173
2928,1903,9,173
Read test finished successfully.
Unmounting SD card...done.
Done

and it is very simple test. when we go for large files, like saving the video file, it i s going to be faster.

@s60sc
Copy link
Owner

s60sc commented Jan 9, 2025

Good, I'll look at adding 4 bit mode once I have completed RTSP integration

@josef2600
Copy link
Author

josef2600 commented Jan 10, 2025

thank you. adding 4-bit is very easy. just 2 line will change.
if(!SD_MMC.setPins(SD_MMC_CLK, SD_MMC_CMD, SD_MMC_D0))
to
if(!SD_MMC.setPins(SD_MMC_CLK, SD_MMC_CMD, SD_MMC_D0, SD_MMC_D1, SD_MMC_D2, SD_MMC_D3))

like this:
  if(!SD_MMC.setPins(SD_MMC_CLK, SD_MMC_CMD, SD_MMC_D0, SD_MMC_D1, SD_MMC_D2, SD_MMC_D3))
  //if(!SD_MMC.setPins(SD_MMC_CLK, SD_MMC_CMD, SD_MMC_D0))
  {
    Serial.println("SD card Pin change failed!");
    return false;
  }

and next is:
res = SD_MMC.begin("/sdcard", true, 1, 40000);//mode1bit==true
to:
res = SD_MMC.begin("/sdcard", false, 1, 40000); //mode1bit==false
that is it.
i am scared to submit a request! because i fear i may change the structure. mainly because it is scary to change the code when you are using NOT instead of true! like this:
#if !(CONFIG_IDF_TARGET_ESP32C3)
please, can you PLEASE fix them? do not use NOT in the code. it is error prone and hard to read. instead use #if (CONFIG_IDF_TARGET_ESP32C3) for if it is C3 and elseif for others. for example in static bool prepSD_MMC() it is so confusing and hard to read. but in static void infoSD() it is OK to use it, but i don't recommend it. yes it is longer to put all of them in there, but it is safe. for example, there are esp32-c6 and H4 and few other chips too. now it becomes trouble.

on another note, the test i did was a real test and not just a speed test for the sake of speed test! i mean, it do other things when writing the file in sd card. and it does check the latency too. the latency is very important.
and i put it here if you like to play with it too! i love playing with hardware !
3.zip

regards, Josef.

@s60sc
Copy link
Owner

s60sc commented Jan 10, 2025

I prefer to remain anonymous for various reasons, but you can open discussions.

@s60sc
Copy link
Owner

s60sc commented Jan 12, 2025

new v10.5.1 allows sd 4 bit mode to be selected, but not tested

@josef2600
Copy link
Author

josef2600 commented Jan 12, 2025

I'll teste it.

@josef2600
Copy link
Author

### speed test for 1-bit and 4-bit SD card:

so, i did the speed test. no changes in code, except putting the pins defs.
short answer : double!
long answer:
1-bit :

[10:04:20.224 closeAvi] ******** AVI recording stats ********
[10:04:20.224 closeAvi] Recorded /20250112/20250112_100409_P_FHD_6_11.avi 
[10:04:20.235 closeAvi] AVI duration: 11 secs
[10:04:20.235 closeAvi] Number of frames: 65
[10:04:20.235 closeAvi] Required FPS: 30
[10:04:20.246 closeAvi] Actual FPS: 6.0
[10:04:20.246 closeAvi] File size: 5.7MB
[10:04:20.246 closeAvi] Average frame length: 92512 bytes
[10:04:20.257 closeAvi] Average frame monitoring time: 133 ms
[10:04:20.257 closeAvi] Average frame buffering time: 0 ms
[10:04:20.267 closeAvi] Average frame storage time: 35 ms
[10:04:20.267 closeAvi] Average SD write speed: 2536 kB/s
[10:04:20.278 closeAvi] File open / completion times: 87 ms / 29 ms
[10:04:20.278 closeAvi] Busy: 100%
[10:04:20.288 checkMemory]  Free: heap 165992, block: 94196, min: 118424, pSRAM 2794096
[10:04:20.289 closeAvi] *************************************
[10:04:20.299 checkFreeStorage] Storage free space: 29.7GB

and 4-bit:

[20:17:36.078 closeAvi] ******** AVI recording stats ********
[20:17:36.079 closeAvi] Recorded /20250113/20250113_201717_FHD_6_19.avi 
[20:17:36.089 closeAvi] AVI duration: 19 secs
[20:17:36.089 closeAvi] Number of frames: 110
[20:17:36.089 closeAvi] Required FPS: 30
[20:17:36.100 closeAvi] Actual FPS: 5.9
[20:17:36.100 closeAvi] File size: 6.2MB
[20:17:36.100 closeAvi] Average frame length: 59094 bytes
[20:17:36.111 closeAvi] Average frame monitoring time: 155 ms
[20:17:36.111 closeAvi] Average frame buffering time: 0 ms
[20:17:36.121 closeAvi] Average frame storage time: 13 ms
[20:17:36.121 closeAvi] Average SD write speed: 4126 kB/s
[20:17:36.132 closeAvi] File open / completion times: 1 ms / 32 ms
[20:17:36.132 closeAvi] Busy: 100%
[20:17:36.142 checkMemory]  Free: heap 166664, block: 94196, min: 112612, pSRAM 2788824
[20:17:36.143 closeAvi] *************************************
[20:17:36.153 checkFreeStorage] Storage free space: 29.7GB

as you can see, speed jumped from 2536 kB/s to 4126 kB/s . i tested them 2 times.
thanks. SD card 4-bit, works!

@s60sc
Copy link
Owner

s60sc commented Jan 13, 2025

Thanks, I've updated the readme

@josef2600
Copy link
Author

josef2600 commented Jan 13, 2025

yes i did see it. thank you.
also, i am thinking of using other libs for SD card. fat is too slow and old and so little. new libs support ExFAT that are way better.
i will check for them and tell you whats what!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants