LittleFS multiple partition mounting/reading/writing problem #8581

zekageri · 2023-08-29T13:13:12Z

Board

ESP32 Wrover

Device Description

ESP32_Wrover 16mb flash 8mb psram

Hardware Configuration

Ethernet

Version

latest master (checkout manually)

IDE Name

Platformio

Operating System

Windows10

Flash frequency

80

PSRAM enabled

yes

Upload speed

115200

Description

I made a custom partition table to separate system files and user configuration files in the flash.
I then mounted these two partitions separtely and tried to write and read from one of them.

Sketch

fs::LittleFSFS sysFS;
fs::LittleFSFS userFS;

boolean systemPartitionMounted = sysFS.begin(false,"/system",10,"system");
boolean userPartitionMounted = userFS.begin(false,"/user",10,"user");

if( debug ){
    Serial.printf("\n***** File System *****\n");

    Serial.printf("%s\n",systemPartitionMounted?"System partition is mounted":"System partition is not mounted");
    Serial.printf("Size: %d byte\n",systemPartitionMounted?sysFS.totalBytes():0);
    Serial.printf("Used: %d byte\n",systemPartitionMounted?sysFS.usedBytes():0);

    Serial.printf("***** ********** *****\n");

    Serial.printf("%s\n",userPartitionMounted?"User partition is mounted":"User partition is not mounted");
    Serial.printf("Size: %d byte\n",userPartitionMounted?userFS.totalBytes():0);
    Serial.printf("Used: %d byte\n",userPartitionMounted?userFS.usedBytes():0);

    Serial.printf("***** ********** *****\n\n");
}

if( userPartitionMounted ){
    write("/test.txt","test");
    char buffer[1024];
    read("/test.txt",buffer,1024);
    Serial.printf("%s\n",buffer);
}


boolean createPath(const char* path){
    if(userFS.exists(path)){ return true; }

    if( debug ){
        Serial.printf("[User FS] - Creating path: %s\n",path);
    }
    if (strchr(path, '/')) {
        char *pathStr = strdup(path);
        if (pathStr) {
            char *ptr = strchr(pathStr, '/');
            while (ptr) {
                *ptr = 0;
                userFS.mkdir(pathStr);
                *ptr = '/';
                ptr = strchr(ptr+1, '/');
            }
        }
        free(pathStr);
    }
    if( debug ){
        Serial.printf("[User FS] - %s %s\n",userFS.exists(path)?"Created":"Failed to create",path);
    }
    return true;
}

boolean write(const char* path, const char* message){
    createPath(path);
    File file = userFS.open(path, FILE_WRITE);
    if(!file){
        if( debug ){
            Serial.printf("[User FS] - (write) File not found: %s\n",path);
        }
        return false;
    }
    file.print(message);
    file.close();
    return true;
}

boolean read(const char* path, char* buffer, size_t size){
    File file = userFS.open(path, FILE_READ);
    if(!file){
        if( debug ){
            Serial.printf("[User FS] - (read) File not found: %s\n",path);
        }
        return false;
    }
    // Check if the file.size is greater than the buffer size
    if(file.size() > size){
        file.close();
        if(debug){
            Serial.printf("[User FS] - (read) File size is greater than buffer size\n");
        }
        return false;
    }
    for (size_t i = 0; i < size; i++){
        buffer[i] = file.read();
    }
    
    file.close();
    return true;
}

Debug Message

E (308) esp_core_dump_f��͡� Core dump data check failed:
Calculated checksum='7924d124'
Image checksum='ffffffff'

***** File System *****    
System partition is mounted
Size: 3997696 byte
Used: 8192 byte
***** ********** *****     
User partition is mounted  
Size: 10022912 byte        
Used: 8192 byte
***** ********** *****

[User FS] - Creating path: /test.txt
[User FS] - Failed to create /test.txt
E (485) esp_littlefs_api: failed to erase addr 00000000, size 00001000, err 258
[User FS] - (write) File not found: /test.txt
[User FS] - (read) File not found: /test.txt
�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������

Other Steps to Reproduce

Partition.csv

# Name,	Type,	SubType,	Offset,	Size,	Flags
nvs,	data,	nvs,	0x9000,	0x5000,
otadata,	data,	ota,	0xe000,	0x2000,
app0,	app,	ota_0,	0x10000,	0x140000,
app1,	app,	ota_1,	0x150000,	0x140000,
coredump,	data,	coredump,	0x290000,	0x10000,
system,	data,	spiffs,	0x2a0000,	0x3d0900,
user,	data,	spiffs,	0x670900,	0x98f700,

I have checked existing issues, online documentation and the Troubleshooting Guide

I confirm I have checked existing issues, online documentation and Troubleshooting guide.

The text was updated successfully, but these errors were encountered:

mrengineer7777 · 2023-08-29T14:25:17Z

Have you tried erasing all flash and reprogramming? When changing partition definitions it's necessary to write the entire flash.

Unrelated: I would expect the coredump to be at the end of flash.

zekageri · 2023-08-29T14:28:05Z

Yes, i have erased my flash. It is mounting correctly.

zekageri · 2023-08-29T14:28:45Z

I can read from sysFS but can't write to it and i can't read or write to userFS

SuGlider · 2023-08-29T15:35:13Z

@zekageri - Do you mind trying it with Arduino IDE + Arduino Core 2.0.11?

zekageri · 2023-08-29T16:03:49Z

Well, if i have to i can install it.

lbernstone · 2023-08-29T17:57:37Z

Works for me in arduino 2.0.11

#include <LittleFS.h>

fs::LittleFSFS sysFS;
fs::LittleFSFS userFS;

void setup() {
  Serial.begin(115200);
  Serial.println();
  sysFS.begin(true, "/system", 10, "system");
  userFS.begin(true, "/user", 10, "user");
  Serial.printf("Sys Size: %d byte\n",sysFS.totalBytes());
  Serial.printf("User Size: %d byte\n",userFS.totalBytes());
}

void loop() {}

partitions.csv

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x140000,
app1,     app,  ota_1,   0x150000,0x140000,
system,   data, spiffs,  0x290000,0x10000,
user,     data, spiffs,  0x2A0000,0x150000,
coredump, data, coredump,0x3F0000,0x10000,

07:54:17.557 -> rst:0xc (SW_CPU_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
07:54:17.557 -> configsip: 0, SPIWP:0xee
07:54:17.557 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
07:54:17.557 -> mode:DIO, clock div:1
07:54:17.557 -> load:0x3fff0030,len:1344
07:54:17.557 -> load:0x40078000,len:13836
07:54:17.557 -> load:0x40080400,len:3608
07:54:17.590 -> entry 0x400805f0
07:54:17.690 -> [     3][D][esp32-hal-cpu.c:244] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
07:55:08.722 -> ./components/esp_littlefs/src/littlefs/lfs.c:1229:error: Corrupted dir pair at {0x0, 0x1}
07:55:08.755 -> E (276) esp_littlefs: mount failed,  (-84)
07:55:08.755 -> E (277) esp_littlefs: Failed to initialize LittleFS
07:55:10.850 -> Sys Size: 65536 byte
07:55:10.850 -> User Size: 1376256 byte

SuGlider · 2023-08-29T19:19:38Z

Thanks @lbernstone

It may be an issue with the Platform IO Package.

zekageri · 2023-08-29T19:23:55Z

Or the partition csv? I will look again tomorrow. Thank you all for your help.

zekageri · 2023-08-29T19:24:40Z

Works for me in arduino 2.0.11

#include <LittleFS.h>

fs::LittleFSFS sysFS;
fs::LittleFSFS userFS;

void setup() {
  Serial.begin(115200);
  Serial.println();
  sysFS.begin(true, "/system", 10, "system");
  userFS.begin(true, "/user", 10, "user");
  Serial.printf("Sys Size: %d byte\n",sysFS.totalBytes());
  Serial.printf("User Size: %d byte\n",userFS.totalBytes());
}

void loop() {}

partitions.csv

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x140000,
app1,     app,  ota_1,   0x150000,0x140000,
system,   data, spiffs,  0x290000,0x10000,
user,     data, spiffs,  0x2A0000,0x150000,
coredump, data, coredump,0x3F0000,0x10000,

07:54:17.557 -> rst:0xc (SW_CPU_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
07:54:17.557 -> configsip: 0, SPIWP:0xee
07:54:17.557 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
07:54:17.557 -> mode:DIO, clock div:1
07:54:17.557 -> load:0x3fff0030,len:1344
07:54:17.557 -> load:0x40078000,len:13836
07:54:17.557 -> load:0x40080400,len:3608
07:54:17.590 -> entry 0x400805f0
07:54:17.690 -> [     3][D][esp32-hal-cpu.c:244] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
07:55:08.722 -> ./components/esp_littlefs/src/littlefs/lfs.c:1229:error: Corrupted dir pair at {0x0, 0x1}
07:55:08.755 -> E (276) esp_littlefs: mount failed,  (-84)
07:55:08.755 -> E (277) esp_littlefs: Failed to initialize LittleFS
07:55:10.850 -> Sys Size: 65536 byte
07:55:10.850 -> User Size: 1376256 byte

Wait a minute. Did you try to read and write to the partitions? Thats whats not working for me

Jason2866 · 2023-08-29T20:14:24Z

Platformio has a bug when offset is not filled in partition scheme.
platformio/platform-espressif32#1178

lbernstone · 2023-08-29T20:33:41Z

I didn't run the full test suite, but I was able to write a file to both partitions.

zekageri · 2023-08-30T04:25:35Z

Platformio has a bug when offset is not filled in partition scheme.
platformio/platform-espressif32#1178

Ohh. Thats interesting. Will try it today. Thank you.

zekageri · 2023-08-30T06:19:35Z

Well, I have replaced my partition.csv

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x140000,
app1,     app,  ota_1,   0x150000,0x140000,
system,   data, spiffs,  0x290000,0x10000,
user,     data, spiffs,  0x2A0000,0x150000,
coredump, data, coredump,0x3F0000,0x10000,

boolean systemPartitionMounted = sysFS.begin(false,"/system",10,"system");
boolean userPartitionMounted = userFS.begin(false,"/user",10,"user");
if( debug ){
    logger.write("\n***** File System *****\n");

    logger.write("%s\n",systemPartitionMounted?"System partition is mounted":"System partition is not mounted");
    logger.write("Size: %d byte\n",systemPartitionMounted?sysFS.totalBytes():0);
    logger.write("Used: %d byte\n",systemPartitionMounted?sysFS.usedBytes():0);

    logger.write("***** ********** *****\n");

    logger.write("%s\n",userPartitionMounted?"User partition is mounted":"User partition is not mounted");
    logger.write("Size: %d byte\n",userPartitionMounted?userFS.totalBytes():0);
    logger.write("Used: %d byte\n",userPartitionMounted?userFS.usedBytes():0);

    logger.write("***** ********** *****\n\n");
}

if( userPartitionMounted ){
    
    if( !write("/test.txt","test") ){ return; }
    char buffer[1024];
    if( read("/test.txt",buffer,1024) ){
        logger.write("%s\n",buffer);
    }
}

Log

***** File System *****
System partition is not mounted
Size: 0 byte
Used: 0 byte
***** ********** *****
User partition is mounted
Size: 1376256 byte
Used: 12288 byte
***** ********** *****

[User FS] - Creating path: /test.txt
[User FS] - Failed to create /test.txt

With this csv, it can not mount sysFS partition and i also can not write to userFS partition.
I have erased the flash, uploaded firmware and data folder again.

zekageri · 2023-08-30T06:56:23Z

I have upgraded my csv to match to a 16mb wrover

# Name,     Type,	SubType,    Offset,     Size,     Flags
nvs,        data,	nvs,	    0x9000,     0x5000,
otadata,    data,	ota,        0xE000,     0x2000,
app0,	    app,	ota_0,	    0x10000,	0x140000,
app1,	    app,	ota_1,	    0x150000,	0x140000,
user,	    data,	spiffs,     0x290000,   0x98f000,
system,	    data,	spiffs,     0xC1F000,   0x3d0000,
coredump,   data,	coredump,   0xFEF000,	0x10000,

I have no warnings or errors during compilation regarding the csv.
Mount is failing but is erased on fail.
After that the mount is ok
and same result.
Will try with Arduino IDE.....

./components/esp_littlefs/src/littlefs/lfs.c:4191:error: Invalid block count (16 != 2447)
E (457) esp_littlefs: mount failed,  (-22)
E (458) esp_littlefs: Failed to initialize LittleFS

***** File System *****    
System partition is mounted
Size: 3997696 byte
Used: 12288 byte
***** ********** *****     
User partition is mounted  
Size: 10022912 byte        
Used: 8192 byte
***** ********** *****

[User FS] - Creating path: /test.txt
[User FS] - Failed to create /test.txt

zekageri · 2023-08-30T07:01:21Z

Also it seems to me that PIO file system upload ( with data folder contents ) is picking the last spiffs subtyped partition to load the data. And i can read from there. If the user partition is the last, i can read from there but if i replace it with the "system" partition i can only read from there.

zekageri · 2023-08-30T07:14:27Z

But how can i add my custom partition in Arduino IDE?

P-R-O-C-H-Y · 2023-08-30T08:50:23Z

@zekageri Check DOCS - Partition table

zekageri · 2023-08-30T08:59:31Z

So after i erased the flash, filled out the offsets in the partition table and uploaded firmware and flash again, it is working fine now. I tested only one partition for now but i will test the other too soon. Thank you all for your help and support. This was a quick help from you guys!!

zekageri · 2023-08-30T09:00:16Z

And i had to swap the user and system partitions in a way that the system partition would be the last so the PIO file system uploader uploads my data folder content to this partition.

zekageri · 2023-08-30T09:01:17Z

Here is my test partition for now.

# Name,	    Type,	SubType,    Offset,	    Size,	Flags
nvs,        data,	nvs,	    0x9000,	0x5000,
otadata,    data,	ota,        0xE000,	0x2000,
app0,	    app,	ota_0,	    0x10000,	0x140000,
app1,	    app,	ota_1,	    0x150000,	0x140000,
user,	    data,	spiffs,     0x290000,   0x98f000,
system,	    data,	spiffs,     0xC1F000,   0x3d0000,
coredump,   data,	coredump,   0xFEF000,	0x10000,

zekageri · 2023-09-01T13:28:07Z

All good just the litttlefs updater missing with optional partition. ;D there is only spiffs updater

zekageri added the Status: Awaiting triage Issue is waiting for triage label Aug 29, 2023

SuGlider added Status: Needs investigation We need to do some research before taking next steps on this issue IDE: PlaformIO Issue relates to PlatformIO IDE and removed Status: Awaiting triage Issue is waiting for triage labels Aug 29, 2023

Jason2866 added Resolution: Wontfix Arduino ESP32 team will not fix the issue Status: Solved Type: For reference Common questions & problems and removed Status: Needs investigation We need to do some research before taking next steps on this issue labels Aug 30, 2023

zekageri closed this as completed Aug 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LittleFS multiple partition mounting/reading/writing problem #8581

LittleFS multiple partition mounting/reading/writing problem #8581

zekageri commented Aug 29, 2023 •

edited

Loading

mrengineer7777 commented Aug 29, 2023

zekageri commented Aug 29, 2023

zekageri commented Aug 29, 2023

SuGlider commented Aug 29, 2023

zekageri commented Aug 29, 2023

lbernstone commented Aug 29, 2023

SuGlider commented Aug 29, 2023

zekageri commented Aug 29, 2023

zekageri commented Aug 29, 2023

Jason2866 commented Aug 29, 2023 •

edited

Loading

lbernstone commented Aug 29, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023 •

edited

Loading

zekageri commented Aug 30, 2023

P-R-O-C-H-Y commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Sep 1, 2023

LittleFS multiple partition mounting/reading/writing problem #8581

LittleFS multiple partition mounting/reading/writing problem #8581

Comments

zekageri commented Aug 29, 2023 • edited Loading

Board

Device Description

Hardware Configuration

Version

IDE Name

Operating System

Flash frequency

PSRAM enabled

Upload speed

Description

Sketch

Debug Message

Other Steps to Reproduce

I have checked existing issues, online documentation and the Troubleshooting Guide

mrengineer7777 commented Aug 29, 2023

zekageri commented Aug 29, 2023

zekageri commented Aug 29, 2023

SuGlider commented Aug 29, 2023

zekageri commented Aug 29, 2023

lbernstone commented Aug 29, 2023

SuGlider commented Aug 29, 2023

zekageri commented Aug 29, 2023

zekageri commented Aug 29, 2023

Jason2866 commented Aug 29, 2023 • edited Loading

lbernstone commented Aug 29, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023 • edited Loading

zekageri commented Aug 30, 2023

P-R-O-C-H-Y commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Aug 30, 2023

zekageri commented Sep 1, 2023

zekageri commented Aug 29, 2023 •

edited

Loading

Jason2866 commented Aug 29, 2023 •

edited

Loading

zekageri commented Aug 30, 2023 •

edited

Loading