Improve QSPIFBlockDevice conformance to SFDP #11531

kyle-cypress · 2019-09-19T19:43:12Z

Description

Proposed fix for #11530 .

In several cases QSPIFBlockDevice does not conform to the SFDP standard JESD216. These include:

Status register reading and writing
Soft resetting
Enabling of 4-byte addressing
Enabling of quad output mode
Use of the legacy erase instruction

In these cases QSPIFBlockDevice often assumes that certain instructions or methods are universal across flash devices when they are not. This can cause issues in devices that do not match these assumptions. This pull request resolves many of these issues and better aligns QSPIFBlockDevice with JESD216.

This PR is dependent on #11604

Pull request type

[x] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[ ] Test update
[ ] Breaking change

Reviewers

Release Notes

kyle-cypress · 2019-09-19T19:43:33Z

This is a WIP submission to gather feedback while we finalize our internal review and testing.

ciarmcom · 2019-09-19T21:00:29Z

@kyle-cypress, thank you for your changes.
@ARMmbed/mbed-os-storage @ARMmbed/mbed-os-core @ARMmbed/mbed-os-test @ARMmbed/mbed-os-hal @ARMmbed/mbed-os-maintainers please review.

0xc0170 · 2019-09-20T08:29:26Z

components/storage/blockdevice/COMPONENT_QSPIF/QSPIFBlockDevice.cpp

@@ -380,7 +380,7 @@ int QSPIFBlockDevice::erase(bd_addr_t addr, bd_size_t in_size)
    int type = 0;
    uint32_t offset = 0;
    uint32_t chunk = 4096;
-    unsigned int cur_erase_inst = _erase_instruction;
+    qspi_inst_t cur_erase_inst = _erase_instruction;


What is the reason for this change? I dont see anything in the commit msg to explain.

Consistency - the current code uses a mixture of signed and unsigned integers to represent instruction values
Clarity - convey the meaning better of the data than a generic int (and encourage consistency)
Efficiency: The instruction is only 8 bits wide, so size the data type to fit.

0xc0170 · 2019-09-26T14:24:55Z

@ARMmbed/mbed-os-storage Can you please review this proposal?

0xc0170 · 2019-09-26T14:28:23Z

drivers/QSPI.h

@@ -160,7 +166,7 @@ class QSPI : private NonCopyable<QSPI> {
     *  @returns
     *    Returns QSPI_STATUS_SUCCESS on successful reads and QSPI_STATUS_ERROR on failed reads.
     */
-    qspi_status_t read(int instruction, int alt, int address, char *rx_buffer, size_t *rx_length);
+    qspi_status_t read(qspi_inst_t instruction, int alt, int address, char *rx_buffer, size_t *rx_length);


int should be sufficient for inst as it can be any number, narrow down this won't make much difference here.

Same as alt, address and others.

This is breaking change, so needs to be justified to go in.

The usage of this in the block device was wrong, should follow what the driver specifies.

The usage of this in the block device was wrong, should follow what the driver specifies.

Can you clarify what you mean? Are you saying that the use of unsigned int instead of int in the block device was wrong?

int should be sufficient for inst as it can be any number, narrow down this won't make much difference here.

Same as alt, address and others.

This is breaking change, so needs to be justified to go in.

This change is not mandatory for the fix here so I can pull it out of this review and into a separate PR if it makes things easier. But in principle, shouldn't values which correspond to bit patterns of a particular width be represented as types of particular widths? If so, then I would argue that a separate PR should also replace alt and address with more specific types.

I'm not that familiar with SQPIF, but if commands are limited to 8 bytes, then I would say that this is acceptable, and it is not a breaking change.

Mbed OS have not guaranteed to have binary compatibility, so moving from int to uint8_t should be OK.

0xc0170 · 2019-09-26T14:30:53Z

This again changes a driver and the block device - as these are two different areas, it would be good to split them, driver first - if accepted, block device can be updated. Otherwise this takes a longer time to review and integrate (they are dependent on each other).

SeppoTakalo

There are lots of changes not related to QSPIF, can you split your changes to separate PRs so that we can properly review each change on its own.

Currently I'm not able to understand why this PR touches things like LittleFS, KVStore and DeviceKey.

kyle-cypress · 2019-09-27T22:02:33Z

Yes, I will split out the related refactoring and cleanups into their own PRs that this one can then depend on.

kyle-cypress · 2019-10-01T00:13:41Z

Separated out #11604 for the qspi_inst_t change (will rebase this PR to remove that commit soon)

kyle-cypress · 2019-10-03T23:47:03Z

I've made the following updates:

Rebased onto Introduce qspi_inst_t type for QSPI instructions #11604 . This can be rebased onto master instead if necessary, but when I tried it was not applying cleanly due the changes in types, so I'd like to hold off on doing so unless it is looking like Introduce qspi_inst_t type for QSPI instructions #11604 is not going to go in before this one.
The lfs changes were extracted into Fix buffer overrun in lfs_cache_zero #11614 (which may be superceded by LittleFS: Update to v1.7.2 #11624). One of the two is required for features-storage-tests-kvstore-filesystemstore_tests to pass, but there is not a strict ordering dependency.
Removed the devicekey/tdb changes. A modified variant of one portion of the devicekey tests was extracted as Make devicekey remainder test more meaningful #11628 . The broader refactoring will be submitted as an additional PR.
I left the changes to enable more tests on PSoC6 in this PR.

SeppoTakalo · 2019-10-04T16:03:19Z

components/storage/blockdevice/COMPONENT_QSPIF/QSPIFBlockDevice.cpp


    _mutex.lock();

+    // All commands other than Read and RSFDP use default 1-1-1 bus mode (Program/Erase are constrained by flash memory performance more than bus performance)
+    _qspi.configure_format(QSPI_CFG_BUS_SINGLE, QSPI_CFG_BUS_SINGLE, _address_size, QSPI_CFG_BUS_SINGLE,


Do you need to check return value from configure_format()?

Yes, the return value needs to be checked for configure_format across the board, but that is handled in #11603 (applying the check to additional/relocated configure_format calls would be handled during the rebase of whichever PR goes in second).

maclobdell · 2019-10-08T19:20:14Z

@ARMmbed/mbed-os-maintainers - Can this be merged. It seems the issues have been addressed. Please confirm.

adbridge · 2019-10-11T11:09:08Z

CI started

Use a vendor id check to only perform this enable on devices which define the second configuration register where the fast mode enable bit lives. Change _enable_fast_mode to use the standard status register reading and writing functions

4-byte addressing has been seen to cause failures on NORDIC boards and with Macronix memories. Suppress the attempt to enable it on that hardware (via vendor quirks and a target check) until either the failure cause can be fixed or a more robust suppression mechanism is implemented.

The addition of trace logging during greentea tests pushes the multithreaded read-write test beyond the limits of the stack it allocates for its threads. The increase of 128 bytes was chosen by experimentation.

kyle-cypress · 2019-11-13T00:10:51Z

1.) Rebased on latest master in hopes that, if I understood the comment correctly, this will pull in a fix for the issue @jeromecoutant was encountering.
2.) Slightly increased the thread stack size for the block device multithreaded test to fix the stack overflow.

This will probably still fail on NRF52840_DK with a timeout. This is because the extensive debug-level tracing in QSPIFBlockDevice, which is now enabled during the CI, causes the general block device test to slow down substantially while it waits for the UART (in my testing it increased from ~2 minutes to 10-15). The kvstore static test looks like it has a similar issue. @maclobdell plans to seek agreement on the proper way to resolve this early tomorrow morning.

jeromecoutant · 2019-11-13T08:41:04Z

debug-level tracing (...) is now enabled during the CI

@0xc0170 Really? trace should not be enabled by default....

0xc0170 · 2019-11-13T09:39:15Z

@jeromecoutant This was to help debugging the previous issues, before we start a new CI, will disable that.

0xc0170 · 2019-11-13T10:58:17Z

CI restarted (tracing disabled)

mbed-ci · 2019-11-13T12:01:50Z

Test run: FAILED

Summary: 1 of 11 test jobs failed
Build number : 8
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_greentea-test

0xc0170 · 2019-11-13T12:10:08Z

@kyle-cypress One failure for k64f in the logs, please review

mbedgt: :463::FAIL: Expected 0 Was 263 [1573647330.85][CONN][RXD] :463::FAIL: Expected 0 Was 263

Looks like deinit test case failing only, @ARMmbed/mbed-os-storage would you be able to reproduce this one for k64f?

adbridge · 2019-11-13T13:35:43Z

@kyle-cypress Looks like the failure is K64F on gcc_arm:

[1573647393.38][CONN][INF] found KV pair in stream: {{__testcase_finish;SEC_kvstore_deinit;1;0}}, queued...
[1573647393.48][CONN][RXD] >>> Test cases: 110 passed, 1 failed with reason 'Test Cases Failed'
[1573647393.48][CONN][RXD] >>> TESTS FAILED!

maclobdell · 2019-11-13T15:05:02Z

@adbridge, @0xc0170 - K64F does not have QSPIF. I strongly believe these failures are unrelated to the changes in this PR and are a result of unstable kvstore tests.

maclobdell · 2019-11-13T15:49:41Z

after talking to the maintainers, I think the K64F test fail could be real. I will test it manually. Since the kvstore tests were touched and this PR is the only one that K64F kvstore tests are failing, it needs t obe investigated. Perhaps another increase of stack size is necessary. Will try to resolve by the end of today. Any help and support anyone can provide is much appreciated.

kyle-cypress · 2019-11-13T15:59:33Z

Thanks @maclobdell . Please let me know if you need any help debugging. For reference, the commit that changed that test is 92829bd (a1c7403 also touches that file, but just to add PSOC6 to the set of targets on which it runs).

kyle-cypress · 2019-11-13T16:02:06Z

The kvstore failure on K64F definitely seems to be intermittent though. In the second run of the results referenced by #11531 (comment), the kvstore tests pass on K64F and the only failure was networking related.

adbridge · 2019-11-13T16:45:23Z

@0xc0170 CI now seems to have passed on this ?

0xc0170 · 2019-11-13T16:54:45Z

yes, restarted as above made a point about the last run (this confirms it, 2x subsequent run are all OK). Lets get this in and check nightly in the morning

maclobdell · 2019-11-13T22:30:43Z

I have been running kvstore tests on 3 different FRDM-K64F on the master branch, before and after this PR was merged. I see different intermittent failures across different boards, but I see the same failures before and after this PR.

mostly I see these tests failing, but it doesn't seem to be related at all to this PR, unless the flash is getting left in a weird state and causing the test to fail subsequent runs.

features-storage-tests-kvstore-filesystemstore_tests | Testing Edge Cases
features-storage-tests-kvstore-general_tests_phase_1 | TDB_kvstore_init

0xc0170 · 2019-11-14T09:26:15Z

Thanks @maclobdell for the report. I've just noticed failures in some other PRs and nightly as well, going to create a new issue to track this.

mbed-os PR 11531 introduced 4-byte addressing in the QSPIF block device: ARMmbed#11531 During testing it was found that this code broke on the NRF52840_DK and DISCO_F769NI. The NRF52840_DK controller seems unable to handle 4-byte addressing at all and has been disabled entirely in another code section. The DISCO_F769NI breakage was attributed to the flash chip but after more research I believe this is related to the QSPI controller, not the 4-byte addressing itself. Now that the QSPI controller has a workaround, enable 4-byte addressing again and hope it works fine this time.

* STM32F7: Unconditionally enable QSPI workarounds On the STM32769NI at least this patch is required for stable QSPI use. Enable it uncondtionally in case other boards need it too. Further discussions: ARMmbed#10049 ARMmbed#15108 STMicroelectronics/STM32CubeF7#52 STMicroelectronics/STM32CubeF7#82 * QSPIF: Attempt 4-byte addressing on Macronix chips mbed-os PR 11531 introduced 4-byte addressing in the QSPIF block device: ARMmbed#11531 During testing it was found that this code broke on the NRF52840_DK and DISCO_F769NI. The NRF52840_DK controller seems unable to handle 4-byte addressing at all and has been disabled entirely in another code section. The DISCO_F769NI breakage was attributed to the flash chip but after more research I believe this is related to the QSPI controller, not the 4-byte addressing itself. Now that the QSPI controller has a workaround, enable 4-byte addressing again and hope it works fine this time.

mbed-os PR 11531 introduced 4-byte addressing in the QSPIF block device: ARMmbed#11531 During testing it was found that this code broke on the NRF52840_DK and DISCO_F769NI. The NRF52840_DK controller seems unable to handle 4-byte addressing at all and has been disabled entirely in another code section. The DISCO_F769NI breakage was attributed to the flash chip but after more research I believe this is related to the QSPI controller, not the 4-byte addressing itself. Now that the QSPI controller has a workaround, enable 4-byte addressing again and hope it works fine this time.

ciarmcom requested review from a team September 19, 2019 21:00

ciarmcom added the needs: review label Sep 19, 2019

0xc0170 removed the request for review from a team September 20, 2019 08:28

0xc0170 reviewed Sep 20, 2019

View reviewed changes

0xc0170 removed request for a team September 26, 2019 14:24

0xc0170 requested changes Sep 26, 2019

View reviewed changes

SeppoTakalo suggested changes Sep 27, 2019

View reviewed changes

adbridge added needs: work and removed needs: review labels Sep 27, 2019

This was referenced Oct 1, 2019

Introduce qspi_inst_t type for QSPI instructions #11604

Merged

Allow for arbitrary QSPI alt sizes #11602

Merged

kyle-cypress force-pushed the pr/qspi-sfdp branch from 29f9288 to e21345d Compare October 3, 2019 23:11

SeppoTakalo approved these changes Oct 4, 2019

View reviewed changes

adbridge added needs: CI and removed needs: work labels Oct 11, 2019

adbridge added the release-version: 5.14.1 label Oct 11, 2019

Kyle Kearney added 3 commits November 12, 2019 12:26

QSPIF: Handle fast mode enable via vendor quirks

2526b9f

Use a vendor id check to only perform this enable on devices which define the second configuration register where the fast mode enable bit lives. Change _enable_fast_mode to use the standard status register reading and writing functions

General Block Device Test: Expand Thread Stack

0103e3a

The addition of trace logging during greentea tests pushes the multithreaded read-write test beyond the limits of the stack it allocates for its threads. The increase of 128 bytes was chosen by experimentation.

kyle-cypress force-pushed the pr/qspi-sfdp branch from 073285a to 0103e3a Compare November 13, 2019 00:06

jeromecoutant approved these changes Nov 13, 2019

View reviewed changes

0xc0170 added ready for merge and removed needs: CI labels Nov 13, 2019

0xc0170 merged commit 539779f into ARMmbed:master Nov 13, 2019

0xc0170 removed the ready for merge label Nov 13, 2019

0xc0170 mentioned this pull request Nov 14, 2019

Storage tests: kvstore-general_tests_phase_1 fails for K64F #11862

Closed

kyle-cypress mentioned this pull request Nov 21, 2019

Gaps in QSPIFBlockDevice SFDP Implementation #11530

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve QSPIFBlockDevice conformance to SFDP #11531

Improve QSPIFBlockDevice conformance to SFDP #11531

kyle-cypress commented Sep 19, 2019 •

edited

Loading

kyle-cypress commented Sep 19, 2019

ciarmcom commented Sep 19, 2019

0xc0170 Sep 20, 2019

kyle-cypress Sep 23, 2019

0xc0170 commented Sep 26, 2019

0xc0170 Sep 26, 2019 •

edited

Loading

0xc0170 Sep 26, 2019

kyle-cypress Sep 26, 2019 •

edited

Loading

kyle-cypress Sep 26, 2019

SeppoTakalo Sep 27, 2019

0xc0170 commented Sep 26, 2019

SeppoTakalo left a comment

kyle-cypress commented Sep 27, 2019

kyle-cypress commented Oct 1, 2019 •

edited

Loading

kyle-cypress commented Oct 3, 2019

SeppoTakalo Oct 4, 2019

kyle-cypress Oct 4, 2019

maclobdell commented Oct 8, 2019

adbridge commented Oct 11, 2019

kyle-cypress commented Nov 13, 2019 •

edited

Loading

jeromecoutant commented Nov 13, 2019

0xc0170 commented Nov 13, 2019

0xc0170 commented Nov 13, 2019 •

edited

Loading

mbed-ci commented Nov 13, 2019

0xc0170 commented Nov 13, 2019 •

edited

Loading

adbridge commented Nov 13, 2019

maclobdell commented Nov 13, 2019

maclobdell commented Nov 13, 2019

kyle-cypress commented Nov 13, 2019

kyle-cypress commented Nov 13, 2019

adbridge commented Nov 13, 2019

0xc0170 commented Nov 13, 2019

maclobdell commented Nov 13, 2019

0xc0170 commented Nov 14, 2019

Improve QSPIFBlockDevice conformance to SFDP #11531

Improve QSPIFBlockDevice conformance to SFDP #11531

Conversation

kyle-cypress commented Sep 19, 2019 • edited Loading

Description

Pull request type

Reviewers

Release Notes

kyle-cypress commented Sep 19, 2019

ciarmcom commented Sep 19, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

0xc0170 commented Sep 26, 2019

0xc0170 Sep 26, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kyle-cypress Sep 26, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

0xc0170 commented Sep 26, 2019

SeppoTakalo left a comment

Choose a reason for hiding this comment

kyle-cypress commented Sep 27, 2019

kyle-cypress commented Oct 1, 2019 • edited Loading

kyle-cypress commented Oct 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maclobdell commented Oct 8, 2019

adbridge commented Oct 11, 2019

kyle-cypress commented Nov 13, 2019 • edited Loading

jeromecoutant commented Nov 13, 2019

0xc0170 commented Nov 13, 2019

0xc0170 commented Nov 13, 2019 • edited Loading

mbed-ci commented Nov 13, 2019

0xc0170 commented Nov 13, 2019 • edited Loading

adbridge commented Nov 13, 2019

maclobdell commented Nov 13, 2019

maclobdell commented Nov 13, 2019

kyle-cypress commented Nov 13, 2019

kyle-cypress commented Nov 13, 2019

adbridge commented Nov 13, 2019

0xc0170 commented Nov 13, 2019

maclobdell commented Nov 13, 2019

0xc0170 commented Nov 14, 2019

kyle-cypress commented Sep 19, 2019 •

edited

Loading

0xc0170 Sep 26, 2019 •

edited

Loading

kyle-cypress Sep 26, 2019 •

edited

Loading

kyle-cypress commented Oct 1, 2019 •

edited

Loading

kyle-cypress commented Nov 13, 2019 •

edited

Loading

0xc0170 commented Nov 13, 2019 •

edited

Loading

0xc0170 commented Nov 13, 2019 •

edited

Loading