-
Notifications
You must be signed in to change notification settings - Fork 13.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable 128K virtual memory via external SPI SRAM #6994
Merged
Merged
Commits on Jan 19, 2020
-
Enable virtual memory 128K+ via external SPI SRAM
Provides a transparently accessible additional block of RAM of 128K to 8MB by using an external SPI SRAM. This memory is managed using the UMM memory manager and can be used by the core as if it were internal RAM (albiet much slower to read or write). The use case would be for things which are quite large but not particularly frequently used or compute intensive. For example, the SSL buffers of 16K++ are a good fit for this, as are the contents of Strings (both to avoid main heap fragmentation as well as allowing Strings of >30KB). A fully associative LRU cache is used to limit the SPI bus bottleneck, and background writeback is supported. Requires `ESP.enableVM()` call to actually add the VM subsystem. If this routine is not called, then the entire VM routines should not be linked in to user apps, so there should be no space penalty w/o it. UMM `malloc` and `new` are modified to support internal and external heap regions. By default, everything comes from the standard heap, but a call to `ESP.setExternalHeap()` before the allocation (followed by a call to `ESP.resetHeap()` will make the allocation come from external RAM. See the `virtualmem.ino` example for use. If there is no external RAM installed, the `setExternalHeap` call is a no-op. The String and BearSSL libraries have been modified to use this external RAM automatically. Theory of Operation: The Xtensa core generates a hardware exception (unrelated to C++ exceptions) when an address that's defined as invalid for load or store. The XTOS ROM routines capture the machine state and call a standard C exception handler routine (or the default one which resets the system). We hook into this exception callback and decode the EXCVADDR (the address being accessed) and use the exception PC to read out the faulting instruction. We decode that instruction and simulate it's behavior (i.e. either loading or storing some data to a register/external memory) and then return to the calling application. We use the hardware SPI interface to talk to an external SRAM/PSRAM, and implement a simple cache to minimize the amount of times we need to go out over the (slow) SPI bus. The SPI is set up in a DIO mode which uses no more pins than normal SPI, but provides for ~2X faster transfers. SIO mode is also supported. NOTE: This works fine for processor accesses, but cannot be used by any of the peripherals' DMA. For that, we'd need a real MMU. Hardware Configuration (only use 3.3V compatible SRAMs!): SPI byte-addressible SRAM/PSRAM: 23LC1024 or smaller CS -> GPIO15 SCK -> GPIO14 MOSI -> GPIO13 MISO -> GPIO12 (note these are GPIO numbers, not the Arduino Dxx pin names. Refer to your ESP8266 board schematic for the mapping of GPIO to pin.) Higher density PSRAM (ESP-PSRAM64H/etc.) should work as well, but I'm still waiting on my chips so haven't done any testing. Biggest concern is their command set and functionality in DIO mode. If DIO mode isn't supported, then a fallback to SIO is possible. This PR originated with code from @pvvx's esp8266web server at https://github.com/pvvx/esp8266web (licensed in the public domain) but doesn't resemble it much any more. Thanks, @pvvx! Keep a list of the last 8 lines in RAM (~.5KB of RAM) and use that to speed up things like memcpys and other operations where the source and destination addresses are inside VM RAM. A custom set of SPI routines is used in the VM system for speed and code size (and because the core cannot be dependent on a library).
Configuration menu - View commit details
-
Copy full SHA for 8788c76 - Browse repository at this point
Copy the full SHA 8788c76View commit details
Commits on Dec 22, 2020
-
Configuration menu - View commit details
-
Copy full SHA for f3927f1 - Browse repository at this point
Copy the full SHA f3927f1View commit details -
Configuration menu - View commit details
-
Copy full SHA for e333d50 - Browse repository at this point
Copy the full SHA e333d50View commit details
Commits on Jan 9, 2021
-
Configuration menu - View commit details
-
Copy full SHA for e1d580e - Browse repository at this point
Copy the full SHA e1d580eView commit details -
Tested and running with latest UMM IRAM heap changes
Ran sanity tests successfully on 23LC1024 128K SRAM. Adjusted defines to ensure UMM knows about our heap.
1Configuration menu - View commit details
-
Copy full SHA for 5bb31a0 - Browse repository at this point
Copy the full SHA 5bb31a0View commit details -
Unify exception handler logic between non32xfer/vm
Use same ASM macro from @mhightower83's non32xfer handler in the VM handler, avoid potential GCC optimization issues.
Configuration menu - View commit details
-
Copy full SHA for 7619858 - Browse repository at this point
Copy the full SHA 7619858View commit details
Commits on Jan 12, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 8474edf - Browse repository at this point
Copy the full SHA 8474edfView commit details
Commits on Jan 13, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 1e1bc49 - Browse repository at this point
Copy the full SHA 1e1bc49View commit details -
Configuration menu - View commit details
-
Copy full SHA for a129c01 - Browse repository at this point
Copy the full SHA a129c01View commit details
Commits on Jan 22, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 8d99fd9 - Browse repository at this point
Copy the full SHA 8d99fd9View commit details
Commits on Jan 23, 2021
-
Configuration menu - View commit details
-
Copy full SHA for d106e88 - Browse repository at this point
Copy the full SHA d106e88View commit details
Commits on Jan 25, 2021
-
Adjust UMM to only map 256K of 1M PSRAM
Because UMM manages RAM in 8 byte chunks, attempting to manage the entire 1M available space on a 1M PSRAM causes the block IDs to overflow, crashing things at some point. Limit the UMM allocation to only 256K in this case. The remaining space can manually be assigned to buffers/etc. managed by the application, not malloc()/free().
Configuration menu - View commit details
-
Copy full SHA for fc85d18 - Browse repository at this point
Copy the full SHA fc85d18View commit details
Commits on Mar 14, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 3fa9779 - Browse repository at this point
Copy the full SHA 3fa9779View commit details -
Configuration menu - View commit details
-
Copy full SHA for bd302ca - Browse repository at this point
Copy the full SHA bd302caView commit details
Commits on Mar 15, 2021
-
Configuration menu - View commit details
-
Copy full SHA for c433fa8 - Browse repository at this point
Copy the full SHA c433fa8View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.