Tracking Issue for merging to upstream (LLVM-57) #4

polarathene · 2019-07-09T06:53:56Z

I understand it will take quite some time for this backend to be merged upstream into LLVM.

This issue is for notifying anyone that subscribes to it when the merge to upstream has completed closing this issue.

DarkWiiPlayer · 2019-07-19T07:02:00Z

Is there a rough ETA or roadmap for this?

igrr · 2019-07-19T07:39:41Z

The patches are in review (see links in https://esp32.com/viewtopic.php?t=11412#p46446) and further timeline will depend on the review process. We will keep updating the patch set to address the feedback!

breckwagner · 2019-12-24T17:40:04Z

igrr · 2020-01-15T11:10:19Z

We are moving this and other issues to https://github.com/espressif/llvm-project, which will host the Xtensa patches for LLVM based on the new monorepo layout. This new repository will be updated with Xtensa patches based on LLVM 9 soon.

andreisfr · 2020-02-10T01:10:53Z

@polarathene, @DarkWiiPlayer , @breckwagner, we updated current patches (1-10), which are in LLVM review, according to changes in upstream version and reviewers requests for changes.

lexxvir · 2020-07-22T13:28:40Z

Hi @andreisfr,

Is there any plan/way to merge this fork into the upstream?

I've skimmed some pull requests on the reviews.llvm.org, it seems the process is stalled.

andreisfr · 2020-07-22T14:15:53Z

Hi @lexxvir,

Yes, we plan to continue integration process of the Xtensa backend to the upstream.

windoze · 2020-09-08T10:00:52Z

Is there any progress for this?
I'm dying to see it merging into upstream, it's really a PITA to build the whole LLVM/Clang project from the scratch in a CI pipeline.

shymega · 2021-05-19T17:40:05Z

Hey, hope you don't mind me chasing this up 😄 I've been getting ready to start work on Rust code for an ESP32, but without official LLVM and Rust support, I fear I'll have to use C. It does take ages to compile as well. I might be able to influence a different chip, but I think the ESP32 is a solid choice.

Thanks.

torpak · 2021-06-25T11:38:43Z

It seems nobody at LLVM is even answering since may. Does anyone know a reason for this?

clouds56 · 2021-06-26T15:26:19Z

@torpak according to this https://reviews.llvm.org/D64830#2615149 comment, I think they are waiting the author to rewrite a RFC since things changed a lot.
Correct me if wrong.

Hao123br · 2021-06-26T16:25:53Z

@clouds56 the author already sent another RFC but had no response. https://lists.llvm.org/pipermail/llvm-dev/2021-March/149090.html

clouds56 · 2021-06-26T16:52:27Z

@Hao123br thanks for the link, shall we also provide the link in the thread and ping aykevl & tstellar, maybe they are too busy and missing the RFC?

@yrnkrn

This patch re-introduces the fix in the commit llvm@66b0cebf7f736 by @yrnkrn > In DwarfEHPrepare, after all passes are run, RewindFunction may be a dangling > > pointer to a dead function. To make sure it's valid, doFinalization nullptrs > RewindFunction just like the constructor and so it will be found on next run. > > llvm-svn: 217737 It seems that the fix was not migrated to `DwarfEHPrepareLegacyPass`. This patch also updates `llvm/test/CodeGen/X86/dwarf-eh-prepare.ll` to include `-run-twice` to exercise the cleanup. Without this patch `llvm-lit -v llvm/test/CodeGen/X86/dwarf-eh-prepare.ll` fails with ``` -- Testing: 1 tests, 1 workers -- FAIL: LLVM :: CodeGen/X86/dwarf-eh-prepare.ll (1 of 1) ******************** TEST 'LLVM :: CodeGen/X86/dwarf-eh-prepare.ll' FAILED ******************** Script: -- : 'RUN: at line 1'; /home/arakaki/build/llvm-project/main/bin/opt -mtriple=x86_64-linux-gnu -dwarfehprepare -simplifycfg-require-and-preserve-domtree=1 -run-twice < /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll -S | /home/arakaki/build/llvm-project/main/bin/FileCheck /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll -- Exit Code: 2 Command Output (stderr): -- Referencing function in another module! call void @_Unwind_Resume(i8* %ehptr) #1 ; ModuleID = '<stdin>' void (i8*)* @_Unwind_Resume ; ModuleID = '<stdin>' in function simple_cleanup_catch LLVM ERROR: Broken function found, compilation aborted! PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace. Stack dump: 0. Program arguments: /home/arakaki/build/llvm-project/main/bin/opt -mtriple=x86_64-linux-gnu -dwarfehprepare -simplifycfg-require-and-preserve-domtree=1 -run-twice -S 1. Running pass 'Function Pass Manager' on module '<stdin>'. 2. Running pass 'Module Verifier' on function '@simple_cleanup_catch' #0 0x000056121b570a2c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Unix/Signals.inc:569:0 #1 0x000056121b56eb64 llvm::sys::RunSignalHandlers() /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Signals.cpp:97:0 #2 0x000056121b56f28e SignalHandler(int) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Unix/Signals.inc:397:0 #3 0x00007fc7e9b22980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980) #4 0x00007fc7e87d3fb7 raise /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0 #5 0x00007fc7e87d5921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0 #6 0x000056121b4e1386 llvm::raw_svector_ostream::raw_svector_ostream(llvm::SmallVectorImpl<char>&) /home/arakaki/repos/watch/llvm-project/llvm/include/llvm/Support/raw_ostream.h:674:0 #7 0x000056121b4e1386 llvm::report_fatal_error(llvm::Twine const&, bool) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/ErrorHandling.cpp:114:0 #8 0x000056121b4e1528 (/home/arakaki/build/llvm-project/main/bin/opt+0x29e3528) #9 0x000056121adfd03f llvm::raw_ostream::operator<<(llvm::StringRef) /home/arakaki/repos/watch/llvm-project/llvm/include/llvm/Support/raw_ostream.h:218:0 FileCheck error: '<stdin>' is empty. FileCheck command line: /home/arakaki/build/llvm-project/main/bin/FileCheck /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll -- ******************** ******************** Failed Tests (1): LLVM :: CodeGen/X86/dwarf-eh-prepare.ll Testing Time: 0.22s Failed: 1 ``` Reviewed By: loladiro Differential Revision: https://reviews.llvm.org/D110979 (cherry picked from commit e8806d7)

arnauorriols · 2022-04-01T14:11:38Z

FYI, just noticed patch 2 is also accepted since March 21 2021

…ified offset and its parents or children with spcified depth." This reverts commit a3b7cb0. symbol-offset.test fails under MSAN: [ 1] ; RUN: llvm-pdbutil yaml2pdb %p/Inputs/symbol-offset.yaml --pdb=%t.pdb [FAIL] llvm-pdbutil yaml2pdb <REDACTED>/llvm/test/tools/llvm-pdbutil/Inputs/symbol-offset.yaml --pdb=<REDACTED>/tmp/symbol-offset.test/symbol-offset.test.tmp.pdb ==9283==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55f975e5eb91 in __libcpp_tls_set <REDACTED>/include/c++/v1/__threading_support:428:12 #1 0x55f975e5eb91 in set_pointer <REDACTED>/include/c++/v1/thread:196:5 #2 0x55f975e5eb91 in void* std::__msan::__thread_proxy<std::__msan::tuple<std::__msan::unique_ptr<std::__msan::__thread_struct, std::__msan::default_delete<std::__msan::__thread_struct> >, llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()::operator()() const::'lambda'()> >(void*) <REDACTED>/include/c++/v1/thread:285:27 #3 0x7f74a1e55b54 in start_thread (<REDACTED>/libpthread.so.0+0xbb54) (BuildId: 64752de50ebd1a108f4b3f8d0d7e1a13) #4 0x7f74a1dc9f7e in clone (<REDACTED>/libc.so.6+0x13cf7e) (BuildId: 7cfed7708e5ab7fcb286b373de21ee76)

saeidakbari · 2022-07-09T08:20:06Z

Is any new news on reviewing process?

flaminisx · 2022-09-21T11:33:06Z

just noticed that patches 4, 5, 6, 8 and 10 are also accepted

igrr · 2022-09-26T19:47:05Z

For future reference, the new RFC is at https://discourse.llvm.org/t/rfc-request-for-upstream-tensilica-xtensa-esp32-backend/65355.

SergioGasquez · 2022-12-13T19:15:38Z

shish · 2022-12-31T11:32:52Z

Looks like these are all merged into the main branch now, if I'm reading it right? Merry Christmas all :D

aq1018 · 2023-03-17T23:24:48Z

Should this be closed? LLVM has has accepted all patches according to this article - https://www.phoronix.com/news/LLVM-Xtensa-Backend

Skgland · 2023-03-18T00:28:28Z

Unless esp-rs/rust#89 (comment) is outdated, there are still some parts missing.

mortenbekditlevsen · 2024-06-29T16:33:51Z

@JetForMe Will you do this through the Swift Evolution Forum?

@andreisfr In case you're wondering about the spike in interest for these patches, the Swift programming language quite recently added an 'embedded' subset of the language.
This is why myself and many others are now following this work with much interest. 😊

MihaelIsaev · 2024-06-29T16:40:11Z

I'm subscribed to all the activity in this repository because waiting for Xtensa support to start using ESP32 with Embedded Swift. Would love to support a thread about that on Swift Forums 😊 there are already at least one thread and on reddit as well

teburd · 2024-06-29T18:33:43Z

Are the hifi patches also upstreamed?

gerekon · 2024-06-29T18:43:16Z

@teburd

Are the hifi patches also upstreamed?

Do you mean ones from this PR?
Not yet I think. They need Boolean and Floating point features as far as I see.
We will do this when Boolean and Float are upstreamed.

bugadani · 2024-06-29T18:45:38Z

Considering how long it takes to upstream few-hundered-LOC PRs, I am somewhat worried about throwing 80k+ LOC patches in 😅

gerekon · 2024-06-29T18:53:12Z

Considering how long it takes to upstream few-hundered-LOC PRs, I am somewhat worried about throwing 80k+ LOC patches in 😅

I expect that after the next 10-15 patches we will start doing this in parallel. Maybe we can try to keep 2-3 active patches in upstream review right now and see if it will speed up the process.

@andreisfr what do you think?

andreisfr · 2024-07-01T21:57:06Z

@gerekon , yes, I think we need to upstream about 10-15 patches and probably we will be able to create parallel PRs

ProfFan · 2024-07-31T01:15:24Z

Is there an update or target timeline?

Skgland · 2024-07-31T07:51:58Z

Have you seen the estimate approximately a month ago #4 (comment)

andreisfr · 2024-07-31T09:11:22Z

Current status. The last PR actually contains 2 base patches.

1. [Xtensa] Initial codegen support from IR llvm/llvm-project#78548. (Split from [D145658: [Xtensa] Initial support of the ALU operations.]
2. https://reviews.llvm.org/D145658))
3. [Xtensa] Implement base CallConvention. llvm/llvm-project#83280.
4. [PATCH] [Xtensa] Implement FrameLowering methods and stack operation lowering. llvm/llvm-project#92960
5. [Xtensa] Lower GlobalAddress/BlockAddress/JumpTable llvm/llvm-project#95256
6. [Xtensa] Implement lowering SELECT_CC, SETCC. llvm/llvm-project#97017
7. [Xtensa] Implement lowering Mul/Div/Shift operations. llvm/llvm-project#99981

johnnyasantoss · 2024-08-22T16:35:11Z

Apparently llvm#99981 just got merged

JetForMe · 2024-08-22T21:49:18Z

IIUC everything is merged? I wonder how long before it makes it into a Swift toolchain.

Skgland · 2024-08-22T22:09:47Z

IIUC everything is merged?

No, #4 (comment) only includes the seven patches already submitted to upstream. Based on #4 (comment) that makes it 7 out of 10-15 patches that need to be submitted in series after which more patches will follow in parallel.

JetForMe · 2024-08-22T23:54:20Z

This comment listed the following items remain. I was unable to easily correlate them to PRs in later comments.

@SergioGasquez There are 13 patches left:

Implement lowering SELECT_CC, SETCC - In Progress

Add support of the Xtensa function calls

Lower stack operations.

Support for a variety of additional LLVM IR constructs.

Lower SHIFT PARTS and shift operations.

Implement load pseudo operations and patterns.

Support for variable arguments

Implement lowering BR_JT operation

Support for address intrinsics.

Add basic support for inline asm constraints

Implement volatile load/store.

Implement branch analysis

Implement support for the BranchRelaxation

andreisfr · 2024-08-26T11:37:34Z

Current status.

1. [Xtensa] Initial codegen support from IR llvm/llvm-project#78548. (Split from [D145658: [Xtensa] Initial support of the ALU operations.]
2. https://reviews.llvm.org/D145658))
3. [Xtensa] Implement base CallConvention. llvm/llvm-project#83280.
4. [PATCH] [Xtensa] Implement FrameLowering methods and stack operation lowering. llvm/llvm-project#92960
5. [Xtensa] Lower GlobalAddress/BlockAddress/JumpTable llvm/llvm-project#95256
6. [Xtensa] Implement lowering SELECT_CC, SETCC. llvm/llvm-project#97017
7. [Xtensa] Implement lowering Mul/Div/Shift operations. llvm/llvm-project#99981
8. [Xtensa] Implement sextload i8 llvm/llvm-project#106053

TurkeyMan · 2024-08-28T07:44:27Z

🎉

What happens next now that these all seem to be done?

bugadani · 2024-08-28T07:51:41Z

Now the next patch will be submitted, then the next, and the next... :)

TurkeyMan · 2024-08-28T08:06:19Z

Lol, but as someone following closely and with much excitement... it seems to be the end of the list of outstanding patches that I was aware of. Wasn't clear if there's another whole bunch, or if we're approaching branch synchronisation?

bugadani · 2024-08-28T08:17:08Z

Considering Andrei has estimated 10-15 patches two months ago to be submitted one by one, and since then 2 got merged, I wouldn't hold my breath. And as I understand even that isn't the end of this story.

andreisfr · 2024-08-28T08:21:47Z

The llvm#106053 PR is merged.
We preparing next PR now.

Skgland · 2024-08-28T08:32:48Z

Maybe the Current status Comment could include an additional bullet point at the and for further patches to avoid this confusion.

andreisfr · 2024-08-28T08:35:14Z

Maybe the Current status Comment could include an additional bullet point at the and for further patches to avoid this confusion.

Ok, I will add this information in next update

andreisfr · 2024-09-11T10:05:53Z

Hi all!

I prepared a plan which describes nearest steps of upstreaming Xtensa backend. The goal is to implement most of functionality to be able generate Rust code and compile ESP-IDF examples.

We have 6 prepared patches with base functionality not upstreamed. This patches are not very large, so I think optimistic estimation is to finish this step till middle of October.

[Xtensa] Lowering FRAMEADDR/RETURNADDR operations. llvm/llvm-project#107363
[Xtensa] Add basic support for inline asm constraints. llvm/llvm-project#108986
[Xtensa] Implement volatile load/store. llvm/llvm-project#110292
[Xtensa] Implement branch analysis. llvm/llvm-project#110959
Implement support for the BranchRelaxation
Support for variable arguments

Implement other Xtensa features like Window, Boolean, Float, etc. At this step we didn't prepare exactly patch list, because it depends on previous step, it will be about 8-12 patches. Also in optimistic case will take another 2-3 month and probably be finished in December. At this stage we probably can make some parallel PRs.

Currently we have feature list to implement:

Implement code density feature
Implement Windowed feature
Implement FP feature
Implement Boolean feature
Implement Mac16 feature and operations.
Implement Debug, DFPAccel, S32C1I, THREADPTR, Extended L32R, ATOMCTL, MEMCTL features.
Implement Exception, HighPriInterrupts, Coprocessor, Interrupt, RelocatableVector, TimerInt, PRID, RegionProtection and
MiscSR features. Implement instructions for Exception, Interrupt and RegionProtection features.

We also have a patches for this features, but number of patches and order maybe changed. I will prepare more precise patch set when first step will be finished.

Implement Xtensa targets in Clang esp32,esp8266, etc" and add support for multilib and linker. It is about 8-12 patches. Some of these patches will be complex, so in optimistic case will take another 2-3 month and will be finished in Febraury/March. At this stage we probably can make some parallel PRs.
After that we can probably build esp-idf and Rust examples.
Implement minor features, like hwloop, etc.

andreisfr · 2024-09-17T15:06:10Z

The Xtensa backend upstreaming status update:

[Xtensa] Lowering FRAMEADDR/RETURNADDR operations. llvm/llvm-project#107363
[Xtensa] Add basic support for inline asm constraints. llvm/llvm-project#108986
Implement volatile load/store.
Implement branch analysis
Implement support for the BranchRelaxation
Support for variable arguments

andreisfr · 2024-10-03T06:18:22Z

The Xtensa backend upstreaming status update:

andreisfr · 2024-11-07T08:31:40Z

The Xtensa backend upstreaming status update. Currently we preparing PR with the last patch from step 1 #4 (comment) . In a few days I will an update with a list of the next patch set for upstream.

igrr transferred this issue from espressif/llvm-xtensa Jan 15, 2020

espressif-bot changed the title ~~Tracking Issue for merging to upstream~~ Tracking Issue for merging to upstream (LLVM-57) Jan 29, 2020

igrr mentioned this issue Jul 13, 2021

Rework of the lua source hierachy to support a unified apporach to ESP8266 and ESP32 nodemcu/nodemcu-firmware#1661

Open

MabezDev mentioned this issue Apr 26, 2022

Merge into main branch rust-lang/rust? esp-rs/rust#89

Open

MabezDev mentioned this issue Aug 9, 2022

How should we expose atomic load/store on targets that don't support full atomics rust-lang/rust#99668

Open

MabezDev mentioned this issue Oct 31, 2022

Book update esp-rs/book#37

Merged

weitzj mentioned this issue Mar 12, 2023

Xtensa Support ziglang/zig#5467

Open

beetrees mentioned this issue Aug 2, 2024

Configure which platforms have f16 and f128 enabled by default rust-lang/compiler-builtins#652

Merged

Reavershark mentioned this issue Aug 7, 2024

Add llvm-xtensa and ldc-xtensa images dlang-dockerized/packaging#19

Merged

kassane mentioned this issue Aug 10, 2024

Add support for Xtensa embedded llvm target c3lang/c3c#1297

Open

MabezDev mentioned this issue Oct 21, 2024

rustc LLVM-18 float extension crash (LLVM-404) #103

Closed

3 tasks

Tracking Issue for merging to upstream (LLVM-57) #4

Tracking Issue for merging to upstream (LLVM-57) #4

Comments

polarathene commented Jul 9, 2019

DarkWiiPlayer commented Jul 19, 2019

igrr commented Jul 19, 2019

breckwagner commented Dec 24, 2019 • edited by igrr Loading

igrr commented Jan 15, 2020

andreisfr commented Feb 10, 2020

lexxvir commented Jul 22, 2020

andreisfr commented Jul 22, 2020

windoze commented Sep 8, 2020

shymega commented May 19, 2021

torpak commented Jun 25, 2021

clouds56 commented Jun 26, 2021

Hao123br commented Jun 26, 2021 • edited Loading

clouds56 commented Jun 26, 2021

arnauorriols commented Apr 1, 2022

saeidakbari commented Jul 9, 2022

flaminisx commented Sep 21, 2022

igrr commented Sep 26, 2022

SergioGasquez commented Dec 13, 2022

shish commented Dec 31, 2022

aq1018 commented Mar 17, 2023

Skgland commented Mar 18, 2023

mortenbekditlevsen commented Jun 29, 2024 • edited Loading

MihaelIsaev commented Jun 29, 2024

teburd commented Jun 29, 2024

gerekon commented Jun 29, 2024 • edited Loading

bugadani commented Jun 29, 2024

gerekon commented Jun 29, 2024 • edited Loading

andreisfr commented Jul 1, 2024

ProfFan commented Jul 31, 2024

Skgland commented Jul 31, 2024

andreisfr commented Jul 31, 2024 • edited by igrr Loading

johnnyasantoss commented Aug 22, 2024

JetForMe commented Aug 22, 2024

Skgland commented Aug 22, 2024

JetForMe commented Aug 22, 2024 • edited Loading

andreisfr commented Aug 26, 2024 • edited Loading

TurkeyMan commented Aug 28, 2024

bugadani commented Aug 28, 2024

TurkeyMan commented Aug 28, 2024

bugadani commented Aug 28, 2024 • edited Loading

andreisfr commented Aug 28, 2024

Skgland commented Aug 28, 2024 • edited Loading

andreisfr commented Aug 28, 2024

andreisfr commented Sep 11, 2024 • edited by tom-borcin Loading

andreisfr commented Sep 17, 2024 • edited by tom-borcin Loading

andreisfr commented Oct 3, 2024 • edited by tom-borcin Loading

andreisfr commented Nov 7, 2024 • edited Loading

breckwagner commented Dec 24, 2019 •

edited by igrr

Loading

Hao123br commented Jun 26, 2021 •

edited

Loading

mortenbekditlevsen commented Jun 29, 2024 •

edited

Loading

gerekon commented Jun 29, 2024 •

edited

Loading

gerekon commented Jun 29, 2024 •

edited

Loading

andreisfr commented Jul 31, 2024 •

edited by igrr

Loading

JetForMe commented Aug 22, 2024 •

edited

Loading

andreisfr commented Aug 26, 2024 •

edited

Loading

bugadani commented Aug 28, 2024 •

edited

Loading

Skgland commented Aug 28, 2024 •

edited

Loading

andreisfr commented Sep 11, 2024 •

edited by tom-borcin

Loading

andreisfr commented Sep 17, 2024 •

edited by tom-borcin

Loading

andreisfr commented Oct 3, 2024 •

edited by tom-borcin

Loading

andreisfr commented Nov 7, 2024 •

edited

Loading