[LLVM] upgrade to LLVM 11 #37842

vchuravy · 2020-10-01T16:19:16Z

Based on LLVM 11-rc5. Still some patch maintenance to do.

TODO:

D75072

vchuravy · 2020-10-09T22:26:56Z

src/clangsa/GCChecker.cpp

@@ -1306,6 +1306,8 @@ bool GCChecker::evalCall(const CallExpr *CE,
  // (globals should not be invalidated, etc), hence the use of evalCall.
 #if LLVM_VERSION_MAJOR >= 9
  const CallExpr *CE = dyn_cast<CallExpr>(Call.getOriginExpr());
+  if (!CE)


As an example I observed:

(gdb) p Call.getOriginExpr()->dump() CXXConstructExpr 0x555557d1e910 'class llvm::APInt' 'void (void)' $3 = void

@Keno does this change look ok?

vchuravy · 2020-10-11T05:09:02Z

@Keno & @vtjnash The x86 Linux failure is peculiar:

What I observed is that we want to perfom a ccall to jl_alloc_array_1d, but instead jump into the middle off jl_alloc_array_3d.
Dumping the LLVM module IR shows that we embed the correct pointer. Running with --debug=dyld and dissasembling the object file before and after relocations. Shows that we changed from the correct address to a wrongly offset address.

As an example: The emitted e815028ef7 changes to e8b1d9590c. ntoh(0x15028ef7) == 0xf78e0215 which is the location we want and IIUC opcode e8 correctly 0xeb342994 + ntoh(0xb1d9590c) == 0xf78e0345, which is consistent with us jumping past jl_alloc_array_1d and landing in the middle of jl_alloc_array_3d.

Looking at RuntimeDyld: (different binary so address are different)

=> 0xeae8698f <+271>:   call   0xf7724335 <jl_alloc_array_3d+106> 
b RuntimeDyldELF.cpp:935 if RE.Offset == 272

resolveRelocation -> Addend: 0xf7724331

(rr) p/x &(RE.Addend)
(rr) w *0x5916a704
(rr) rc
> addRelocationForSymbol
> RECopy.Addend += SymInfo.getOffset();

b processRelocationRef
(rr) rc
in processRelocationRef: L1210

Value.SymbolName == ""
Value.Addend == 0
Offset == 272

in processRelocationRef: L1772
Value.Addend == 0xf7724205


(rr) disas jl_alloc_array_1d                                                                   
Dump of assembler code for function jl_alloc_array_1d:
   0xf7724209 <+0>:     endbr32 
   0xf772420d <+4>:     push   %ebp
   0xf772420e <+5>:     mov    %esp,%ebp

SymbolName will be the empty string
- As an absolute symbol it can have 0x0, but https://github.com/llvm/llvm-project/blame/176249bd6732a8044d457092ed932768724a6f06/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp#L1199 fixes that to "".
We will call processSimpleRelocation https://github.com/llvm/llvm-project/blame/176249bd6732a8044d457092ed932768724a6f06/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp#L990 and that only checks if SymbolName is 0x0.
addRelocationForSymbol https://github.com/llvm/llvm-project/blob/176249bd6732a8044d457092ed932768724a6f06/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp#L936 comes in and will fudge the address.

The peculiar thing is that this behavior is positively old. @Keno you touched some of this code so maybe you have a in insight what the correct behaviour should be? Right now it is unclear to me what changed between LLVM 10 and LLVM 11.

Checking in https://github.com/llvm/llvm-project/blob/176249bd6732a8044d457092ed932768724a6f06/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp#L930 for SymbolName.empty() let's me compile Julia, but makes me wonder why the empty symbol is in the GlobalSymbolTable.

-- edit:
Indeed we are entering an empty string here: https://github.com/llvm/llvm-project/blame/176249bd6732a8044d457092ed932768724a6f06/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp#L343

The command line parser used to be tolerant of this oddity, since the first argument is supposed to use different escaping rules than the other arguments. But now we are using the same rules for all arguments, so it no longer worked.

vchuravy · 2020-10-12T21:36:54Z

LLVM 11 final landed this morning, absent any CI issue this is ready to land.

vchuravy · 2020-10-12T21:38:57Z

@nanosoldier runbenchmarks(ALL, vs = ":master")

vchuravy · 2020-10-12T21:39:18Z

@nanosoldier runtests(ALL, vs = ":master")

nanosoldier · 2020-10-12T22:13:33Z

Something went wrong when running your job:

NanosoldierError: failed to run benchmarks against primary commit: failed process: Process(`sudo cset shield -c 1,2,3,4 -k on`, ProcessExited(2)) [2]

Logs and partial data can be found here
cc @christopher-dG

christopher-dG · 2020-10-12T23:22:36Z

Oops, forgot to restart the server after I merged my fix.

@nanosoldier runbenchmarks(ALL, vs = ":master")

christopher-dG · 2020-10-12T23:23:37Z

@nanosoldier runbenchmarks(ALL, vs = ":master")

oops

edit: Oh wait, I'm not permitted to trigger this since I'm not in JuliaLang am I?

vchuravy · 2020-10-12T23:43:10Z

@nanosoldier runbenchmarks(ALL, vs = ":master")

nanosoldier · 2020-10-13T02:18:25Z

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

nanosoldier · 2020-10-13T08:17:38Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @christopher-dG

KristofferC · 2020-10-13T08:28:08Z

Looks like the commit doesn't exist.

maleadt · 2020-10-13T11:13:45Z

Probably https://github.com/JuliaCI/BaseBenchmarkReports/blob/21d444a950a052db8f220a3050eac6c998c4cbfc/benchmark/by_hash/3525155_vs_f9ee161/report.md -- there was a merge which confused Nanosoldier.jl.

KristofferC · 2020-10-13T11:22:51Z


["misc", "afoldl", "Int"] | 13.18 (5%) ❌ | 1.00 (1%)

["array", "index", "(\"sumcartesian_view\", \"BitMatrix\")"] | 2.18 (50%) ❌ | 1.00 (1%)

might be interesting

vchuravy force-pushed the vc/llvm11 branch from e85222f to da42e3d Compare October 3, 2020 01:11

vchuravy mentioned this pull request Oct 7, 2020

latency regressions #37915

Closed

vchuravy force-pushed the vc/llvm11 branch from 95f8ea6 to 1af0a47 Compare October 9, 2020 17:45

vchuravy marked this pull request as ready for review October 9, 2020 22:25

vchuravy commented Oct 9, 2020

View reviewed changes

vchuravy and others added 8 commits October 12, 2020 17:35

[LLVM] upgrade to LLVM 11

6fb2bcb

add patch for empty debuginfo arange

62dba5a

test: fix deprecation in stack_overflow test

475b0f1

fix quoting in julia.bat

a8019ee

The command line parser used to be tolerant of this oddity, since the first argument is supposed to use different escaping rules than the other arguments. But now we are using the same rules for all arguments, so it no longer worked.

forward port D76072

18f0cde

[LLVM] prerelease are now hosted on Github

2e44691

[gcanalyzer] skip non-CallExpr from CallEvent

4f5606f

add patch for rtlyd issue in linux32

9563180

vchuravy force-pushed the vc/llvm11 branch from c036040 to 9563180 Compare October 12, 2020 21:36

vchuravy added external dependencies Involves LLVM, OpenBLAS, or other linked libraries merge me PR is reviewed. Merge when all tests are passing labels Oct 12, 2020

vchuravy merged commit 21e7486 into master Oct 13, 2020

vchuravy deleted the vc/llvm11 branch October 13, 2020 02:27

kimikage mentioned this pull request Oct 18, 2020

Performance regression in broadcasting with CartesianIndices on v1.6.0-DEV #38086

Closed

DilumAluthge removed the merge me PR is reviewed. Merge when all tests are passing label Jan 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[LLVM] upgrade to LLVM 11 #37842

[LLVM] upgrade to LLVM 11 #37842

vchuravy commented Oct 1, 2020 •

edited

Loading

vchuravy Oct 9, 2020

vchuravy commented Oct 11, 2020 •

edited

Loading

vchuravy commented Oct 12, 2020 •

edited

Loading

vchuravy commented Oct 12, 2020

vchuravy commented Oct 12, 2020

nanosoldier commented Oct 12, 2020

christopher-dG commented Oct 12, 2020

christopher-dG commented Oct 12, 2020 •

edited

Loading

vchuravy commented Oct 12, 2020

nanosoldier commented Oct 13, 2020

nanosoldier commented Oct 13, 2020

KristofferC commented Oct 13, 2020

maleadt commented Oct 13, 2020

KristofferC commented Oct 13, 2020

[LLVM] upgrade to LLVM 11 #37842

[LLVM] upgrade to LLVM 11 #37842

Conversation

vchuravy commented Oct 1, 2020 • edited Loading

vchuravy Oct 9, 2020

Choose a reason for hiding this comment

vchuravy commented Oct 11, 2020 • edited Loading

vchuravy commented Oct 12, 2020 • edited Loading

vchuravy commented Oct 12, 2020

vchuravy commented Oct 12, 2020

nanosoldier commented Oct 12, 2020

christopher-dG commented Oct 12, 2020

christopher-dG commented Oct 12, 2020 • edited Loading

vchuravy commented Oct 12, 2020

nanosoldier commented Oct 13, 2020

nanosoldier commented Oct 13, 2020

KristofferC commented Oct 13, 2020

maleadt commented Oct 13, 2020

KristofferC commented Oct 13, 2020

vchuravy commented Oct 1, 2020 •

edited

Loading

vchuravy commented Oct 11, 2020 •

edited

Loading

vchuravy commented Oct 12, 2020 •

edited

Loading

christopher-dG commented Oct 12, 2020 •

edited

Loading