[obj2yaml] - Dump allocatable SHT_STRTAB, SHT_SYMTAB and SHT_DYNSYM s…

…ections.

Sometimes we need to dump an object and build it again from a YAML
description produced. The problem is that obj2yaml does not dump some
of sections, like string tables and symbol tables.

Because of that yaml2obj implicitly creates them and sections created
are not placed at their original locations. They are added to the end of a section list.
That makes a preparing test cases task harder than it can be.

This patch teaches obj2yaml to dump parts of allocatable SHT_STRTAB, SHT_SYMTAB
and SHT_DYNSYM sections to print placeholders for them.
This also allows to preserve usefull parameters, like virtual address.

Differential revision: https://reviews.llvm.org/D74955

llvm/test/tools/obj2yaml/duplicate-symbol-and-section-names.test

@@ -125,13 +125,7 @@ Symbols:
 # RUN: yaml2obj --docnum=3 %s -o %t3
 # RUN: obj2yaml %t3 | FileCheck %s --check-prefix=CASE3
 
-# CASE3:      --- !ELF
-# CASE3-NEXT: FileHeader:
-# CASE3-NEXT:   Class:   ELFCLASS64
-# CASE3-NEXT:   Data:    ELFDATA2LSB
-# CASE3-NEXT:   Type:    ET_DYN
-# CASE3-NEXT:   Machine: EM_X86_64
-# CASE3-NEXT: Symbols:
+# CASE3:      Symbols:
 # CASE3-NEXT:   - Name:    foo
 # CASE3-NEXT:     Binding: STB_GLOBAL
 # CASE3-NEXT: DynamicSymbols:

llvm/test/tools/obj2yaml/implicit-sections-order.yaml

@@ -0,0 +1,163 @@
+## Check that obj2yaml dumps SHT_STRTAB/SHT_SYMTAB/SHT_DYNSYM sections
+## when they are allocatable.
+
+## In the following test we check the normal case: when .dynsym (SHT_DYNSYM)
+## and .dynstr (SHT_STRTAB) are allocatable sections and .symtab (SHT_SYMTAB),
+## .strtab (SHT_STRTAB) and .shstrtab (SHT_STRTAB) are not.
+## Check we explicitly declare allocatable sections.
+
+# RUN: yaml2obj %s -o %t1.so -D FLAG1=SHF_ALLOC -D FLAG2=""
+# RUN: llvm-readelf -S %t1.so | FileCheck %s --check-prefixes=RE,RE-1
+# RUN: obj2yaml %t1.so | FileCheck %s --check-prefix=OUTPUT
+
+## Check the information about sections using an independent tool.
+
+# RE:        Section Headers:
+# RE-NEXT:   [Nr] Name      Type     Address          Off    Size   ES Flg Lk Inf Al
+# RE-NEXT:   [ 0]           NULL     0000000000000000 000000 000000 00     0   0  0
+# RE-NEXT:   [ 1] .foo.1    PROGBITS 0000000000000000 000040 000000 00     0   0  0
+# RE-1-NEXT: [ 2] .dynsym   DYNSYM   0000000000001000 000040 000030 18   A 4   2  0
+# RE-2-NEXT: [ 2] .dynsym   DYNSYM   0000000000001000 000040 000030 18     4   2  0
+# RE-NEXT:   [ 3] .foo.2    PROGBITS 0000000000000000 000070 000000 00     0   0  0
+# RE-1-NEXT: [ 4] .dynstr   STRTAB   0000000000002000 000070 000005 00   A 0   0  0
+# RE-2-NEXT: [ 4] .dynstr   STRTAB   0000000000002000 000070 000005 00     0   0  0
+# RE-NEXT:   [ 5] .foo.3    PROGBITS 0000000000000000 000075 000000 00     0   0  0
+# RE-1-NEXT: [ 6] .symtab   SYMTAB   0000000000003000 000075 000030 18     8   2  0
+# RE-2-NEXT: [ 6] .symtab   SYMTAB   0000000000003000 000075 000030 18   A 8   2  0
+# RE-NEXT:   [ 7] .foo.4    PROGBITS 0000000000000000 0000a5 000000 00     0   0  0
+# RE-1-NEXT: [ 8] .strtab   STRTAB   0000000000004000 0000a5 000005 00     0   0  0
+# RE-2-NEXT: [ 8] .strtab   STRTAB   0000000000004000 0000a5 000005 00   A 0   0  0
+# RE-NEXT:   [ 9] .foo.5    PROGBITS 0000000000000000 0000aa 000000 00     0   0  0
+# RE-1-NEXT: [10] .shstrtab STRTAB   0000000000005000 0000aa 000055 00     0   0  0
+# RE-2-NEXT: [10] .shstrtab STRTAB   0000000000005000 0000aa 000055 00   A 0   0  0
+# RE-NEXT:   [11] .foo.6    PROGBITS 0000000000000000 0000ff 000000 00     0   0  0
+
+# OUTPUT:      --- !ELF
+# OUTPUT-NEXT: FileHeader:
+# OUTPUT-NEXT:   Class:   ELFCLASS64
+# OUTPUT-NEXT:   Data:    ELFDATA2LSB
+# OUTPUT-NEXT:   Type:    ET_DYN
+# OUTPUT-NEXT:   Machine: EM_X86_64
+# OUTPUT-NEXT: Sections:
+# OUTPUT-NEXT:   - Name: .foo.1
+# OUTPUT-NEXT:     Type: SHT_PROGBITS
+# OUTPUT-NEXT:   - Name:    .dynsym
+# OUTPUT-NEXT:     Type:    SHT_DYNSYM
+# OUTPUT-NEXT:     Flags:   [ SHF_ALLOC ]
+# OUTPUT-NEXT:     Address: 0x0000000000001000
+# OUTPUT-NEXT:     Link:    .dynstr
+# OUTPUT-NEXT:     EntSize: 0x0000000000000018
+# OUTPUT-NEXT:   - Name: .foo.2
+# OUTPUT-NEXT:     Type: SHT_PROGBITS
+# OUTPUT-NEXT:   - Name:    .dynstr
+# OUTPUT-NEXT:     Type:    SHT_STRTAB
+# OUTPUT-NEXT:     Flags:   [ SHF_ALLOC ]
+# OUTPUT-NEXT:     Address: 0x0000000000002000
+# OUTPUT-NEXT:   - Name: .foo.3
+# OUTPUT-NEXT:     Type: SHT_PROGBITS
+# OUTPUT-NEXT:   - Name: .foo.4
+# OUTPUT-NEXT:     Type: SHT_PROGBITS
+# OUTPUT-NEXT:   - Name: .foo.5
+# OUTPUT-NEXT:     Type: SHT_PROGBITS
+# OUTPUT-NEXT:   - Name: .foo.6
+# OUTPUT-NEXT:     Type: SHT_PROGBITS
+# OUTPUT-NEXT: Symbols:
+# OUTPUT-NEXT:   - Name: foo
+# OUTPUT-NEXT: DynamicSymbols:
+# OUTPUT-NEXT:   - Name: bar
+# OUTPUT-NEXT: ...
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_DYN
+  Machine: EM_X86_64
+Sections:
+  - Name: .foo.1
+    Type: SHT_PROGBITS
+  - Name:    .dynsym
+    Type:    SHT_DYNSYM
+    Address: 0x1000
+    Flags: [ [[FLAG1]] ]
+  - Name: .foo.2
+    Type: SHT_PROGBITS
+  - Name:    .dynstr
+    Type:    SHT_STRTAB
+    Address: 0x2000
+    Flags: [ [[FLAG1]] ]
+  - Name: .foo.3
+    Type: SHT_PROGBITS
+  - Name:    .symtab
+    Type:    SHT_SYMTAB
+    Address: 0x3000
+    Flags: [ [[FLAG2]] ]
+  - Name: .foo.4
+    Type: SHT_PROGBITS
+  - Name:    .strtab
+    Type:    SHT_STRTAB
+    Address: 0x4000
+    Flags: [ [[FLAG2]] ]
+  - Name: .foo.5
+    Type: SHT_PROGBITS
+  - Name:    .shstrtab
+    Type:    SHT_STRTAB
+    Address: 0x5000
+    Flags: [ [[FLAG2]] ]
+  - Name: .foo.6
+    Type: SHT_PROGBITS
+Symbols:
+  - Name: foo
+DynamicSymbols:
+  - Name: bar
+
+## Now test the abnormal case: when .symtab (SHT_SYMTAB),
+## .strtab (SHT_STRTAB) and .shstrtab (SHT_STRTAB) are
+## allocatable sections, but .dynsym (SHT_DYNSYM) and
+## .dynstr (SHT_STRTAB) are not.
+## Check that only allocatable versions are explicitly declared.
+
+# RUN: yaml2obj %s -o %t2.so -D FLAG1="" -D FLAG2=SHF_ALLOC
+# RUN: llvm-readelf -S %t2.so | FileCheck %s --check-prefixes=RE,RE-2
+# RUN: obj2yaml %t2.so | FileCheck %s --check-prefix=OUTPUT2
+
+## Check we explicitly declare only allocatable
+## SHT_STRTAB/SHT_SYMTAB/SHT_DYNSYM sections.
+# OUTPUT2:      --- !ELF
+# OUTPUT2-NEXT: FileHeader:
+# OUTPUT2-NEXT:   Class:   ELFCLASS64
+# OUTPUT2-NEXT:   Data:    ELFDATA2LSB
+# OUTPUT2-NEXT:   Type:    ET_DYN
+# OUTPUT2-NEXT:   Machine: EM_X86_64
+# OUTPUT2-NEXT: Sections:
+# OUTPUT2-NEXT:   - Name: .foo.1
+# OUTPUT2-NEXT:     Type: SHT_PROGBITS
+# OUTPUT2-NEXT:   - Name: .foo.2
+# OUTPUT2-NEXT:     Type: SHT_PROGBITS
+# OUTPUT2-NEXT:   - Name: .foo.3
+# OUTPUT2-NEXT:     Type: SHT_PROGBITS
+# OUTPUT2-NEXT:   - Name:    .symtab
+# OUTPUT2-NEXT:     Type:    SHT_SYMTAB
+# OUTPUT2-NEXT:     Flags:   [ SHF_ALLOC ]
+# OUTPUT2-NEXT:     Address: 0x0000000000003000
+# OUTPUT2-NEXT:     Link:    .strtab
+# OUTPUT2-NEXT:     EntSize: 0x0000000000000018
+# OUTPUT2-NEXT:   - Name: .foo.4
+# OUTPUT2-NEXT:     Type: SHT_PROGBITS
+# OUTPUT2-NEXT:   - Name:    .strtab
+# OUTPUT2-NEXT:     Type:    SHT_STRTAB
+# OUTPUT2-NEXT:     Flags:   [ SHF_ALLOC ]
+# OUTPUT2-NEXT:     Address: 0x0000000000004000
+# OUTPUT2-NEXT:   - Name: .foo.5
+# OUTPUT2-NEXT:     Type: SHT_PROGBITS
+# OUTPUT2-NEXT:   - Name:    .shstrtab
+# OUTPUT2-NEXT:     Type:    SHT_STRTAB
+# OUTPUT2-NEXT:     Flags:   [ SHF_ALLOC ]
+# OUTPUT2-NEXT:     Address: 0x0000000000005000
+# OUTPUT2-NEXT:   - Name:    .foo.6
+# OUTPUT2-NEXT:     Type:    SHT_PROGBITS
+# OUTPUT2-NEXT: Symbols:
+# OUTPUT2-NEXT:   - Name: foo
+# OUTPUT2-NEXT: DynamicSymbols:
+# OUTPUT2-NEXT:   - Name: bar
+# OUTPUT2-NEXT: ...

llvm/test/tools/obj2yaml/versym-section.yaml

@@ -19,7 +19,8 @@
 # CHECK-NEXT:     AddressAlign:    0x0000000000000002
 # CHECK-NEXT:     EntSize:         0x0000000000000002
 # CHECK-NEXT:     Entries:         [ 0, 3, 4 ]
-# CHECK-NEXT: DynamicSymbols:
+# CHECK-NEXT:   - Name:
+# CHECK:      DynamicSymbols:
 # CHECK-NEXT:   - Name:    f1
 # CHECK-NEXT:     Binding: STB_GLOBAL
 # CHECK-NEXT:   - Name:    f2

llvm/tools/obj2yaml/elf2yaml.cpp

@@ -254,9 +254,22 @@ ELFDumper<ELFT>::dumpSections() {
     }
     case ELF::SHT_STRTAB:
     case ELF::SHT_SYMTAB:
-    case ELF::SHT_DYNSYM:
-      // Do not dump these sections.
+    case ELF::SHT_DYNSYM: {
+      // The contents of these sections are described by other parts of the YAML
+      // file. We still dump them so that their positions in the section header
+      // table are correctly recorded. We only dump allocatable section because
+      // their positions and addresses are important, e.g. for creating program
+      // headers. Some sections, like .symtab or .strtab normally are not
+      // allocatable and do not have virtual addresses. We want to avoid noise
+      // in the YAML output and assume that they are placed at the end.
+      if (Sec.sh_flags & ELF::SHF_ALLOC) {
+        auto S = std::make_unique<ELFYAML::RawContentSection>();
+        if (Error E = dumpCommonSection(&Sec, *S.get()))
+          return std::move(E);
+        Ret.emplace_back(std::move(S));
+      }
       break;
+    }
     case ELF::SHT_SYMTAB_SHNDX: {
       Expected<ELFYAML::SymtabShndxSection *> SecOrErr =
           dumpSymtabShndxSection(&Sec);