[llvm-gsymutil] Option --symtab-file to specify a separate binary for…

… symbol table

Summary:
I made minimal changes possible to enable this for Strobelight's llvm-gsymutil migration.

Since Strobelight is only on linux dealing with ELF, and no Universal Binary will be involved, I am making the `--symtab-file` option limited to the simple object file case.

We will need to refactor the code to better handle the `MachOUniversalBinary` type given there're different arch involved.

Test Plan:
I made a private build of `llvm-gsymutil` that can be downloaded with
```
jf download GICWmABaZb_9K94FAI4uIkiuhtZebpZBAAAB --file "llvm-gsymutil"
```

Manually tested the conversion using files from IndexerTest.cpp unit tests.

Use both `main` & `main.debug`
```
$ /home/wanyi/llvm-sand/build/Debug/fbcode-x86_64/toolchain/bin/llvm-gsymutil --convert main.debug --symtab-file main -o main.gsym
Input file: main.debug
Output file (x86_64): main.gsym
Loaded 1 functions from DWARF.
Using symbol table file: main
Loaded 9 functions from symbol table.
Pruned 0 functions, ended with 10 total
```

Use only `main.debug` (debug info only)
```
$ /home/wanyi/llvm-sand/build/Debug/fbcode-x86_64/toolchain/bin/llvm-gsymutil --convert main.debug -o debug_info_only.gsym
Input file: main.debug
Output file (x86_64): debug_info_only.gsym
Loaded 1 functions from DWARF.
Loaded 0 functions from symbol table.
Pruned 0 functions, ended with 1 total
```

Use only `main` (symbol table only)
```
$ /home/wanyi/llvm-sand/build/Debug/fbcode-x86_64/toolchain/bin/llvm-gsymutil --convert main -o symbol_table_only.gsym
Input file: main
Output file (x86_64): symbol_table_only.gsym
Loaded 0 functions from DWARF.
Loaded 10 functions from symbol table.
Pruned 0 functions, ended with 10 total
```

ELF debug info + universal binary for symbol table
```
[wanyi@devvm9268.frc0 ~/fbsource/fbcode/services_efficiency/symbol_service/tests/testpackage3 (71a39fcf2|remote/master)]$ file universal universal: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE>] [arm64:Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE>]
[wanyi@devvm9268.frc0 ~/fbsource/fbcode/services_efficiency/symbol_service/tests/testpackage3 (71a39fcf2|remote/master)]$ /home/wanyi/llvm-sand/build/Debug/fbcode-x86_64/toolchain/bin/llvm-gsymutil --convert main.debug --symtab-file universal
Input file: main.debug
Output file (x86_64): main.debug.gsym
Loaded 1 functions from DWARF.
Using symbol table file: universal
DWARF conversion failed: : Input symbol table file universal is not a valid object file.
Supported files are ELF and mach-o (exclude universal binary) files.
```

Symbol table only
```
Desktop/tmp/test_llvm-gsymutil
▶ /Users/wanyi/llvm-sand/build/Debug/Darwin-arm64/toolchain/bin/llvm-gsymutil --convert universal
Input file: universal
Output file (x86_64): universal.gsym.x86_64
Loaded 0 functions from DWARF.
Loaded 50 functions from symbol table.
Pruned 0 functions, ended with 50 total
Output file (arm64): universal.gsym.arm64
Loaded 0 functions from DWARF.
Loaded 50 functions from symbol table.
Pruned 0 functions, ended with 50 total
```

dSYM -both debug info & symbol table
```
Desktop/tmp/test_llvm-gsymutil
▶ /Users/wanyi/llvm-sand/build/Debug/Darwin-arm64/toolchain/bin/llvm-gsymutil --convert universal.dSYM/Contents/Resources/DWARF/universal -o debug_info
Input file: universal.dSYM/Contents/Resources/DWARF/universal
Output file (x86_64): debug_info.x86_64
Loaded 49 functions from DWARF.
Loaded 1 functions from symbol table.
Pruned 0 functions, ended with 50 total
Output file (arm64): debug_info.arm64
Loaded 49 functions from DWARF.
Loaded 1 functions from symbol table.
Pruned 0 functions, ended with 50 total
```

`--symtab-file` flag should fail
```
Desktop/tmp/test_llvm-gsymutil
▶ /Users/wanyi/llvm-sand/build/Debug/Darwin-arm64/toolchain/bin/llvm-gsymutil --convert universal.dSYM/Contents/Resources/DWARF/universal --symtab-file universal
Input file: universal.dSYM/Contents/Resources/DWARF/universal
DWARF conversion failed: : --symtab-file is not accepted for universal binary conversion
```

I wasn't able to strip the symbol table out of the DWARF
Tried following
```
Desktop/tmp/test_llvm-gsymutil
▶ /opt/llvm/bin/llvm-objcopy --only-keep-debug arm64.dSYM/Contents/Resources/DWARF/arm64 arm64.debug
Desktop/tmp/test_llvm-gsymutil
▶ objdump --syms arm64.debug | awk '{print $5;}' > symbols
Desktop/tmp/test_llvm-gsymutil
▶ /opt/llvm/bin/llvm-objcopy --strip-symbols=./symbols arm64.debug
```

dSYM - both debug info & symbol table
```
Desktop/tmp/test_llvm-gsymutil
▶ /Users/wanyi/llvm-sand/build/Debug/Darwin-arm64/toolchain/bin/llvm-gsymutil --convert arm64.dSYM/Contents/Resources/DWARF/arm64 -o arm64_debug_info
Input file: arm64.dSYM/Contents/Resources/DWARF/arm64
Output file (aarch64): arm64_debug_info
Loaded 49 functions from DWARF.
Loaded 1 functions from symbol table.
Pruned 0 functions, ended with 50 total
```

Symbol table only
```
Desktop/tmp/test_llvm-gsymutil
▶ /Users/wanyi/llvm-sand/build/Debug/Darwin-arm64/toolchain/bin/llvm-gsymutil --convert arm64 -o arm64_symbol_table
Input file: arm64
Output file (aarch64): arm64_symbol_table
Loaded 0 functions from DWARF.
Loaded 50 functions from symbol table.
Pruned 0 functions, ended with 50 total
```

Reviewers: slinger, jeffreytan

Reviewed By: jeffreytan

Subscribers: ayermolo, #lldb_team

Differential Revision: https://phabricator.intern.facebook.com/D45341225

Tasks: T151296270

Tags: aarch64

Commit #2 Differential Revision: https://phabricator.intern.facebook.com/D45792908

Commit #2 Tasks: T152287419

Commit llvm#3 Differential Revision: https://phabricator.intern.facebook.com/D49360184

llvm/tools/llvm-gsymutil/Opts.td

@@ -40,3 +40,6 @@ def addresses_from_stdin :
 defm json_summary_file :
   Eq<"json-summary-file",
      "Output a categorized summary of errors into the JSON file specified.">;
+defm symtab_file :
+  Eq<"symtab-file",
+     "Specify a separate file for symbol table to GSYM conversion.\nIn case the symbol table and debug info are not in the same binary. Does not support universal binary.">;

llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp

@@ -96,6 +96,7 @@ static bool Quiet;
 static std::vector<uint64_t> LookupAddresses;
 static bool LookupAddressesFromStdin;
 static bool StoreMergedFunctionInfo = false;
+static std::string SymbolTableFilename;
 
 static void parseArgs(int argc, char **argv) {
   GSYMUtilOptTable Tbl;
@@ -114,8 +115,9 @@ static void parseArgs(int argc, char **argv) {
         "information in each GSYM file.\n"
         "Specify a single GSYM file along with one or more --lookup options to "
         "lookup addresses within that GSYM file.\n"
-        "Use the --convert option to specify a file with option --out-file "
-        "option to convert to GSYM format.\n";
+        "Use the --convert option to specify a file (with --symtab-file if "
+        "needed) "
+        "with option --out-file option to convert to GSYM format.\n";
 
     Tbl.printHelp(llvm::outs(), "llvm-gsymutil [options] <input GSYM files>",
                   Overview);
@@ -177,6 +179,9 @@ static void parseArgs(int argc, char **argv) {
 
   LookupAddressesFromStdin = Args.hasArg(OPT_addresses_from_stdin);
   StoreMergedFunctionInfo = Args.hasArg(OPT_merged_functions);
+
+  if (const llvm::opt::Arg *A = Args.getLastArg(OPT_symtab_file_EQ))
+    SymbolTableFilename = A->getValue();
 }
 
 /// @}
@@ -367,7 +372,38 @@ static llvm::Error handleObjectFile(ObjectFile &Obj, const std::string &OutFile,
     Gsym.prepareMergedFunctions(Out);
 
   // Get the UUID and convert symbol table to GSYM.
-  if (auto Err = ObjectFileTransformer::convert(Obj, Out, Gsym))
+  // Use a separate file for symbol table if specified
+  std::string SymtabFile = SymbolTableFilename;
+  ErrorOr<std::unique_ptr<MemoryBuffer>> SymtabBuffOrErr = nullptr;
+  std::unique_ptr<MemoryBuffer> SymtabBuffer = nullptr;
+  if (!SymtabFile.empty()) {
+    outs() << "Using symbol table file: " << SymbolTableFilename << "\n";
+    SymtabBuffOrErr = MemoryBuffer::getFileOrSTDIN(SymtabFile);
+    error(SymtabFile, SymtabBuffOrErr.getError());
+    SymtabBuffer = std::move(SymtabBuffOrErr.get());
+    Expected<std::unique_ptr<Binary>> SymtabBinOrErr =
+        object::createBinary(*SymtabBuffer);
+    error(SymtabFile, errorToErrorCode(SymtabBinOrErr.takeError()));
+    if (auto Symtab = dyn_cast<ObjectFile>(SymtabBinOrErr->get())) {
+      Triple ObjTriple(Obj.makeTriple());
+      Triple SymtabTriple(Symtab->makeTriple());
+      if (ObjTriple.getArchName() != SymtabTriple.getArchName())
+        return createStringError(std::errc::invalid_argument,
+                                 "Cannot use symbol table file %s in %s for "
+                                 "binary in %s architecture.",
+                                 SymtabFile.c_str(),
+                                 ObjTriple.getArchName().data(),
+                                 SymtabTriple.getArchName().data());
+      if (auto Err = ObjectFileTransformer::convert(*Symtab, Out, Gsym))
+        return Err;
+    } else
+      return createStringError(std::errc::invalid_argument,
+                               "Input symbol table file %s is not a "
+                               "valid object file.\n"
+                               "Supported files are ELF and mach-o "
+                               "(exclude universal binary) files.",
+                               SymtabFile.c_str());
+  } else if (auto Err = ObjectFileTransformer::convert(Obj, Out, Gsym))
     return Err;
 
   // Finalize the GSYM to make it ready to save to disk. This will remove
@@ -410,6 +446,12 @@ static llvm::Error handleBuffer(StringRef Filename, MemoryBufferRef Buffer,
     if (auto Err = handleObjectFile(*Obj, OutFile, Out))
       return Err;
   } else if (auto *Fat = dyn_cast<MachOUniversalBinary>(BinOrErr->get())) {
+    // Symbol table file is not accepted with universal binary
+    std::string SymtabFile = SymbolTableFilename;
+    if (!SymtabFile.empty())
+      return createStringError(std::errc::invalid_argument,
+                               "--symtab-file is not accepted for "
+                               "universal binary conversion");
     // Iterate over all contained architectures and filter out any that were
     // not specified with the "--arch <arch>" option. If the --arch option was
     // not specified on the command line, we will process all architectures.