Tracking issue for removing PDB plugin in LLDB

[ZequanWu]

This is to track the progress of fixing all failed tests when only using native PDB plugin (-DLLVM_ENABLE_DIA_SDK=NO).

Currently, there are 12 failed tests and 1 unexpected passed tests.

Failed Tests (12):
  lldb-shell :: Expr/nodefaultlib.cpp
  lldb-shell :: SymbolFile/DWARF/dwo-static-data-member-access.test
  lldb-shell :: SymbolFile/PDB/class-layout.test
  lldb-shell :: SymbolFile/PDB/enums-layout.test
  lldb-shell :: SymbolFile/PDB/expressions.test
  lldb-shell :: SymbolFile/PDB/func-symbols.test
  lldb-shell :: SymbolFile/PDB/pointers.test
  lldb-shell :: SymbolFile/PDB/type-quals.test
  lldb-shell :: SymbolFile/PDB/typedefs.test
  lldb-shell :: SymbolFile/PDB/udt-layout.test
  lldb-shell :: SymbolFile/PDB/variables.test
  lldb-shell :: Unwind/windows-unaligned-x86_64.test

********************
Unexpectedly Passed Tests (1):
  lldb-shell :: Commands/command-disassemble-mixed.c

Here are the causes for those failed tests:

• No decl information for record types and functions: class-layout.test, enums-layout.test, func-symbols.test, pointers.test
• LLDB failed to lookup method names in NativePDB plugin #51933: expressions.test
• Some types not showing up in Types under lldb-test: pointers.test, type-quals.test, typedefs.test
• Crash with Assertion failed: D->isCompleteDefinition() && "Cannot layout type before complete!": udt-layout.test
• Native PDB doesn't showing block inside function when dumping symbols with lldb-test: variables.test
• Other: windows-unaligned-x86_64.test, nodefaultlib.cpp, dwo-static-data-member-access.test

Once those tests are fixed, we might be able to proceed with removing PDB plugin.

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Zequan Wu (ZequanWu)

Details

This is to track the progress of fixing all failed tests when only using native PDB plugin (`-DLLVM_ENABLE_DIA_SDK=NO`).

Currently, there are 12 failed tests and 1 unexpected passed tests.

Failed Tests (12):
  lldb-shell :: Expr/nodefaultlib.cpp
  lldb-shell :: SymbolFile/DWARF/dwo-static-data-member-access.test
  lldb-shell :: SymbolFile/PDB/class-layout.test
  lldb-shell :: SymbolFile/PDB/enums-layout.test
  lldb-shell :: SymbolFile/PDB/expressions.test
  lldb-shell :: SymbolFile/PDB/func-symbols.test
  lldb-shell :: SymbolFile/PDB/pointers.test
  lldb-shell :: SymbolFile/PDB/type-quals.test
  lldb-shell :: SymbolFile/PDB/typedefs.test
  lldb-shell :: SymbolFile/PDB/udt-layout.test
  lldb-shell :: SymbolFile/PDB/variables.test
  lldb-shell :: Unwind/windows-unaligned-x86_64.test

********************
Unexpectedly Passed Tests (1):
  lldb-shell :: Commands/command-disassemble-mixed.c

Here are the causes for those failed tests:

• No decl information for record types and functions: class-layout.test, enums-layout.test, func-symbols.test, pointers.test
• LLDB failed to lookup method names in NativePDB plugin #51933: expressions.test
• Some types not showing up in Types under lldb-test: pointers.test, type-quals.test, typedefs.test
• Crash with Assertion failed: D->isCompleteDefinition() && "Cannot layout type before complete!": udt-layout.test
• Native PDB doesn't showing block inside function when dumping symbols with lldb-test: variables.test
• Other: windows-unaligned-x86_64.test, nodefaultlib.cpp, dwo-static-data-member-access.test

Once those tests are fixed, we might be able to proceed with removing PDB plugin.

[Walnut356]

Some types not showing up in Types under lldb-test: pointers.test, type-quals.test, typedefs.test

Not sure how relevant this is to these test failures specifically, but I have a bit of information about typedefs from working on a typesystem implementation that may be of use.

There seems to be a couple major defects with the way typedefs are handled currently:

• SymbolFileNativePDB::FindType (called by the front-end SB stuff for type lookups) only looks in the type stream. As it stands, typedefs only live in the symbol stream and hold references into the type stream, so the type lookup function can never find them unless they've already been parsed.

• Local variable codeview nodes store the type of the object via a type index, not a symbol index. Because of the above, that means the type will always end up being the underlying type rather than the typedef.

I don't know exactly how the debug info generation works on LLVM's end so bear with me. Assuming LLVM itself generates the raw codeview nodes and those nodes are what end up in the final PDB file, I don't think LLVM is making use of lf_alias which is the type stream node for typedefs. From the bit I looked into, handling that node would require a small change in CodeViewTypes.def and then a bunch of followup changes to add visitor functions to the serializers/deserializers. I'm not 100% sure what's necessary after that to get LLVM to generate the node through the DIBuilder API though.

[Nerixyz]

I don't think LLVM is making use of lf_alias which is the type stream node for typedefs.

MSVC doesn't seem to emit this either. Compiling https://github.com/llvm/llvm-project/blob/04f98889ae497d895be7c478415e0e9a8c6d330b/lldb/test/Shell/SymbolFile/PDB/Inputs/SimpleTypesTest.cpp with MSVC and dumping the PDB (with llvm's dump, cvdump, and MS's rust version) doesn't show any LF_ALIAS. Do you have an example where Clang/LLVM doesn't generate this, but MSVC does?

SymbolFileNativePDB::FindType (called by the front-end SB stuff for type lookups) only looks in the type stream.

From what I can tell, C typedefs are emitted as S_UDT. LLDB does parse these in ParseTypes. Still looks like something is wrong with that.

To be able to find these in FindTypes, we'd need to walk the globals stream. We could probably cache this after the first call.

[Walnut356]

MSVC doesn't seem to emit this either.

Tbh, i'm not sure if MSVC is right or wrong in this case. The node exists, it's labeled for use to designate typedefs, but then the original codeview spec also lists the udt node for typedefs. To be fair though, lf_alias didn't exist in the original codeview spec so UDT is all they had at that point. In my eyes, the fact that they went out of their way to make a specialized one means it should probably be used when possible, and downstream consumers (e.g. TypeSystem) would absolutely benefit from being able to tell what's a typedef and what's not. There are other instances of nodes not being used by MSVC when they probably should, such as the TaggedUnion UDT kind, which specifically notes its use in Rust, but is never actually emitted for Rust code afaik.

From what I can tell, C typedefs are emitted as S_UDT.

I should specify, I'm mainly coming from a Rust perspective. Rust outputs msvc-style primitive names (that disappear because codeview doesn't have a name field for primitives anyway), and then wraps them in a typedef with the Rust-style name (e.g. u8). These typedefs appear in the symbol stream, but there is no UDT (or any other type-stream node) with type name u8 that I can find.

For what it's worth, I tried manually wrapping pointer/ref nodes in structs a while ago so that the UDT node would be guaranteed to exist and be attached to the local variable. It worked on my end (on windows-msvc, windows-gnu, and linux-gnu) but CI gave cryptic errors about broken modules so iunno.

Later today I can look into the C side of things a bit more. I did some tests comparing C and Rust output a few months ago and I remember there being similar issues with typedefs between the two, but I don't remember well enough to say anything definitive.

LLDB does parse these in ParseTypes.

afaict ParseTypes is only called via Module::ParseAllDebugSymbols, and I don't think that ends up called as a result of the type lookup command, nor is it it called via SBModule::FindFirstType.

msdia is still able to snag some of them during regular processing, as evidenced by this PR actually working (we heavily rely on SBModule::FindFirstType for container visualizers since PDB doesn't output template args), but there's also still no way to associate those typedefs with local variables. I checked via a PDB dump, and even with pointers getting ref$<u8> typedefs, there's nothing discoverable by that name in the type stream. I'd test it with the native pdb parser, but it's currently broken for Rust i think? It looks like it's using the unique ID as the type name instead of the name field so the output isn't super helpful lol

Native:

Dia:

[Nerixyz]

In my eyes, the fact that they went out of their way to make a specialized one means it should probably be used when possible, and downstream consumers (e.g. TypeSystem) would absolutely benefit from being able to tell what's a typedef and what's not.

Through S_UDT, the symbol file can already tell if something is a typedef. Maybe I'm missing something here.

afaict ParseTypes is only called via Module::ParseAllDebugSymbols, and I don't think that ends up called as a result of the type lookup command, nor is it it called via SBModule::FindFirstType.

Oh yeah, I was referring to the typedefs.test which parses all types.

but there is no UDT (or any other type-stream node) with type name u8 that I can find.

Compiling a small Rust program yields UTDs for u8, u16, u32, and u64.

I'd test it with the native pdb parser, but it's currently broken for Rust i think? It looks like it's using the unique ID as the type name instead of the name field so the output isn't super helpful lol

I don't think there's an issue for this yet - not sure what the best approach would be because UniqueName yields consistent type names for C++. Would be great to get this fixed.

---

Are you compiling LLDB by yourself? If so, you could try Nerixyz@745d749. This should resolve types that are specified by S_UDT.

Either way, this discussion feels a bit too specific for this issue. So it might be good to make a sub-issue for typedef/S_UDT/LF_ALIAS.

[Walnut356]

Through S_UDT, the symbol file can already tell if something is a typedef. Maybe I'm missing something here.

Local variables store their type via a type index, this is also true for global variables. Since typedef nodes don't live in the type stream, variables cannot have typedefs as types.

The front-end asks the TypeSystem about the type of a variable, but the TypeSystem and SymbolFile can't tell the difference between an unsigned char var; and a uint8_t var; because the actual type-node referenced by the variable will always be the underlying raw unsigned char node.

I don't know exactly how the test is failing, or exactly how the test file is executed, but if it ever tries to check if a local or global variable has a typedef-type, it will always fail with the way things currently are. If typedefs were also generated in the type stream via lf_alias (equivalent to dwarf's DW_TAG_typedef), variables could references those nodes directly and typedefs would appear properly in the front-end.

Compiling a small Rust program yields UTDs for u8, u16, u32, and u64.

My b, I conflated the UDT symbol node and the Class type node. UDT nodes are generated for typedefs, but the above problem will exist so long as typedefs only live in symbol nodes.

[DavidSpickett]

We have been discussing this in https://discourse.llvm.org/t/rfc-removing-the-dia-pdb-plugin-from-lldb/87827 and have concluded that we still want to remove the DIA plugin eventually. With the first step being to make native the default.

This issue started with a list of failing tests, so here's the current state of things.

First results from @Nerixyz on x86_64 (from https://discourse.llvm.org/t/rfc-removing-the-dia-pdb-plugin-from-lldb/87827/20):

Sure, there are still 12 failures (x86, but a few different ones):

• Settings/TestFrameFormatFunctionReturnObjC.test (new) and SymbolFile/DWARF/dwo-static-data-member-access.test (known)
  breakpoint set -n <name> doesn’t work with the native plugin(?). Might be because AddSymbols isn’t implemented yet ([LLDB][NativePDB] Implement `AddSymbols` by Nerixyz · Pull Request #154121 · llvm/llvm-project · GitHub). Though for the second test, it’s unexpected that this runs with PDB on Windows in the first place.
• SymbolFile/PDB/expressions.test (known)
  Can’t find static variable in namespace.
• SymbolFile/PDB/func-symbols.test (known)
  Argument and return types are not created by the native plugin.
• SymbolFile/PDB/function-nested-block.test (known)
  In the DIA plugin, the main function is found as ‘name = “main”, mangled = “main”’ but in the native plugin, there’s no mangled name.
• SymbolFile/PDB/native-setting.cpp (new)
  This fails because there’s no DIA plugin. So the native plugin is always used. Maybe there should be an additional REQUIRES for DIA support?
• SymbolFile/PDB/pointers.test (known)
  Fails because function types in the native plugin don’t have a name. The DIA plugin created the function types with a name. For example, int foo(int) would create a Type{...} name="foo" compiler_type = ... int (int). The native plugin creates Type{...} compiler_type = ... int (int).
  Furthermore, the native plugin doesn’t include any blocks and their variables.
• SymbolFile/PDB/type-quals.test (known)
  Same reason as SymbolFile/PDB/func-symbols.test
• SymbolFile/PDB/typedefs.test (known)
  Fails because the native plugin creates typedefs with the typedef’d type. So typedef char MyChar would show as Type{..} , name = "MyChar", size = 1, compiler_type = char whereas DIA would show ... compiler_type = typedef MyChar. Seems like a simple fix.
• SymbolFile/PDB/udt-layout.test (known)
  This hits an assertion failure, because it tries to print a static whose type doesn’t have a layout through target variable. It’s a type from the CRT that is forward declared without a complete definition.
• SymbolFile/PDB/variables.test (known)
  Fails because functions don’t have a mangled name yet (might be more failures after that).
• Unwind/windows-unaligned-x86_64.test (known)
  The symbols for the assembly functions are not found.

SymbolFile/PDB/class-layout.test and SymbolFile/PDB/enums-layout.test are no longer failing.

I did the same on Windows on Arm (https://discourse.llvm.org/t/rfc-removing-the-dia-pdb-plugin-from-lldb/87827/21) and got the same results when we ignore the x86_64 specific tests, and a toolchain issue in my local env. Plus 1 unexpected pass.

********************
Failed Tests (12):
  lldb-shell :: BuildScript/toolchain-msvc.test    <<<< (due to my local environment)
  lldb-shell :: Settings/TestFrameFormatFunctionReturnObjC.test
  lldb-shell :: SymbolFile/DWARF/dwo-static-data-member-access.test
  lldb-shell :: SymbolFile/PDB/expressions.test
  lldb-shell :: SymbolFile/PDB/func-symbols.test
  lldb-shell :: SymbolFile/PDB/function-nested-block.test
  lldb-shell :: SymbolFile/PDB/native-setting.cpp
  lldb-shell :: SymbolFile/PDB/pointers.test
  lldb-shell :: SymbolFile/PDB/type-quals.test
  lldb-shell :: SymbolFile/PDB/typedefs.test
  lldb-shell :: SymbolFile/PDB/udt-layout.test
  lldb-shell :: SymbolFile/PDB/variables.test

********************
Unexpectedly Passed Tests (1):
  lldb-shell :: Commands/command-disassemble-mixed.c

So we have a couple of things fixed, a couple of new failures and the same UPASS as seen before.