This patch changes the default mode of Dexter from heuristic-mode to
script-mode. The --use-script argument is replaced with --use-heuristic,
some comments/docs/error messages are updated accordingly, and tests
have their flags switched accordingly.
In the test `ctor.cpp`, Dexter tests that when we step into the
constructor used in the test, the value of `this` isn't irretrievable.
To protect against a rotten pass it also checked that we stepped once,
but this caused failures on some platforms where we would step twice on
the constructor - this patch resolves the issue by accepting any
non-zero value for the number of steps.
This patch replaces uses of heuristic-mode Dexter in the dexter-tests
suite with uses of the script-mode, for tests that use DAP (via
lldb-dap). The updates are largely straightforward but occasionally
non-trivial, and in some cases some slight modifications have been made
to keep the "spirit" of the test intact.
This patch makes some required changes prior to switching over to using
the script model by default, to keep the tests in `dexter-tests` working
and meaningful.
In addition to the existing metrics, we split the "missing_vars" metric
into 3 - adding "irretrievable" and "optimized_out", so that we can
specifically test those properties in the output.
We also add some function name sanitizing to lldb-dap, to account for
the fact that some function names get presented with a suffix " [opt]",
which causes the script to fail.
This patch adds documentation for the script model to the Dexter README,
shunting heuristic-mode information into a separate doc, creating a new
doc for script-mode, and linking to both (with a brief summary of the
differences) from the base README.
Adds a new node type, !float, which can be used to match debugger ouptut
as
float values rather than as strings, optionally allowing a range to be
specified for inexact matches. This new node allows a list of values to
be
given, effectively a shorthand for a list of individual !float nodes.
This patch slightly extends the matching of aggregate members to allow
for lists of expected values for individual functioning the same as
lists of expected values for scalar values. This is accomplished by
recursively calling `get_expect_match` instead of `DebuggerExpectMatch`,
which contains logic for matching against all possible expected values
and selecting the best match (and committing any associated context
changes, e.g. modified address labels).
This patch adds the second kind of variable expect, !type, which tests
the type of a variable as reported by the debugger. As with !value, this
is a string comparison of the debugger output with the script expected
value - this means that even if two types are identical (e.g. typedef),
a !type node will only match the one that the debugger displays by
default.
Script writing and aggregates work the same for !type as for !value, and
the metrics reported are largely similar, with the exception that
"unexpected", "seen", and "missing" metrics are reported separately for
values and types.
This patch enables the ability for state nodes to check conditions,
meaning they will be active only if the condition is met.
Condition evaluation is somewhat language specific; we directly check
whether the value of the evaluated expression is "true"
(case-insensitive), which works for the languages we actually use Dexter
with, but may require generalizing in future.
We also cache conditions as they are evaluated; each time we step, we
clear all cached conditions for the current frame and any expired
frames, but we keep the cached conditions for any frames rootwards from
the current frame; this prevents us from unexpectedly exiting out of a
callee frame because of debug info not surviving a stack unwind; if the
early exit is desired, an !and{at_frame_idx, condition} under the lower
frame may suffice.
The after_hit_count attribute for a state node causes it to become
active only after it would have become active N times. This uses the
existing logic for incrementing hit counts, i.e. after the node becomes
"active", we will not add another hit count until it stops being active
for at least one step. Since state nodes with after_hit_count do not
become active before reaching the required hit count, this requires us
to keep track of an "early" set of state nodes, meaning nodes that would
be active if not for their after_hit_count.
Following from the previous patch, this patch adds support to Dexter for
generating expected values for !step nodes. This is relatively limited:
the kind of !step which this is most well-suited to this is !step
exactly, as the !step order of ignoring extra lines is redundant (all
lines are added as expected values), and !step never can't know what
lines could have been stepped on but weren't without some extra work
(e.g. finding viable breakpoint locations in the enclosing state node).
This patch adds a node for generating metrics based on lines stepped on.
The new node has 3 versions: !step exactly, !step order, and !step
never, which check an expected list of line numbers against the actual
line numbers seen while the expect is active. "exactly" and "order" both
check that the list of line numbers appear in-order in the debugger
while the expect is in scope; "exactly" treats any seen line numbers
outside of this list as incorrect, while "order" ignores them. "never"
does not have an order, but treats any line numbers from the list that
are seen in the debugger as being incorrect.
This patch adds a new attribute for !and nodes, `at_frame_idx`, which
matches against frames above its parent node; for example, in the
script:
```
!where {function: foo}:
!where {function: bar}:
!and {at_frame_idx: 1}:
!value x: 0
```
The `!value x` node checks the value of 'x' in 'foo' while the debugger
is inside 'bar'. Use of this attribute comes with some restrictions: a
!where node can never be nested under a !and{at_frame_idx} node, and
neither can another !and{at_frame_idx} node.
This patch adds the ability for state nodes to use a `for_hit_count:
<int>` field to limit the number of times that a given state node will
be active.
The hit count for each state node increments each time it is _entered_,
meaning it must be inactive (not matching any frame on the stack) in the
prior step; for example, if a `!where {lines}` node covers the entirety
of a loop, repeated iterations of that loop will not increment the
hit_count - only once the line range is existed and entered again will
the hit_count increment.
Following on from the previous patch, this patch adds support for
writing expects from !value/all nodes, generating separate expects for
each variable in the requested debugger scope, for each continuous range
of lines it is live for.
NB: As with the other patch that implements script-writing features,
each test in this patch has a corresponding "expected" file, which is
almost identical (including the `RUN` lines), and exists to be `diff`'d
against the output of Dexter's script generation.
To further improve Dexter's script writing ability, this patch starts
implementing the ability for Dexter to fetch all variables with in a
given "scope", as defined by the DAP "scopes" request. This allows the
test to collect all available variables without needing to specify them
explicitly in the script, aiding in fast script
generation/re-generation.
This patch does not add any script-writing functionality, but adds the
!value/all Node, which fetches all variable values from the given scope,
and enables fetching these values from DAP-based debuggers.
This patch fixes an error that caused some Dexter test failures, driven
by two separate causes. The first issue is that frames below main were
appearing in the program stacktrace; while Dexter tries to filter frames
below main during the stacktrace collection step based on a pre-written
list, this list may not be comprehensive enough, as the symbol
"___lldb_unnamed_symbol_2a150" has also appeared. In order to guard
against this and future cases that might appear, this patch adds a check
to Dexter for "presentationHint: deemphasize" in the DAP response; this
is added by LLDB (and other dap-based debuggers) as a hint that the
frame is not user source, and should be a generally useful way of
avoiding evaluating frames that are not wanted.
The second issue is a mismatch between the breakpoint-setting logic and
the state-matching logic: the former allows root !where nodes to omit
the "file" field, using the script file as a default file. The state
matching logic does not perform any checking for an omitted file.
Together, this means that we may correctly set breakpoints for e.g.
"test.cpp:10", but when we go to match against frames, the !where node
may match against some other line-10 that appears below (e.g. as a
caller of) the frame at test.cpp:10. This patch copies the default_file
logic to state matching, meaning that we will only match against the
intended frame.
This reverts commit 2cf48dca33.
The original commit is failing sometimes in pre-commit CI for linux
builds, possibly due to some unspecified environmental dependency.
Adds a node type for Dexter that allows checking abstract labels instead
of concrete addresses. Each address node has a label and optional
offset, and the first time during evaluation that a given address label
is matched against a valid pointer value, the address label will be
assigned a value that matches the seen address (adjusting for any
offset). From that point, the resolved address value will be used for
the remainder of the test evaluation.
Following on from the previous patch, this patch allows Dexter to write
disaggregated expected values for aggregate variables. Dexter eagerly
tries to disaggregate whenever subvalues are available, but will
fallback to the root/parent value if all available subvalues are
unavailable.
NB: Each test in this patch has a corresponding "expected" file, which
is almost identical (including the `RUN` lines), and exists to be
`diff`'d against the output of Dexter's script generation.
Reverts llvm/llvm-project#205657
The original commit was causing pre-merge CI to fail for AArch64, as one
of the tests expects stepping behaviour that is seen on not seen on
AArch64 targets; the test suite containing the failing test is meant to
be configured to not run for AArch64, but the unsupported label was not
being applied, due to an error in the unsupported check. This patch
fixes the unsupported check in scripts/lit.local.cfg, which should
prevent further errors.
Reverts llvm/llvm-project#202799 to fix the failing
cross-project-tests::debuginfo-tests/dexter/feature_tests/scripts/rewriting/rewrite_expect_list.cpp
test and unblock pre-commit CI.
This patch adds a feature to Dexter that allows scripts to be passed to
Dexter with missing expected values (`null` values in YAML), which
Dexter will attempt to "fill-in" with expected values that match the
debugger's actual output. The result is written to a file with the same
name as the original test file, in the directory given by
--results-directory if one is present; all content outside of the Dexter
script itself is preserved exactly as-is.
NB: Each test in this patch has a corresponding "expected" file, which
is almost identical (including the `RUN` lines), and exists to be
`diff`'d against the output of Dexter's script generation.
In order to exercise more control over stepping in Dexter tests, this
patch adds the `!then` node which can be used to step out of a function
or exit the current test. Unlike expect nodes, !then nodes appear as
direct singular children of a state node:
!where {lines: 10}: !then finish
The two currently available commands are "step_out" and "finish".
step_out performs a debugger "step out" command, skipping over all
!wheres in the current frame and not stepping into any lower !wheres.
The finish command ends the debugger session immediately after finishing
the current step.
Allows aggregate expects to be written in Dexter scripts, in the form:
```
!value expect:
member1: 0
member2:
submember1: 1
submember2: 2
```
This uses the "variables" feature provided in the DAP interface to
recursively fetch sub-variables from each evaluated variable, and
modifies the DebuggerExpectMatch class to also be recursive, containing
submatches for each expected aggregate member value.
Some key concepts that are affected by this are the "match_result",
which has changed from a bool to a 3-way result ("FALSE", "TRUE",
"PARTIAL"), where "PARTIAL" means that at least one submatch was TRUE
and at least one was FALSE. We also add a `match_distance`, which is
used as a measure of correctness. The match distance ranges from 0.0 to
1.0: a total match has a distance of 0.0, a total mismatch has 1.0, and
a partial match's distance is equal to the average distance of its
immediate children.
This reverts commit
2a789821b0.
The original commit previously caused pre-merge check failures for Linux
AArch64 cross-project-tests, due to the stepping behaviour being
slightly different to x86_64. The tests have been adjusted to be less
brittle to exact stepping behaviour, but this reapply also disables the
tests for ARM targets as there is not currently the infrastructure to
reliably test them, meaning we may end up with latent failures.
Reverts llvm/llvm-project#202544
Commit appears to be causing failures in the pre-merge checks for
AArch64 specifically, though this is not entirely consistent. Reverting
to fix pre-merge checks immediately, will aim to reland with either a
fix or disabling tests for AArch64.
This patch adds a !label node to Dexter scripts, which references lines
at a position in the source program marked with "!dex_label
<identifier>". Each label use can be given a positive or negative
offset, and file lookup is based on the provided --source-root-dir
(using the test file's directory if none is provided).
A previous commit (78ae864) added a test that contained off-by-1 errors
in the line numbers used in the Dexter script; this patch adjusts these
to the correct line number.
This patch adds support for nested state nodes, !where and a new !and
node. Nested state nodes are evaluated only at a frame relative to the
frame that their parent matched to:
- `!where` can only match the frame immediately called from/leafwards
from their parent frame.
- `!and` can only match the same frame as their parent frame.
The recently-added structured script feature currently relies on
DAP-based debuggers, of which the only one currently supported by Dexter
is LLDB. In order to prevent the tests that depend on this feature from
running for other debuggers, we require LLDB for the script test
directory.
This patch adds evaluation for structured scripts, completing the
features required to run simple Dexter tests using structured scripts.
The basic output from these evaluations is a list of named metrics
aggregating the results of evaluating !value nodes. The verbose output
gives a per-step summary of the results for each expect node active at
that step.
Most of the new functionality is in the evaluation/ dir, which has also
absorbed some functionality previously stored in the
ScriptDebuggerController for matching !where nodes to a debugger StepIR,
as this is logic which is common to both managing a debugger session and
evaluating the end result.
This patch adds a debugger controller for structured scripts. This
controller operates as follows:
- !where nodes at the root of the script (currently the only kind
allowed) have function or line breakpoints set to cover them.
- Whenever the debugger stops, the controller will examine the stack to
determine whether any !where nodes are in scope.
- While any !where is in scope, its associated !values will be evaluated
in the debugger and the results stored, and the debugger will
single-step.
- When no !where is in scope, the debugger will continue.
This is a simplified implementation compared to the final version, as it
is missing support for nested !where nodes, Scope evaluation, and
conditions/hit counts; these will be added in later commits.
See PSA:
https://discourse.llvm.org/t/psa-planned-changes-to-dexter/90402
This patch begins adding support for "structured scripts" to Dexter,
starting with some of the core classes and the ability to parse script
files. This patch does not add the ability to actually run scripts, or
any of the underlying functionality required to do so.
NB: This patch adds a dependency on PyYAML, which is specified in a new
requirements.txt file.
This patch adds some extra state collection methods to DebuggerBase and
implements them for DAP only. These methods are used to fetch a
stacktrace without variable information, and to populate variable
information into a StepIR containing only a stacktrace. These methods
are currently unused, making this patch NFC, but this is a necessary
precursor to the new script model, where we examine the stacktrace to
determine what variable info we will collect.
As part of the stacktrace-collection function, we also fetch the
instruction address for each stack frame, if it is made available by the
debugger; to enable this, this patch adds a new value with default
`None` to `FrameIR`.
DAP logs are currently optionally output by Dexter to assist in
debugging or analyzing Dexter sessions. The output currently includes
the contents of every DAP message sent to/from the debug adapter, but
for some long-running programs it can also be useful to know when
messages have been sent and received; to assist, this patch adds
timestamps to DAP messages in the log.
This patch ensures we run the `cross-project-tests` on `llvm/` PRs. At
the very least those tests depend on debug-info and
`llvm/ADT/`/`llvm/Support` (and the data-formatters that also live in
`llvm/`).
This would've helped catch
https://github.com/llvm/llvm-project/pull/188483 at pre-merge time.
Initially added in #187709. It was reverted in #188833, because
[llvm-clang-x86_64-sie-win](https://lab.llvm.org/buildbot/#/builders/46/builds/32873)
was failing in
`cross-project-tests/debuginfo-tests/dexter-tests/nrvo.cpp`.
The test passed for me locally. After checking on another machine, I
found that `S_DEFRANGE_REGISTER_REL_INDIR` is only supported by
dbgeng/WinDbg from Windows 10.0 Build 19041 (released 2020) onwards.
SDKs before this will fail to read the value. That buildbot is on
Windows 10.0 Build 17763.
I'm not sure if we should make the generation of that record
conditional. Debuggers that can't read the record will skip it. They'll
still see that there's some local variable, but won't be able to display
the value.
As far as I know, users of older Windows 10 builds should be able to
install a newer Windows SDK and use the WinDbg from that version. But I
haven't tested that.
This reverts commit 08a4085. The change breaks `nvro.cpp` in the
debugging tests on the buildbot
(https://lab.llvm.org/buildbot/#/builders/46/builds/32873) but works
locally for me. It might be because the buildbot is using an older
Windows SDK.
In addition, it reverts parts of #188769 (using `.` over `->`).
After #187709, the
[nrvo.cpp](3a56470a0e/cross-project-tests/debuginfo-tests/dexter-tests/nrvo.cpp)
test failed (e.g.
https://lab.llvm.org/buildbot/#/builders/46/builds/32858), because it
couldn't evaluate the expression `result.i` anymore. This was because it
actually evaluated `result->i`, which doesn't work anymore as `result`
is not a pointer or reference. Before #187709, `result` was declared as
a reference that lives at `frame-pointer+offset`.
Now it's no longer a reference, but the location is
`*(frame-pointer+offset) + 0`.
There weren't any other tests that were accessing struct fields and used
dbgeng as the debugger.
This patch adds a `has-gdb` LIT feature that I then use in the
`REQUIRES` clause for `llgdb` tests.
These tests were failing because `llgdb.py` couldn't find `gdb` on the
host when I tried enabling these tests on PR CI
(https://github.com/llvm/llvm-project/issues/188775).
We should get `gdb` installed on the buildbots. But we should also be
able to run the rest of the `cross-project-tests` in the meantime even
if gdb isn't installed.
PointerUnion stores a fixed-width `ceil(log2(N))`-bit tag in the low
bits of the pointer. This works only when every member type provides at
least that many low bits — if the least-aligned type doesn't,
compilation fails, even though the higher-aligned types may have plenty
of spare bits going to waste.
Introduce a variable-length escape-encoded tag that exploits the extra
low bits of higher-aligned types, analogous to UTF-8: types are grouped
into tiers by NumLowBitsAvailable; each non-final tier reserves one code
as an escape prefix, and the next tier extends the tag into the newly
available bits. This allows PointerUnion to hold more type variants than
a fixed-width tag permits.
The fixed-width path is used when the minimum alignment already provides
enough bits (the common case); the variable-width path activates only
when it doesn't, and requires types to be listed in non-decreasing
NumLowBitsAvailable order.
I need this for https://github.com/llvm/llvm-project/pull/186923 which
requires a 6-member PointerUnion in MLIR TypeRange/ValueRange. On 32-bit
systems, some members only provide 2 low bits, insufficient for a 3-bit
fixed-width tag.
In #188242, we replaced `PointerUnion`'s `PointerIntPair` storage with
`PunnedPointer<void*>`. The old formatters relied on the PIP synthetic
provider (LLDB) / `get_pointer_int_pair helper` (GDB) which no longer
work.
Instead, read raw bytes from `PunnedPointer` and compute the active tag
from template argument type alignments -- the same fixed-width encoding
the C++ implementation uses. When template arg enumeration is truncated
(e.g., function-local types in GDB), the formatters fall back to showing
a tag-stripped `void*` instead of silently misdecoding.
Alternatives that didn't work out:
- Adding a C++ helper (`getActiveMemberIdx`) callable from Python: gets
optimized out even with `__attribute__((used, noinline))`, and
expression evaluation fails for synthetic children.
- Using `isa`/`dyn_cast` checks from Python: requires expression
evaluation, which does not work for local types or synthetic children
without a frame context.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
