Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion book.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ cache-timeout = 90000
warning-policy = "error"

[output.html.redirect]
"/backend/inline-asm.html" = "/backend/asm.html"
"/backend/inline-asm.html" = "/asm.html"
"/borrow_check.html" = "borrow-check.html"
"/borrow_check/drop_check.html" = "/borrow-check/drop-check.html"
"/borrow_check/moves_and_initialization.html" = "/borrow-check/moves-and-initialization.html"
Expand Down
66 changes: 32 additions & 34 deletions src/backend/codegen.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ Usually, rustc uses LLVM for code generation,
but there is also support for [Cranelift] and [GCC].
The key is that rustc doesn't implement codegen itself.
It's worth noting, though, that in the Rust source code,
many parts of the backend have `codegen` in their names
(there are no hard boundaries).
many parts of the backend have `codegen` in their names (there are no hard boundaries).

[Cranelift]: https://github.com/bytecodealliance/wasmtime/tree/main/cranelift
[GCC]: https://github.com/rust-lang/rustc_codegen_gcc
Expand All @@ -20,56 +19,55 @@ many parts of the backend have `codegen` in their names
## What is LLVM?

[LLVM](https://llvm.org) is "a collection of modular and reusable compiler and
toolchain technologies". In particular, the LLVM project contains a pluggable
toolchain technologies".
In particular, the LLVM project contains a pluggable
compiler backend (also called "LLVM"), which is used by many compiler projects,
including the `clang` C compiler and our beloved `rustc`.

LLVM takes input in the form of LLVM IR. It is basically assembly code with
additional low-level types and annotations added. These annotations are helpful
for doing optimizations on the LLVM IR and outputted machine code. The end
result of all this is (at long last) something executable (e.g. an ELF object,
LLVM takes input in the form of LLVM IR.
It is basically assembly code with additional low-level types and annotations added.
These annotations are helpful for doing optimizations on the LLVM IR and outputted machine code.
The end result of all this is (at long last) something executable (e.g. an ELF object,
an EXE, or wasm).

There are a few benefits to using LLVM:

- We don't have to write a whole compiler backend. This reduces implementation
and maintenance burden.
- We don't have to write a whole compiler backend.
This reduces implementation and maintenance burden.
- We benefit from the large suite of advanced optimizations that the LLVM
project has been collecting.
- We can automatically compile Rust to any of the platforms for which LLVM has
support. For example, as soon as LLVM added support for wasm, voila! rustc,
clang, and a bunch of other languages were able to compile to wasm! (Well,
there was some extra stuff to be done, but we were 90% there anyway).
- We and other compiler projects benefit from each other. For example, when the
[Spectre and Meltdown security vulnerabilities][spectre] were discovered,
- We can automatically compile Rust to any of the platforms for which LLVM has support.
For example, as soon as LLVM added support for wasm, voila!
rustc, clang, and a bunch of other languages were able to compile to wasm!
(Well, there was some extra stuff to be done, but we were 90% there anyway).
- We and other compiler projects benefit from each other.
For example, when the [Spectre and Meltdown security vulnerabilities][spectre] were discovered,
only LLVM needed to be patched.

[spectre]: https://meltdownattack.com/

## Running LLVM, linking, and metadata generation

Once LLVM IR for all of the functions and statics, etc is built, it is time to
start running LLVM and its optimization passes. LLVM IR is grouped into
"modules". Multiple "modules" can be codegened at the same time to aid in
multi-core utilization. These "modules" are what we refer to as _codegen
units_. These units were established way back during monomorphization
collection phase.
start running LLVM and its optimization passes.
LLVM IR is grouped into "modules".
Multiple "modules" can be codegened at the same time to aid in multi-core utilization.
These "modules" are what we refer to as _codegen units_.
These units were established way back during monomorphization collection phase.

Once LLVM produces objects from these modules, these objects are passed to the
linker along with, optionally, the metadata object and an archive or an
executable is produced.

It is not necessarily the codegen phase described above that runs the
optimizations. With certain kinds of LTO, the optimization might happen at the
linking time instead. It is also possible for some optimizations to happen
before objects are passed on to the linker and some to happen during the
linking.

This all happens towards the very end of compilation. The code for this can be
found in [`rustc_codegen_ssa::back`][ssaback] and
[`rustc_codegen_llvm::back`][llvmback]. Sadly, this piece of code is not
really well-separated into LLVM-dependent code; the [`rustc_codegen_ssa`][ssa]
contains a fair amount of code specific to the LLVM backend.
linker along with, optionally, the metadata object and an archive or an executable is produced.

It is not necessarily the codegen phase described above that runs the optimizations.
With certain kinds of LTO, the optimization might happen during linking time instead.
It is also possible for some optimizations to happen
before objects are passed on to the linker and some to happen during the linking.

This all happens towards the very end of compilation.
The code for this can be found in [`rustc_codegen_ssa::back`][ssaback] and
[`rustc_codegen_llvm::back`][llvmback].
Sadly, this piece of code is not really well-separated into LLVM-dependent code;
the [`rustc_codegen_ssa`][ssa] contains a fair amount of code specific to the LLVM backend.

Once these components are done with their work you end up with a number of
files in your filesystem corresponding to the outputs you have requested.
Expand Down
87 changes: 50 additions & 37 deletions src/backend/implicit-caller-location.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Implicit caller location

Approved in [RFC 2091], this feature enables the accurate reporting of caller location during panics
initiated from functions like `Option::unwrap`, `Result::expect`, and `Index::index`. This feature
adds the [`#[track_caller]`][attr-reference] attribute for functions, the
initiated from functions like `Option::unwrap`, `Result::expect`, and `Index::index`.
This feature adds the [`#[track_caller]`][attr-reference] attribute for functions, the
[`caller_location`][intrinsic] intrinsic, and the stabilization-friendly
[`core::panic::Location::caller`][wrapper] wrapper.

Expand Down Expand Up @@ -40,14 +40,16 @@ library which propagate caller information.
## Reading caller location

Previously, `panic!` made use of the `file!()`, `line!()`, and `column!()` macros to construct a
[`Location`] pointing to where the panic occurred. These macros couldn't be given an overridden
location, so functions which intentionally invoked `panic!` couldn't provide their own location,
[`Location`] pointing to where the panic occurred.
These macros couldn't be given an overridden location,
so functions which intentionally invoked `panic!` couldn't provide their own location,
hiding the actual source of error.

Internally, `panic!()` now calls [`core::panic::Location::caller()`][wrapper] to find out where it
was expanded. This function is itself annotated with `#[track_caller]` and wraps the
[`caller_location`][intrinsic] compiler intrinsic implemented by rustc. This intrinsic is easiest
explained in terms of how it works in a `const` context.
was expanded.
This function is itself annotated with `#[track_caller]` and wraps the
[`caller_location`][intrinsic] compiler intrinsic implemented by rustc.
This intrinsic is easiest explained in terms of how it works in a `const` context.

## Caller location in `const`

Expand All @@ -57,35 +59,38 @@ to find the right location and allocating a const value to return.
### Finding the right `Location`

In a const context we "walk up the stack" from where the intrinsic is invoked, stopping when we
reach the first function call in the stack which does *not* have the attribute. This walk is in
[`InterpCx::find_closest_untracked_caller_location()`][const-find-closest].
reach the first function call in the stack which does *not* have the attribute.
This walk is in [`InterpCx::find_closest_untracked_caller_location()`][const-find-closest].

Starting at the bottom, we iterate up over stack [`Frame`][const-frame]s in the
[`InterpCx::stack`][const-stack], calling
[`InstanceKind::requires_caller_location`][requires-location] on the
[`Instance`s from each `Frame`][frame-instance]. We stop once we find one that returns `false` and
[`Instance`s from each `Frame`][frame-instance].
We stop once we find one that returns `false` and
return the span of the *previous* frame which was the "topmost" tracked function.

### Allocating a static `Location`

Once we have a `Span`, we need to allocate static memory for the `Location`, which is performed by
the [`TyCtxt::const_caller_location()`][const-location-query] query. Internally this calls
[`InterpCx::alloc_caller_location()`][alloc-location] and results in a unique
[memory kind][location-memory-kind] (`MemoryKind::CallerLocation`). The SSA codegen backend is able
to emit code for these same values, and we use this code there as well.
the [`TyCtxt::const_caller_location()`][const-location-query] query.
Internally this calls [`InterpCx::alloc_caller_location()`][alloc-location] and results in a unique
[memory kind][location-memory-kind] (`MemoryKind::CallerLocation`).
The SSA codegen backend is able to emit code for these same values,
and we use this code there as well.

Once our `Location` has been allocated in static memory, our intrinsic returns a reference to it.

## Generating code for `#[track_caller]` callees

To generate efficient code for a tracked function and its callers, we need to provide the same
behavior from the intrinsic's point of view without having a stack to walk up at runtime. We invert
the approach: as we grow the stack down we pass an additional argument to calls of tracked functions
rather than walking up the stack when the intrinsic is called. That additional argument can be
returned wherever the caller location is queried.

The argument we append is of type `&'static core::panic::Location<'static>`. A reference was chosen
to avoid unnecessary copying because a pointer is a third the size of
behavior from the intrinsic's point of view without having a stack to walk up at runtime.
We invert the approach:
as we grow the stack down we pass an additional argument to calls of tracked functions
rather than walking up the stack when the intrinsic is called.
That additional argument can be returned wherever the caller location is queried.

The argument we append is of type `&'static core::panic::Location<'static>`.
A reference was chosen to avoid unnecessary copying because a pointer is a third the size of
`std::mem::size_of::<core::panic::Location>() == 24` at time of writing.

When generating a call to a function which is tracked, we pass the location argument the value of
Expand All @@ -105,7 +110,8 @@ stack downward.

### Codegen examples

What does this transformation look like in practice? Take this example which uses the new feature:
What does this transformation look like in practice?
Take this example which uses the new feature:

```rust
#![feature(track_caller)]
Expand Down Expand Up @@ -139,13 +145,15 @@ fn main() {
### Dynamic dispatch

In codegen contexts we have to modify the callee ABI to pass this information down the stack, but
the attribute expressly does *not* modify the type of the function. The ABI change must be
transparent to type checking and remain sound in all uses.
the attribute expressly does *not* modify the type of the function.
The ABI change must be transparent to type checking and remain sound in all uses.

Direct calls to tracked functions will always know the full codegen flags for the callee and can
generate appropriate code. Indirect callers won't have this information and it's not encoded in
generate appropriate code.
Indirect callers won't have this information and it's not encoded in
the type of the function pointer they call, so we generate a [`ReifyShim`] around the function
whenever taking a pointer to it. This shim isn't able to report the actual location of the indirect
whenever taking a pointer to it.
This shim isn't able to report the actual location of the indirect
call (the function's definition site is reported instead), but it prevents miscompilation and is
probably the best we can do without modifying fully-stabilized type signatures.

Expand All @@ -163,16 +171,18 @@ function:
* is not a closure
* is not `#[naked]`

If the use is valid, we set [`CodegenFnAttrsFlags::TRACK_CALLER`][attrs-flags]. This flag influences
the return value of [`InstanceKind::requires_caller_location`][requires-location] which is in turn
If the use is valid, we set [`CodegenFnAttrsFlags::TRACK_CALLER`][attrs-flags].
This flag influences the return value of
[`InstanceKind::requires_caller_location`][requires-location] which is in turn
used in both const and codegen contexts to ensure correct propagation.

### Traits

When applied to trait method implementations, the attribute works as it does for regular functions.

When applied to a trait method prototype, the attribute applies to all implementations of the
method. When applied to a default trait method implementation, the attribute takes effect on
When applied to a trait method prototype,
the attribute applies to all implementations of the method.
When applied to a default trait method implementation, the attribute takes effect on
that implementation *and* any overrides.

Examples:
Expand Down Expand Up @@ -235,26 +245,29 @@ fn main() {
}
```

## Background/History
## Background/history

Broadly speaking, this feature's goal is to improve common Rust error messages without breaking
stability guarantees, requiring modifications to end-user source, relying on platform-specific
debug-info, or preventing user-defined types from having the same error-reporting benefits.

Improving the output of these panics has been a goal of proposals since at least mid-2016 (see
[non-viable alternatives] in the approved RFC for details). It took two more years until RFC 2091
was approved, much of its [rationale] for this feature's design having been discovered through the
[non-viable alternatives] in the approved RFC for details).
It took two more years until RFC 2091 was approved,
much of its [rationale] for this feature's design having been discovered through the
discussion around several earlier proposals.

The design in the original RFC limited itself to implementations that could be done inside the
compiler at the time without significant refactoring. However in the year and a half between the
approval of the RFC and the actual implementation work, a [revised design] was proposed and written
up on the tracking issue. During the course of implementing that, it was also discovered that an
compiler at the time without significant refactoring.
However in the year and a half between the approval of the RFC and the actual implementation work,
a [revised design] was proposed and written up on the tracking issue.
During the course of implementing that, it was also discovered that an
implementation was possible without modifying the number of arguments in a function's MIR, which
would simplify later stages and unlock use in traits.

Because the RFC's implementation strategy could not readily support traits, the semantics were not
originally specified. They have since been implemented following the path which seemed most correct
originally specified.
They have since been implemented following the path which seemed most correct
to the author and reviewers.

[RFC 2091]: https://github.com/rust-lang/rfcs/blob/master/text/2091-inline-semantic.md
Expand Down
39 changes: 21 additions & 18 deletions src/backend/lowering-mir.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,24 @@
# Lowering MIR to a Codegen IR

Now that we have a list of symbols to generate from the collector, we need to
generate some sort of codegen IR. In this chapter, we will assume LLVM IR,
since that's what rustc usually uses. The actual monomorphization is performed
as we go, while we do the translation.
generate some sort of codegen IR.
In this chapter, we will assume LLVM IR,
since that's what rustc usually uses.
The actual monomorphization is performed as we go, while we do the translation.

Recall that the backend is started by
[`rustc_codegen_ssa::base::codegen_crate`][codegen1]. Eventually, this reaches
[`rustc_codegen_ssa::mir::codegen_mir`][codegen2], which does the lowering from
MIR to LLVM IR.
Recall that the backend is started by [`rustc_codegen_ssa::base::codegen_crate`][codegen1].
Eventually, this reaches
[`rustc_codegen_ssa::mir::codegen_mir`][codegen2], which does the lowering from MIR to LLVM IR.

[codegen1]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/base/fn.codegen_crate.html
[codegen2]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/mir/fn.codegen_mir.html

The code is split into modules which handle particular MIR primitives:

- [`rustc_codegen_ssa::mir::block`][mirblk] will deal with translating
blocks and their terminators. The most complicated and also the most
interesting thing this module does is generating code for function calls,
blocks and their terminators.
The most complicated and also the most interesting thing this module does
is generating code for function calls,
including the necessary unwinding handling IR.
- [`rustc_codegen_ssa::mir::statement`][mirst] translates MIR statements.
- [`rustc_codegen_ssa::mir::operand`][mirop] translates MIR operands.
Expand All @@ -31,25 +32,27 @@ The code is split into modules which handle particular MIR primitives:
[mirrv]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/mir/rvalue/index.html

Before a function is translated a number of simple and primitive analysis
passes will run to help us generate simpler and more efficient LLVM IR. An
example of such an analysis pass would be figuring out which variables are
passes will run to help us generate simpler and more efficient LLVM IR.
An example of such an analysis pass would be figuring out which variables are
SSA-like, so that we can translate them to SSA directly rather than relying on
LLVM's `mem2reg` for those variables. The analysis can be found in
[`rustc_codegen_ssa::mir::analyze`][mirana].
LLVM's `mem2reg` for those variables.
The analysis can be found in [`rustc_codegen_ssa::mir::analyze`][mirana].

[mirana]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/mir/analyze/index.html

Usually a single MIR basic block will map to a LLVM basic block, with very few
exceptions: intrinsic or function calls and less basic MIR statements like
`assert` can result in multiple basic blocks. This is a perfect lede into the
non-portable LLVM-specific part of the code generation. Intrinsic generation is
fairly easy to understand as it involves very few abstraction levels in between
`assert` can result in multiple basic blocks.
This is a perfect lede into the non-portable LLVM-specific part of the code generation.
Intrinsic generation is fairly easy to understand
as it involves very few abstraction levels in between
and can be found in [`rustc_codegen_llvm::intrinsic`][llvmint].

[llvmint]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_llvm/intrinsic/index.html

Everything else will use the [builder interface][builder]. This is the code that gets
called in the [`rustc_codegen_ssa::mir::*`][ssamir] modules discussed above.
Everything else will use the [builder interface][builder].
This is the code that gets called in the
[`rustc_codegen_ssa::mir::*`][ssamir] modules discussed above.

[builder]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_llvm/builder/index.html
[ssamir]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/mir/index.html
Expand Down
Loading
Loading