1
1
# Queries: demand-driven compilation
2
2
3
+ <!-- toc -->
4
+
3
5
As described in [ the high-level overview of the compiler] [ hl ] , the Rust compiler
4
6
is still (as of <!-- date: 2021-07 --> July 2021) transitioning from a
5
- traditional "pass-based" setup to a "demand-driven" system. ** The Compiler Query
6
- System is the key to our new demand-driven organization.** The idea is pretty
7
- simple. You have various queries that compute things about the input – for
8
- example, there is a query called ` type_of(def_id) ` that, given the [ def-id] of
7
+ traditional "pass-based" setup to a "demand-driven" system. The compiler query
8
+ system is the key to rustc's demand-driven organization.
9
+ The idea is pretty simple. Instead of entirely independent passes
10
+ (parsing, type-checking, etc.), a set of function-like * queries*
11
+ compute information about the input source. For example,
12
+ there is a query called ` type_of ` that, given the [ ` DefId ` ] of
9
13
some item, will compute the type of that item and return it to you.
10
14
11
- [ def-id ] : appendix/glossary.md#def-id
15
+ [ `DefId` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/def_id/struct.DefId.html
12
16
[ hl ] : ./compiler-src.md
13
17
14
- Query execution is ** memoized** – so the first time you invoke a
18
+ Query execution is * memoized* . The first time you invoke a
15
19
query, it will go do the computation, but the next time, the result is
16
20
returned from a hashtable. Moreover, query execution fits nicely into
17
- ** incremental computation** ; the idea is roughly that, when you do a
18
- query, the result ** may* * be returned to you by loading stored data
19
- from disk (but that's a separate topic we won't discuss further here).
21
+ * incremental computation* ; the idea is roughly that, when you invoke a
22
+ query, the result * may* be returned to you by loading stored data
23
+ from disk. [ ^ incr-comp-detail ]
20
24
21
- The overall vision is that, eventually, the entire compiler
22
- control-flow will be query driven. There will effectively be one
23
- top-level query (" compile" ) that will run compilation on a crate; this
25
+ Eventually, we want the entire compiler
26
+ control-flow to be query driven. There will effectively be one
27
+ top-level query (` compile ` ) that will run compilation on a crate; this
24
28
will in turn demand information about that crate, starting from the
25
29
* end* . For example:
26
30
27
- - This " compile" query might demand to get a list of codegen-units
31
+ - The ` compile ` query might demand to get a list of codegen-units
28
32
(i.e. modules that need to be compiled by LLVM).
29
33
- But computing the list of codegen-units would invoke some subquery
30
34
that returns the list of all modules defined in the Rust source.
31
35
- That query in turn would invoke something asking for the HIR.
32
36
- This keeps going further and further back until we wind up doing the
33
37
actual parsing.
34
38
35
- However, that vision is not fully realized. Still, big chunks of the
36
- compiler (for example, generating MIR) work exactly like this.
37
-
38
- ### Incremental Compilation in Detail
39
+ Although this vision is not fully realized, large sections of the
40
+ compiler (for example, generating [ MIR] ( ./mir/ ) ) currently work exactly like this.
39
41
40
- The [ Incremental Compilation in Detail] [ query-model ] chapter gives a more
42
+ [ ^ incr-comp-detail ] : The [ " Incremental Compilation in Detail] ( queries/incremental-compilation-in-detail.md ) chapter gives a more
41
43
in-depth description of what queries are and how they work.
42
44
If you intend to write a query of your own, this is a good read.
43
45
44
- [ query-model ] : queries/incremental-compilation-in-detail.md
45
-
46
46
### Invoking queries
47
47
48
- To invoke a query is simple. The tcx ("type context") offers a method
49
- for each defined query. So, for example, to invoke the ` type_of `
48
+ Invoking a query is simple. The [ ` TyCtxt ` ] ("type context") struct offers a method
49
+ for each defined query. For example, to invoke the ` type_of `
50
50
query, you would just do this:
51
51
52
52
``` rust,ignore
53
53
let ty = tcx.type_of(some_def_id);
54
54
```
55
55
56
+ [ `TyTcx` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html
57
+
56
58
### How the compiler executes a query
57
59
58
60
So you may be wondering what happens when you invoke a query
@@ -162,13 +164,13 @@ they define both a `provide` and a `provide_extern` function, through
162
164
[ rustc_metadata ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/index.html
163
165
[ wasm_import_module_map ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/back/symbol_export/fn.wasm_import_module_map.html
164
166
165
- ### Adding a new kind of query
167
+ ### Adding a new query
166
168
167
- So suppose you want to add a new kind of query, how do you do so ?
168
- Well, defining a query takes place in two steps:
169
+ How do you add a new query?
170
+ Defining a query takes place in two steps:
169
171
170
- 1 . first, you have to specify the query name and arguments; and then,
171
- 2 . you have to supply query providers where needed.
172
+ 1 . Specify the query name and its arguments.
173
+ 2 . Supply query providers where needed.
172
174
173
175
To specify the query name and arguments, you simply add an entry to
174
176
the big macro invocation in
@@ -190,21 +192,22 @@ rustc_queries! {
190
192
```
191
193
192
194
Queries are grouped into categories (` Other ` , ` Codegen ` , ` TypeChecking ` , etc.).
193
- Each group contains one or more queries. Each query definition is broken up like
194
- this:
195
+ Each group contains one or more queries.
196
+
197
+ A query definition has the following form:
195
198
196
199
``` rust,ignore
197
200
query type_of(key: DefId) -> Ty<'tcx> { ... }
198
- ^^ ^^^^^^^ ^^^^^ ^^^^^^^^ ^^^
201
+ ^^^^^ ^^^^^^^ ^^^^^ ^^^^^^^^ ^^^
199
202
| | | | |
200
203
| | | | query modifiers
201
- | | | result type of query
204
+ | | | result type
202
205
| | query key type
203
206
| name of query
204
207
query keyword
205
208
```
206
209
207
- Let's go over them one by one:
210
+ Let's go over these elements one by one:
208
211
209
212
- ** Query keyword:** indicates a start of a query definition.
210
213
- ** Name of query:** the name of the query method
@@ -217,11 +220,7 @@ Let's go over them one by one:
217
220
- ** Result type of query:** the type produced by this query. This type
218
221
should (a) not use ` RefCell ` or other interior mutability and (b) be
219
222
cheaply cloneable. Interning or using ` Rc ` or ` Arc ` is recommended for
220
- non-trivial data types.
221
- - The one exception to those rules is the ` ty::steal::Steal ` type,
222
- which is used to cheaply modify MIR in place. See the definition
223
- of ` Steal ` for more details. New uses of ` Steal ` should ** not** be
224
- added without alerting ` @rust-lang/compiler ` .
223
+ non-trivial data types.[ ^ steal ]
225
224
- ** Query modifiers:** various flags and options that customize how the
226
225
query is processed (mostly with respect to [ incremental compilation] [ incrcomp ] ).
227
226
@@ -234,11 +233,16 @@ So, to add a query:
234
233
- Link the provider by modifying the appropriate ` provide ` method;
235
234
or add a new one if needed and ensure that ` rustc_driver ` is invoking it.
236
235
236
+ [ ^ steal ] : The one exception to those rules is the ` ty::steal::Steal ` type,
237
+ which is used to cheaply modify MIR in place. See the definition
238
+ of ` Steal ` for more details. New uses of ` Steal ` should ** not** be
239
+ added without alerting ` @rust-lang/compiler ` .
240
+
237
241
#### Query structs and descriptions
238
242
239
- For each kind , the ` rustc_queries ` macro will generate a "query struct"
240
- named after the query. This struct is a kind of a place-holder
241
- describing the query. Each such struct implements the
243
+ For each query , the ` rustc_queries ` macro will generate a "query struct"
244
+ named after the query. This struct is a kind of placeholder
245
+ describing the query. Each query struct implements the
242
246
[ ` self::config::QueryConfig ` ] [ QueryConfig ] trait, which has associated types for the
243
247
key/value of that particular query. Basically the code generated looks something
244
248
like this:
@@ -308,3 +312,4 @@ More discussion and issues:
308
312
[ GitHub issue #42633 ] : https://github.com/rust-lang/rust/issues/42633
309
313
[ Incremental Compilation Beta ] : https://internals.rust-lang.org/t/incremental-compilation-beta/4721
310
314
[ Incremental Compilation Announcement ] : https://blog.rust-lang.org/2016/09/08/incremental.html
315
+
0 commit comments