Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stable Functions, Stable Objects, and Stable Classes #4789

Open
wants to merge 99 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
7e6aed0
Runtime system implementation of stable functions
luc-blaeser Oct 18, 2024
d793a91
Continue implementation
luc-blaeser Oct 21, 2024
0056bf7
Continue RTS implementation
luc-blaeser Oct 21, 2024
d509abf
Continue compiler implementation
luc-blaeser Oct 21, 2024
de353ce
Merge branch 'master' into luc/stable-functions
luc-blaeser Oct 21, 2024
5cdb086
Refine compiler support
luc-blaeser Oct 22, 2024
6d6b701
Continue
luc-blaeser Oct 22, 2024
0824a78
Provisional support for flexible function references
luc-blaeser Oct 26, 2024
884af72
Add test case
luc-blaeser Oct 26, 2024
043bb44
Remove debugging code
luc-blaeser Oct 26, 2024
71d25ab
Extend type system for stable functions
luc-blaeser Oct 28, 2024
375d668
Code refactoring
luc-blaeser Oct 30, 2024
2e01e16
Adjust test
luc-blaeser Oct 30, 2024
49968e1
Adjust interpreter
luc-blaeser Oct 30, 2024
8ede27e
Adjust RTS tests
luc-blaeser Oct 30, 2024
a01ea76
Add RTS unit test
luc-blaeser Oct 30, 2024
2c3d872
Adjust test
luc-blaeser Oct 30, 2024
427d34f
Refine system functions type
luc-blaeser Oct 30, 2024
8cb1763
Adjust tests
luc-blaeser Oct 30, 2024
c7a2235
Obtain qualified name for functions
luc-blaeser Oct 31, 2024
05d6e8e
Use qualified identifier for stable functions
luc-blaeser Oct 31, 2024
e911810
Distinguish modules
luc-blaeser Oct 31, 2024
fe42575
Adjust function type
luc-blaeser Oct 31, 2024
61b4da0
Refine type check for flexible functions
luc-blaeser Oct 31, 2024
1e5a400
Remove debug functionality
luc-blaeser Nov 1, 2024
a9d9a6e
Add test case
luc-blaeser Nov 1, 2024
9ea3dad
Support generic stable functions
luc-blaeser Nov 1, 2024
74e9b25
Refine generic stable functions
luc-blaeser Nov 1, 2024
06ddfae
Add test case
luc-blaeser Nov 1, 2024
204101a
Prepare stable closure compatibility check
luc-blaeser Nov 4, 2024
d65a429
Continue preparation of stable closures
luc-blaeser Nov 4, 2024
273fef9
Continue implementation of closure compatibility checks
luc-blaeser Nov 7, 2024
d2dfe08
Continue closure type check
luc-blaeser Nov 7, 2024
271bfd6
Redesign stable type table creation
luc-blaeser Nov 8, 2024
9fdd68d
Adjust persistent type descriptor loading
luc-blaeser Nov 11, 2024
ce5d51f
Constant functions have no captured variables
luc-blaeser Nov 11, 2024
bb1cfba
Remove debug outputs
luc-blaeser Nov 11, 2024
a656d19
Relax assertions
luc-blaeser Nov 11, 2024
dafd670
Remove debug output
luc-blaeser Nov 11, 2024
65afe07
Check stable closure compatibility
luc-blaeser Nov 11, 2024
a8ad4a1
Remove debug output
luc-blaeser Nov 11, 2024
31dba98
Stable closures can only close over stable variables
luc-blaeser Nov 11, 2024
2f465b2
Code refactoring
luc-blaeser Nov 12, 2024
ac3a64c
Identify imported modules
luc-blaeser Nov 13, 2024
304372f
Remove debug outputs and fix tests
luc-blaeser Nov 13, 2024
6b3b437
Refine tests
luc-blaeser Nov 13, 2024
37a4ff2
Support non-canister Wasm
luc-blaeser Nov 13, 2024
807efcb
Support zero function annotations
luc-blaeser Nov 13, 2024
3f537ff
Handle anonymous object definitions assigned to variable
luc-blaeser Nov 13, 2024
1e374d5
Exclude shared functions from stable
luc-blaeser Nov 13, 2024
81860a7
Bug fix
luc-blaeser Nov 13, 2024
38739a9
Refined capture analysis
luc-blaeser Nov 13, 2024
b42ecbf
Support generics in stable closures
luc-blaeser Nov 14, 2024
6b88366
Start with stable function GC
luc-blaeser Nov 20, 2024
3074d51
Continue stable function GC
luc-blaeser Nov 21, 2024
ca5242f
Continue stable function GC
luc-blaeser Nov 21, 2024
8911632
Exclude async functions from being stable
luc-blaeser Nov 21, 2024
7d7f558
Identify generics in stable closures
luc-blaeser Nov 22, 2024
4f29953
Supported nested stable classes
luc-blaeser Nov 22, 2024
011465e
Fix WASI mode
luc-blaeser Nov 22, 2024
e35a26f
Small fix
luc-blaeser Nov 22, 2024
ab89d6e
Handle recursive types in stable function visiting
luc-blaeser Nov 22, 2024
fd40011
Handle actor type in stable function GC
luc-blaeser Nov 22, 2024
f968a6f
Closure stabilization
luc-blaeser Nov 25, 2024
b6f0ab6
Adjust tests
luc-blaeser Nov 25, 2024
ff52f4b
Adjust memory check
luc-blaeser Nov 25, 2024
a259be0
Refactor initialization
luc-blaeser Nov 27, 2024
9c74d12
Refine stable function capture analysis
luc-blaeser Nov 28, 2024
c07c934
Adjust tests
luc-blaeser Nov 28, 2024
792e22e
Refine capture analysis
luc-blaeser Nov 28, 2024
538461b
Adjust tests
luc-blaeser Nov 28, 2024
49a7fea
Add comment
luc-blaeser Nov 28, 2024
02e3bb5
Adjust tests
luc-blaeser Nov 28, 2024
7a282ff
Adjust tests
luc-blaeser Nov 28, 2024
4f262a8
Disable stable functions in classical mode
luc-blaeser Nov 28, 2024
2245191
Support object sharing with type-directed function GC
luc-blaeser Nov 29, 2024
284eb07
Call stable function GC at right point
luc-blaeser Nov 29, 2024
cd83002
Refactor stable generic closure
luc-blaeser Nov 29, 2024
4472f86
Refactor generic stable closure bounds
luc-blaeser Nov 29, 2024
18771bb
Renumber error codes
luc-blaeser Nov 29, 2024
913acb8
Temporarily disable ocamlformat
luc-blaeser Nov 29, 2024
09f80e9
Refine capture analysis
luc-blaeser Nov 30, 2024
5e70b91
Temporarily disable base lib build
luc-blaeser Nov 30, 2024
9f5df3b
Merge branch 'master' into luc/stable-functions
luc-blaeser Nov 30, 2024
f792c1b
Manual merge conflict resolution
luc-blaeser Nov 30, 2024
0a8fa9a
Adjusting comments
luc-blaeser Nov 30, 2024
ecfd254
Adjust tests
luc-blaeser Nov 30, 2024
192d291
Adjust tests
luc-blaeser Nov 30, 2024
fe50a61
Adjust tests
luc-blaeser Nov 30, 2024
8c244af
Temporarily disable base lib building
luc-blaeser Nov 30, 2024
1f3717a
Adjust test
luc-blaeser Nov 30, 2024
7106906
Adjust tests
luc-blaeser Dec 2, 2024
0a4fbb3
Merge branch 'master' into luc/stable-functions
luc-blaeser Jan 8, 2025
282d4b0
Manual merge conflict resolution
luc-blaeser Jan 8, 2025
dadde1f
Adjust tests for renumbered error codes
luc-blaeser Jan 8, 2025
e0b61ca
Generic closure: Add test case, design rationale
luc-blaeser Jan 9, 2025
db79cd9
Some documentation refactoring
luc-blaeser Jan 9, 2025
e37e17b
Small adjustment for test
luc-blaeser Jan 9, 2025
4fd9821
Adjust test
luc-blaeser Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Refine generic stable functions
luc-blaeser committed Nov 1, 2024
commit 74e9b25934fa5b8d64381d4d7045d056f19e9c35
90 changes: 42 additions & 48 deletions rts/motoko-rts/src/idl.rs
Original file line number Diff line number Diff line change
@@ -61,7 +61,8 @@ const IDL_CON_alias: i32 = 1;

const IDL_PRIM_lowest: i32 = -17;

// Extended Candid only
// Extended Candid for enhanced orthogonal persistence
#[enhanced_orthogonal_persistence]
const IDL_STABLE_LOCAL_FUNC_ANNOTATION: u8 = u8::MAX;

// Only used for memory compatiblity checks for orthogonal persistence.
@@ -149,11 +150,6 @@ unsafe fn parse_fields(mode: CompatibilityMode, buf: *mut Buf, n_types: u32) {
}
}

unsafe fn parse_type_bounds(buf: *mut Buf) {
let count = leb128_decode(buf);
assert_eq!(count, 0); // TODO: Support type bounds
}

// NB. This function assumes the allocation does not need to survive GC
// Therefore, no post allocation barrier is applied.
unsafe fn alloc<M: Memory>(mem: &mut M, size: Words<usize>) -> *mut u8 {
@@ -259,13 +255,10 @@ unsafe fn parse_idl_header<M: Memory>(
// Annotations
for _ in 0..leb128_decode(buf) {
let a = read_byte(buf);
// Only during persistence: `IDL_STABLE_LOCAL_FUNC_ANNOTATION` denotes a stable local function.
if !(1 <= a && a <= 3 || extended && a == IDL_STABLE_LOCAL_FUNC_ANNOTATION) {
// Note: `IDL_STABLE_LOCAL_FUNC_ANNOTATION` is not supported during Candid stabilization.
if !(1 <= a && a <= 3) {
idl_trap_with("invalid func annotation");
}
if a == IDL_STABLE_LOCAL_FUNC_ANNOTATION {
parse_type_bounds(buf);
}
// TODO: shouldn't we also check
// * 1 (query) or 2 (oneway), but not both
// * 2 -> |Ret types| == 0
@@ -806,46 +799,47 @@ pub(crate) unsafe fn memory_compatible(
return false;
}
}
// check annotations (that we care about)
// TODO: more generally, we would check equality of 256-bit bit-vectors,
// but validity ensures each entry is 1, 2, 3, or `IDL_STABLE_LOCAL_FUNC_ANNOTATION` (for now)
// c.f. https://github.com/dfinity/candid/issues/318
// `IDL_STABLE_LOCAL_FUNC_ANNOTATION` denotes a stable local function that is only supported for persistence.
let mut a11 = false;
let mut a12 = false;
let mut a13 = false;
let mut a1_stable_local_func = false;
for _ in 0..leb128_decode(&mut tb1) {
match read_byte(&mut tb1) {
1 => a11 = true,
2 => a12 = true,
3 => a13 = true,
4 => a1_stable_local_func = true,
_ => {}
}
// There is exactly one annotation per function in our persistent type table.
let annotation_count1 = leb128_decode(&mut tb1);
assert_eq!(annotation_count1, 1);
let annotation1 = read_byte(&mut tb1);

let annotation_count2 = leb128_decode(&mut tb2);
assert_eq!(annotation_count2, 1);
let annotation2 = read_byte(&mut tb2);

if annotation1 != annotation2 {
return false;
}
let mut a21 = false;
let mut a22 = false;
let mut a23 = false;
let mut a2_stable_local_func = false;
for _ in 0..leb128_decode(&mut tb2) {
match read_byte(&mut tb2) {
1 => a21 = true,
2 => a22 = true,
3 => a23 = true,
4 => a2_stable_local_func = true,
_ => {}
if annotation1 == IDL_STABLE_LOCAL_FUNC_ANNOTATION {
let type_bounds_length1 = leb128_decode(&mut tb1);
let type_bounds_length2 = leb128_decode(&mut tb2);
if type_bounds_length1 != type_bounds_length2 {
return false;
}
for _ in 0..type_bounds_length1 {
let bind_sort1 = read_byte(&mut tb1);
let bind_sort2 = read_byte(&mut tb2);
assert_eq!(bind_sort1, 0); // TODO: Support scope bind
assert_eq!(bind_sort2, 0); // TODO: Support scope bind
let bound1 = sleb128_decode(&mut tb1);
let bound2 = sleb128_decode(&mut tb2);
if !memory_compatible(
rel,
TypeVariance::Invariance,
typtbl1,
typtbl2,
end1,
end2,
bound1,
bound2,
false,
) {
return false;
}
}
}
if a1_stable_local_func {
let type_bounds_length = leb128_decode(&mut tb1);
assert_eq!(type_bounds_length, 0); // TODO: Implement type bounds
}
if a2_stable_local_func {
let type_bounds_length = leb128_decode(&mut tb1);
assert_eq!(type_bounds_length, 0); // TODO: Implement type bounds
}
a11 == a21 && a12 == a22 && a13 == a23 && a1_stable_local_func == a2_stable_local_func
true
}
(IDL_EXT_type_variable, IDL_EXT_type_variable) => {
let index1 = leb128_decode(&mut tb1);
2 changes: 1 addition & 1 deletion src/codegen/compile_enhanced.ml
Original file line number Diff line number Diff line change
@@ -6724,7 +6724,7 @@ module Serialization = struct
let add_type_bound generic =
match generic.sort with
| Type ->
add_leb128 0; (* type bound *)
add_u8 0; (* type bound *)
add_idx generic.bound
| Scope ->
assert false (* TODO: Stable functions: Support scope bounds *)
15 changes: 15 additions & 0 deletions test/run-drun/generic-stable-functions.mo
Original file line number Diff line number Diff line change
@@ -17,4 +17,19 @@ actor {

stable let f2 = genericFunction2;
assert (f2<Int, Nat>(0, 1) == ?0);

class Test<T>() {
public func genericFunction3<U <: T>(x : U) : ?T {
Prim.debugPrint("GENERIC FUNCTION 3");
?x;
}
};
stable let f3 = Test<Int>().genericFunction3;
assert (f3<Nat>(1) == ?1);
};

//SKIP run
//SKIP run-low
//SKIP run-ir
//SKIP comp-ref
//CALL upgrade ""
5 changes: 5 additions & 0 deletions test/run-drun/ok/generic-stable-functions.drun-run.ok
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
ingress Completed: Reply: 0x4449444c016c01b3c4b1f204680100010a00000000000000000101
debug.print: GENERIC FUNCTION 1
debug.print: GENERIC FUNCTION 2
debug.print: GENERIC FUNCTION 3
ingress Completed: Reply: 0x4449444c0000
debug.print: GENERIC FUNCTION 1
debug.print: GENERIC FUNCTION 2
debug.print: GENERIC FUNCTION 3
ingress Completed: Reply: 0x4449444c0000
2 changes: 0 additions & 2 deletions test/run-drun/ok/generic-stable-functions.run-ir.ok

This file was deleted.

2 changes: 0 additions & 2 deletions test/run-drun/ok/generic-stable-functions.run-low.ok

This file was deleted.

2 changes: 0 additions & 2 deletions test/run-drun/ok/generic-stable-functions.run.ok

This file was deleted.