Skip to content

Zero-allocation tree evaluation with buffer #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Dec 12, 2024
Merged

Conversation

MilesCranmer
Copy link
Member

@MilesCranmer MilesCranmer commented Dec 11, 2024

This PR adds the ability to evaluate expression trees with zero allocations by reusing a pre-allocated buffer.

    buffer = zeros(10, size(X, 2))
    eval_options = EvalOptions(; buffer=ArrayBuffer(buffer, Ref(0)))
    result, ok = eval_tree_array(tree, X, operators; eval_options)

The buffer size must be at least (num_intermediate_results, num_samples), where num_intermediate_results depends on the tree structure. You can usually just set this equal to the maximum number of leafs.

Added tests as well.

Copy link
Contributor

github-actions bot commented Dec 11, 2024

Benchmark Results

master 0b5e4a8... master/0b5e4a85b09271...
eval/ComplexF32/evaluation 7.26 ± 0.51 ms 7.25 ± 0.52 ms 1
eval/ComplexF64/evaluation 10.7 ± 0.95 ms 10.8 ± 1.3 ms 0.996
eval/Float32/derivative 11.9 ± 1.3 ms 12.2 ± 1.4 ms 0.977
eval/Float32/derivative_turbo 11.9 ± 1.4 ms 12.2 ± 1.5 ms 0.97
eval/Float32/evaluation 2.73 ± 0.24 ms 2.74 ± 0.24 ms 0.998
eval/Float32/evaluation_bumper 0.609 ± 0.014 ms 0.616 ± 0.014 ms 0.989
eval/Float32/evaluation_turbo 0.581 ± 0.03 ms 0.588 ± 0.029 ms 0.988
eval/Float32/evaluation_turbo_bumper 0.608 ± 0.014 ms 0.614 ± 0.014 ms 0.991
eval/Float64/derivative 15.5 ± 0.78 ms 15.7 ± 0.9 ms 0.993
eval/Float64/derivative_turbo 15.6 ± 1 ms 15.4 ± 0.74 ms 1.02
eval/Float64/evaluation 3.21 ± 0.29 ms 3.18 ± 0.33 ms 1.01
eval/Float64/evaluation_bumper 1.27 ± 0.044 ms 1.27 ± 0.042 ms 1
eval/Float64/evaluation_turbo 1.09 ± 0.059 ms 1.06 ± 0.065 ms 1.03
eval/Float64/evaluation_turbo_bumper 1.27 ± 0.041 ms 1.27 ± 0.043 ms 1
utils/combine_operators/break_sharing 0.0386 ± 0.00054 ms 0.0384 ± 0.00042 ms 1.01
utils/convert/break_sharing 27.7 ± 2.7 μs 27 ± 2 μs 1.03
utils/convert/preserve_sharing 0.0997 ± 0.0043 ms 0.0995 ± 0.004 ms 1
utils/copy/break_sharing 28.4 ± 2.3 μs 28.7 ± 2.4 μs 0.991
utils/copy/preserve_sharing 0.0997 ± 0.004 ms 0.099 ± 0.004 ms 1.01
utils/count_constant_nodes/break_sharing 9.11 ± 0.23 μs 8.88 ± 0.23 μs 1.03
utils/count_constant_nodes/preserve_sharing 0.0861 ± 0.0036 ms 0.0857 ± 0.0038 ms 1.01
utils/count_depth/break_sharing 9.76 ± 0.21 μs 9.75 ± 0.26 μs 1
utils/count_nodes/break_sharing 8.32 ± 0.21 μs 8.32 ± 0.16 μs 0.999
utils/count_nodes/preserve_sharing 0.0852 ± 0.0034 ms 0.0851 ± 0.0033 ms 1
utils/get_set_constants!/break_sharing 0.033 ± 0.0019 ms 0.0337 ± 0.0028 ms 0.978
utils/get_set_constants!/preserve_sharing 0.177 ± 0.006 ms 0.179 ± 0.0058 ms 0.991
utils/get_set_constants_parametric 0.0443 ± 0.0026 ms 0.0462 ± 0.0022 ms 0.96
utils/has_constants/break_sharing 4.06 ± 0.13 μs 4.24 ± 0.13 μs 0.958
utils/has_operators/break_sharing 2.01 ± 0.073 μs 2.02 ± 0.081 μs 0.994
utils/hash/break_sharing 22.7 ± 0.57 μs 23.3 ± 0.94 μs 0.973
utils/hash/preserve_sharing 0.0979 ± 0.0037 ms 0.0981 ± 0.0034 ms 0.998
utils/index_constant_nodes/break_sharing 24.9 ± 0.89 μs 25.6 ± 1.4 μs 0.973
utils/index_constant_nodes/preserve_sharing 0.101 ± 0.004 ms 0.102 ± 0.0038 ms 0.988
utils/is_constant/break_sharing 4.39 ± 0.14 μs 4.94 ± 0.15 μs 0.888
utils/simplify_tree/break_sharing 0.174 ± 0.0033 ms 0.174 ± 0.0027 ms 0.997
utils/simplify_tree/preserve_sharing 0.226 ± 0.0043 ms 0.223 ± 0.0045 ms 1.01
utils/string_tree/break_sharing 0.461 ± 0.015 ms 0.466 ± 0.019 ms 0.989
utils/string_tree/preserve_sharing 0.57 ± 0.02 ms 0.571 ± 0.026 ms 0.999
time_to_load 0.233 ± 0.0021 s 0.234 ± 0.002 s 0.994

@MilesCranmer MilesCranmer merged commit 15d9efb into master Dec 12, 2024
14 checks passed
@MilesCranmer MilesCranmer deleted the buffered-evals branch December 12, 2024 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant