Skip to content

Commit 33f1371

Browse files
paulmckrcurostedt
authored andcommitted
ftrace: Use asynchronous grace period for register_ftrace_direct()
When running heavy test workloads with KASAN enabled, RCU Tasks grace periods can extend for many tens of seconds, significantly slowing trace registration. Therefore, make the registration-side RCU Tasks grace period be asynchronous via call_rcu_tasks(). Link: https://lore.kernel.org/linux-trace-kernel/ac05be77-2972-475b-9b57-56bef15aa00a@paulmck-laptop Reported-by: Jakub Kicinski <kuba@kernel.org> Reported-by: Alexei Starovoitov <ast@kernel.org> Reported-by: Chris Mason <clm@fb.com> Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
1 parent c5963a0 commit 33f1371

File tree

1 file changed

+9
-4
lines changed

1 file changed

+9
-4
lines changed

kernel/trace/ftrace.c

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5366,6 +5366,13 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long
53665366
}
53675367
}
53685368

5369+
static void register_ftrace_direct_cb(struct rcu_head *rhp)
5370+
{
5371+
struct ftrace_hash *fhp = container_of(rhp, struct ftrace_hash, rcu);
5372+
5373+
free_ftrace_hash(fhp);
5374+
}
5375+
53695376
/**
53705377
* register_ftrace_direct - Call a custom trampoline directly
53715378
* for multiple functions registered in @ops
@@ -5464,10 +5471,8 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr)
54645471
out_unlock:
54655472
mutex_unlock(&direct_mutex);
54665473

5467-
if (free_hash && free_hash != EMPTY_HASH) {
5468-
synchronize_rcu_tasks();
5469-
free_ftrace_hash(free_hash);
5470-
}
5474+
if (free_hash && free_hash != EMPTY_HASH)
5475+
call_rcu_tasks(&free_hash->rcu, register_ftrace_direct_cb);
54715476

54725477
if (new_hash)
54735478
free_ftrace_hash(new_hash);

0 commit comments

Comments
 (0)