Least connections algorithm

[karlvr]

Perhaps to reopen the conversation from #71, I have ported just the least connections algorithm to the master branch. We have abandoned the weighted and slow start that complicated the previous implementation. We have been using LEAST in production for > 5 years. We've tried switching back to the default RR implementation but experience issues when one of our backends runs slowly.

As you noted in #71 it isn't a guarantee of least connections (as the connection is not reserved) however we are treating it as a heuristic, and as such it has worked really well for us.

Again, if you're interested, I am happy to work this code to get it into a shape you like. I've tried to reproduce the basic logic from dom_find in dom_find_leastconn without worrying about the alt behaviour as it uses dom_find as a fall-back anyway. I look forward to hearing your thoughts when you have the time & energy!

[nigoroll]

I really wish we had a way in Varnish-Cache to add backends to directors other than from VCL ¹, such that dynamic could be given an instance of another director to use as the next layer. This way, we would not need to reinvent the wheel whenever we want to add another load balancing method, avoiding the otherwise unavoidable path towards "unidirectors" and keeping the separation of concerns...
Maybe I should really bite the bullet and look into how such an implementation could look like, but I guess the main issue will be additional parameters...
Other than that, yes, I get your point and I appreciate bringing the topic up again, thank you.

Footnotes

1. Also relevant in another context, see "Reverse HTTP" here ↩

[karlvr]

@nigoroll thank you, and thanks for that explanation. I really clearly understand what you're thinking and I agree... we don't want to reinvent wheels, especially as they'll end up buggy (see my several additional commits!). This sounds like a joyful change to Varnish but well outside of my wheel-house. Can you point me to a spot to look into what supporting this kind of effort looks like?

[nigoroll]

@karlvr I am sorry for having overlooked your question

can you point me to a spot to look into what supporting this kind of effort looks like?

I am not sure if you mean by throwing money at it or technically. Regarding the former, I added the respective link, regarding the latter, my broad ideas are the following:

• Define director methods to add and remove backends.
• Add a VMOD method to dynamic to set another backend as the "loadbalancing" implementation
• If this is set, dynamic calls the add/remove backend methods of the "loadbalancer" when it creates/removes backends

One open question is what we do for per-backend parameters. Take, as an example, shard with its ident, rampup and weight per-backend parameters. How would we bake these into an API? A simple way would be a string for the director to parse. But also how would we tell dynamic which parameters to set for which backend? We really would not want for dynamic to call into VCL code, or would we? I mean, actually, we could leave the add and remove operations entirely to VCL, but that would mean now a director's code would execute VCL ... 👿 🙈

sub vcl_init {
  new shard = directors.shard();
}

# called when dyn creates or removes backends
sub dyn_add_shard_mem {
  shard.add_backend(dyn.new_backend(), weigt=std.random(0,100));
  shard.reconfigure();
}
sub dyn_del_shard_mem {
  shard.remove_backend(dyn.dying_backend());
  shard.reconfigure();
}

sub vcl_init {
  new dyn = dynamic.director(add_backend=dyn_add_shard_mem, remove_backend=dyn_del_shard_mem);
}

hmmm...