Re: Requirements for new dns backend, factoring considerations

[Laurent Bercot <ska-dietlibc@skarnet.org>]

  Hi Rich,
  Great work, as usual.


> The problem however is implementing this on top of something that
> looks like res_send. Even if not for search paths, res_search and
> res_query need parallel A and AAAA queries, whereas res_send has no
> means to request both. We could imagine implementing res_send on top
> of a hypothetical "res_multisend" with a count of 1. While this would
> work, it's not a very friendly interface for implementing res_search
> or res_query since they would have to provide a large number of
> pre-generated query packets (6 search domains * 2 RR types * up to 280
> bytes per query packet = 3360 bytes of stack usage) despite it being
> trivial to generate the Nth packet in just 280 bytes of storage. The
> storage requirements for storing all the results are even worse
> (6*2*512 = 6144) compared to what's actually needed (2*512 = 1024).

  I actually never thought about that. Since s6-dns stores answers in the
heap, it doesn't have to pre-allocate storage for them, so it happily
sends everything in parallel.


> The alternative I see is some sort of "res_multisend" that uses a
> callback to let the caller generate the Nth packet and a callback to
> notify the caller of replies. Then res_send could be implemented with
> callbacks that just feed in and save out the single query/response.
> And res_search would generate all the query packets but only save the
> "current best match" for each address family.

  That sounds reasonable.


> As another alternative, we could drop the goal of doing search
> suffixes in parallel.

  The best choice depends on your timeout values and retry policy.

  As I already said: this is network, so it's going to suck, and this is
DNS, so it's going to suck even more. If a server does not answer, which
will happen, and you have to wait for 20 seconds before considering it a
failure, then serial search is just going to make users mad. Multiple
server failure is a common occurrence (client-side network issue, for
instance); I'd rather have a clue before 2 minutes, and most users will
^C anything that takes longer than 30 seconds - and ^C doesn't bring any
debugging information.

  So if your timeouts are very short, sure, serial search will work. But
if they are not, I'd say parallel search or drop search entirely. To me,
even 6k of stack for every getaddrinfo() invocation is better than
potentially huge network latency, especially since the getaddrinfo
(or even the res_*) API provides no way of setting bounds to the waiting
time.

  This is the 21st century, and even though I hate being wasteful as much
as you do, memory is definitely cheaper than users' time. And memory
usage - especially stack memory - is so much more trackable than network
latency.


> negligible, I think) and make a trivial res_multisend that does N(=2)
> queries in parallel using packets provided by the caller.

  That's what the synchronous s6-dns resolver does, and I think that's
the least amount of parallelism that's still acceptable in a v4+v6 world.

-- 
  Laurent