DigitalOcean DNS Resolvers
Right now, DigitalOcean uses Google Public DNS resolvers 8.8.8.8 and 8.8.4.4. While this is great for general purpose DNS resolution, their rate limiting makes it extremely difficult for us to use Digital Ocean to run our software. It would be great if Digital Ocean could provide DNS resolvers to its customers that do not impose significant rate limiting themselves and are not backed by upstream DNS servers that impose rate limiting as well.
We operate the largest Minecraft server list, Minestatus. We built custom software that queries Minecraft servers for statistics (players online, MOTD, etc.) and in order to do this, the address of the servers must be resolved. DNS resolution rate limiting is the largest hurdle for us, as we must update over 34,000 servers in (hopefully) 10 minutes. We could get around the problem by using more droplets and slow down the query rate of each individual server, but that isn't very cost effective for us.
-
Guest
- Sep 11 2018
- Will not implement
Related ideas
What about the DNS resolvers from DIGITALOCEAN? This already done. Do you know that?
67.207.67.3
67.207.67.2
Attachments Open full size
You could just install BIND or any other DNS server on one of your droplets. Enable DNS caching to save on the amount of bandwidth that you use if that is an option for the server software you decide to use.
Then all you have to do is set your droplets up to use your own DNS server rather than Google's public DNS servers. You can (and should) set your server up so that it only responds to requests from your own droplets, and you can then choose not to limit the number of lookup requests your Minecraft server list software is allowed to make.
Of course, it is possible that there could be some advantage in Digital Ocean setting up their own DNS servers for use by the droplets in the local datacenter, if just in the bandwidth savings thanks to caching, but if they don't decide to set up DNS servers for use by the droplets in each datacenter, this would be a way for you to get around the limits placed on you by Google.
Attachments Open full size
In fact you can go even simpler.
Just install your favourite resolver (bind, unbound, pdns-resover, etc) on each droplet, have it listen(2) to 127.0.0.1:53 and put »nameserver 127.0.0.1« as the first line in resolv.conf.
(The debian and ubuntu packages for unbound, as one example, default to listen(2)ing only on 127.0.0.1:53.)
Even a 512 is large enough to run its own caching, resolving (and preferably verifying) name server.
Attachments Open full size
Unfortunately at this time we aren't looking into this feature but you can setup two small droplets and run DNS services and it should resolve the issue you are having with requests since a single DNS server can handle quite a significant load.
Thanks
Attachments Open full size