Lowend anycast-as-a-service or geo+anycast+API DNS

tetech · July 2020

Interested in hearing ideas about the following:

DNS service with anycast, with GeoDNS, updateable via REST API, around $5/mo mark.

Considered but not suitable:

I do/will not use Cloudflare for anything.
I've been using NS1's free plan and it has been great for several years, but they limit it to one filter chain (so can't do both IPv4 and IPv6) and 0.5M queries. I'm starting to outgrow that, and their lowest paid plan is $45/mo.
Rage4 GeoDNS plans start 20 Euro/month; too expensive.
Route53 is OK. Health check costs start to add up for multiple endpoints; API is painful; and I prefer to support smaller businesses. But it is within my target price range.
Constellix is probably closest to what I'm looking for at the moment, although more costly than Route 53 and API still isn't fantastic (at least vs NS1).

Very happy to run my own DNS, so have also considered anycast-as-a-service, however:

BuyVM's POPs are limited, and if one VM goes down there's no way of removing that endpoint, meaning users in that region get pointed to a black hole (unless I'm missing something). @Francisco
Rage4 has good POPs and (as I understand it) the ability to add/remove nodes on the fly. But their plans are overkill for me. I don't need 100GB/mo and 25 Euro is way too much. @gbshouse
Ditto for @wdmg. 2TB for $30/mo is way in excess of what I'd need.

So maybe anycast-as-a-service just isn't for me, which is fair enough (not looking for a free ride). In that case I think my options are Route53 or Constellix, but wanted to see whether I'm missing anything.

sgheghele · July 2020

@tetech

0.5M queries. I'm starting to outgrow that,

Any way to assess this beforehand with Cloudflare?

Anyway, there is also ClouDNS GeoDNS for $9.95: https://www.cloudns.net/geodns/

tetech · July 2020

@sgheghele said:
@tetech

0.5M queries. I'm starting to outgrow that,

Any way to assess this beforehand with Cloudflare?

I don't/won't use Cloudflre. Anyway, NS1 billing is for an average of 0.7M per month, so the assessment is already done. This means I get charged overage by NS1 ($8/M), but NS1 is still quite OK price-wise. But the issue is not query volume. The main issue with NS1 is that they limit the free plan to one filter chain, and there is no way to pay for an extra one except upgrade to a $45/month plan.

Anyway, there is also ClouDNS GeoDNS for $9.95: https://www.cloudns.net/geodns/

Thanks for the suggestion, but that is 2x $5. Constellix would be cheaper/better for me.

sgheghele · July 2020

@tetech said: Thanks for the suggestion, but that is 2x $5. Constellix would be cheaper/better for me.

Isn't Constellix $5.00 for the first domain + $5.00 for GeoDNS add-on + costs for queries? See https://constellix.com/pricing and click on the various "see pricing example" buttons for examples.

They also write "Please note, GTD Domain pricing is an additional cost to the regular domain pricing. That means, if you have one domain that uses GTD then you will be charged $5 for the cost of the domain plus and additional $5 since GTD is enabled for that domain."

ClouDNS gets cheaper if you pay for 6 or 12 months.

Edit: an equivalent GTD domain with 100M queries would cost $49/month with Constellix.

tetech · July 2020

@sgheghele said:

@tetech said: Thanks for the suggestion, but that is 2x $5. Constellix would be cheaper/better for me.

Isn't Constellix $5.00 for the first domain + $5.00 for GeoDNS add-on + costs for queries? See https://constellix.com/pricing and click on the various "see pricing example" buttons for examples.

They also write "Please note, GTD Domain pricing is an additional cost to the regular domain pricing. That means, if you have one domain that uses GTD then you will be charged $5 for the cost of the domain plus and additional $5 since GTD is enabled for that domain."

ClouDNS gets cheaper if you pay for 6 or 12 months.

For Constellix, GTD is their "global traffic director" which is a more advanced thing including RUM - not the same as GeoDNS. The basic pricing is $5 for a domain + $0.06 for each filter rule + $0.60/M geoproximity queries + any health checks. Their API is good enough that I'd probably integrate Hetrix monitoring (OK to do, though not as easy as NS1), so cost is ~$5.48/month and adding more filter rules is cheap.

For Route 53, it is more like $0.50 for the domain + $0.49 for geoproximity queries, or $0.99. Sounds good, but a real pain to integrate Hetrix health checks, so if I have 6 POPs and AWS charge $0.75 each it works out to $4.50 just for the health checks and I might as well go with Constellix.

In either case, I'm in the $5 ballpark.

Of course, if there was an anycast-as-a-service option, I'd pay a bit more since it lets me do more things (like a mini-CDN).

sgheghele · July 2020

@tetech said:

@sgheghele said:

@tetech said: Thanks for the suggestion, but that is 2x $5. Constellix would be cheaper/better for me.

Isn't Constellix $5.00 for the first domain + $5.00 for GeoDNS add-on + costs for queries? See https://constellix.com/pricing and click on the various "see pricing example" buttons for examples.

They also write "Please note, GTD Domain pricing is an additional cost to the regular domain pricing. That means, if you have one domain that uses GTD then you will be charged $5 for the cost of the domain plus and additional $5 since GTD is enabled for that domain."

ClouDNS gets cheaper if you pay for 6 or 12 months.

For Constellix, GTD is their "global traffic director" which is a more advanced thing including RUM - not the same as GeoDNS. The basic pricing is $5 for a domain + $0.06 for each filter rule + $0.60/M geoproximity queries + any health checks. Their API is good enough that I'd probably integrate Hetrix monitoring (OK to do, though not as easy as NS1), so cost is ~$5.48/month and adding more filter rules is cheap.

For Route 53, it is more like $0.50 for the domain + $0.49 for geoproximity queries, or $0.99. Sounds good, but a real pain to integrate Hetrix health checks, so if I have 6 POPs and AWS charge $0.75 each it works out to $4.50 just for the health checks and I might as well go with Constellix.

In either case, I'm in the $5 ballpark.

Of course, if there was an anycast-as-a-service option, I'd pay a bit more since it lets me do more things (like a mini-CDN).

Thanks for the explanation regarding GTD!

If Constellix really is at about $5 per month, I do not see how you might go cheaper than that. I also did some research some weeks ago and found nothing cheaper than route 53, for my use case. Constellix might be better or just about there.

tetech · July 2020

@sgheghele said: If Constellix really is at about $5 per month, I do not see how you might go cheaper than that. I also did some research some weeks ago and found nothing cheaper than route 53, for my use case. Constellix might be better or just about there.

You might be right that nothing else is competitive! Good to ask though.

Especially for the anycast-as-a-service option, I am not sure if I missed other providers. For anycast, if someone did a lowend annual plan (e.g. 200GB for the year) that could be very interesting.

I think when you compare Route 53 and Constellix the real question is health checks and failover. If you are happy to integrate Hetrix (or Uptime Robot or whatever) and you have a low query volume, then your % saving can be high. There is not much difference between Route 53 and Constellix on extra domains and on cost per query.

willie · July 2020

@tetech said: BuyVM's POPs are limited, and if one VM goes down there's no way of removing that endpoint, meaning users in that region get pointed to a black hole (unless I'm missing something). @Francisco

I thought the way anycast works is that if an endpoint goes down, the route to it fails (how?) and the next lowest cost route gets the query.

DO has free DNS for subscribers but it's not clear if they have anycast: https://www.digitalocean.com/docs/networking/dns/

OVH has anycast dns for 0.99€/m per domain for domains you register with them: I don't know about query limits or API. Also not that many endpoints, and expensive if you want to serve a lot of domains. https://www.ovh.ie/domains/dns-anycast/

Vultr has anycast dns with an api. Not clear about pricing or limitations. https://www.vultr.com/docs/introduction-to-vultr-dns

umi · July 2020

Please explain in layman words to me what is an AWS Route53's traffic policy ($50 per each) and what it does. Thanks! Do you need it enabled to get geo-based DNS working?

wdmg · July 2020

Ditto for @wdmg. 2TB for $30/mo is way in excess of what I'd need.

If you need something <100GB, we’re happy to provide, but we’d want yearly payment. Feel free to reach out! Always happy to do custom plans.

tetech · July 2020

@willie said:

@tetech said: BuyVM's POPs are limited, and if one VM goes down there's no way of removing that endpoint, meaning users in that region get pointed to a black hole (unless I'm missing something). @Francisco

I thought the way anycast works is that if an endpoint goes down, the route to it fails (how?) and the next lowest cost route gets the query.

That is how anycast works. The issue is the granularity. I'm happy to be corrected if wrong, but the way BuyVM implements it is on a datacenter granularity. In other words, if the whole BuyVM location goes down, then anycast works as you describe and all their IPs (en masse) get routed to the next-nearest location. But there is no monitoring of an individual VPS, therefore no knowledge if (e.g.) HTTP server stops running, so traffic still gets blindly routed to it.

They do have a "failover IP" concept but from what I can tell this means within the same datacenter. So effectively you start to need 2x VPS per datacenter = 2x3x$2 = $12/month and then it is only 3 POPs. This is great if you are building a CDN focusing on North America & Europe, but less good if you need better coverage of the world (specifically, I need an Australia POP).

DO has free DNS for subscribers but it's not clear if they have anycast: https://www.digitalocean.com/docs/networking/dns/

OVH has anycast dns for 0.99€/m per domain for domains you register with them: I don't know about query limits or API. Also not that many endpoints, and expensive if you want to serve a lot of domains. https://www.ovh.ie/domains/dns-anycast/

Vultr has anycast dns with an api. Not clear about pricing or limitations. https://www.vultr.com/docs/introduction-to-vultr-dns

None of these are GeoDNS. Ditto Linode.

tetech · July 2020

@umi said:
Please explain in layman words to me what is an AWS Route53's traffic policy ($50 per each) and what it does. Thanks! Do you need it enabled to get geo-based DNS working?

You do NOT need a traffic policy to get GeoDNS working with Route 53. Traffic policy is designed for more complex routing rules. Example: if you want GeoDNS that returns multiple values (rather than the nearest IP) then you start to need traffic policy. If you want to balance load across regions, you can 'bias' the geo result (so for a New York-centric service, a positive bias might mean your region includes Chicago/Dallas but a negative bias means it might shrink to not even include Washington DC). Traffic policy is the way you achieve complex stuff like this. But for "geolocation" that returns a single IP based on user's region, you do not need traffic policy.

Traffic policy is pretty weak for the price (IMHO) because you can make more powerful rules using Constellix GTD or even regular NS1 filter chains for a fraction of the cost.

tetech · July 2020

@wdmg said:

Ditto for @wdmg. 2TB for $30/mo is way in excess of what I'd need.

If you need something <100GB, we’re happy to provide, but we’d want yearly payment. Feel free to reach out! Always happy to do custom plans.

Thanks! I have sent you a message. Yearly payment would be fine for a lowend plan. How does it work with your system if one of my VPSes goes down - do you have any sort of public docs or setup guide?

tetech · July 2020

Another option I forgot is Oracle. Their "cloud DNS" has no per-domain charge, $4/M for "traffic director" queries (so 0.7M = $2.80) and their 30 sec health checks are a comparatively reasonable $0.30/month. So 2.80 + 6x0.30 = $4.60 (like AWS their API is a nightmare and it wouldn't be fun to integrate Hetrix).

I guess this model works quite well for low traffic but doesn't scale up very well compared to Constellix or Route 53. Plus their "traffic director" rules are a bit limited.

However, I've been using this for a few domains and found the response time to be very good.

willie · July 2020

I didn't understand what GeoDNS was until reading about it just now. I just thought it meant setting up anycast routing so that client queries would tend to go to a nearby server. It instead means the DNS server itself uses a GeoIP database to guess where the client is, and serves an address near the client location. Cool, now I know another thing to look for. But yeah Vultr and OVH don't mention it.

Also, needing Australia POP complicates things a little for you.

tetech · July 2020

@willie said:
I didn't understand what GeoDNS was until reading about it just now. I just thought it meant setting up anycast routing so that client queries would tend to go to a nearby server. It instead means the DNS server itself uses a GeoIP database to guess where the client is, and serves an address near the client location. Cool, now I know another thing to look for. But yeah Vultr and OVH don't mention it.

Yeah, "anycast DNS" refers to the property of the DNS server itself, where a single IP takes you to the nearest DNS server to improve latency. If I've got users in Australia then going to the wrong DNS can add 300 ms to the lookup (if not cached etc.)

GeoDNS is more about the answers that the DNS server returns, i.e. returning a different answer based on the requester location.

Of course, if your web server is also anycast then you'd always return the anycast IP address - the answer doesn't vary by location so you don't really need GeoDNS at all. But there's at least two issues with this "full anycast" approach. First is that you still need some way of managing failover at one of your anycast POPs - typically that should be done at the network level but if that's not possible (e.g. BuyVM) then it needs to be done at the DNS. Second is that anycast bandwidth tends to be expensive. So OK for DNS and stuff like that, but less OK for web sites with video. For the average user, an anycast website is total overkill and simply using GeoDNS to return the nearest (non-anycast) IP is perfectly fine.

willie · July 2020

Wouldn't you use a CDN for the video? Or is it for realtime video chat or sth like that. Anyway it sounds like australia may be a sticking point. I wonder why Vultr/etc. don't support GeoDNS. It's not that complicated to add on, I'd hope.

tetech · July 2020

@willie said:
Wouldn't you use a CDN for the video? Or is it for realtime video chat or sth like that.

For on-demand video, may not even need geolocation. I guess my point was that when most people start out with web hosting they just put everything on one web server including large static downloads (software, video, etc.) and that could get expensive if you blindly moved it onto anycast.

Anyway it sounds like australia may be a sticking point.

At least for DNS, it isn't really an issue. I guess most providers who offer anycast and GeoDNS will have POPs in Australia. Certainly all of Route 53, Constellix, Oracle etc. do. I'd guess that someone serious about offering anycast-as-a-service would also have POPs in Australia, but may only offer them on some plans.

I wonder why Vultr/etc. don't support GeoDNS. It's not that complicated to add on, I'd hope.

Requires a bit of work, in terms of needing a UI, needing the architecture for updating rules across POPs, and effort involved in the rule application itself. My impression is that Vultr etc. are either doing a white-label of someone else's service or are offering it due to competitive pressure (i.e. competitor X includes it so they better too) but don't see it as a product on its own.

aveline · July 2020

We may have what you need. But the only available filters in production are country, continent and ISP for now.

Our AnyCast DNS is tuned properly and have a better latency ins some locations, you can check 188.244.98.1 or our domain misaka.io for testing.

Have been too busy working on other projects recently :-(

tetech · July 2020

@aveline said:
We may have what you need. But the only available filters in production are country, continent and ISP for now.

Our AnyCast DNS is tuned properly and have a better latency ins some locations, you can check 188.244.98.1 or our domain misaka.io for testing.

Have been too busy working on other projects recently :-(

Thanks for the link! Good to see more providers with competitive offerings, and nice list of POPs. Probably I need sub-country for US (60% of my traffic is US and I have POPs in West/Central/East, so not necessarily state but at least region). Any timeframe for adding that? Would also be good to get some API info and details about the company - I'm a bit wary of "opaque" companies who I've never heard of and don't even give an address on their web page. Generally though pricing seems like it might be similar to Constellix.

aveline · July 2020

@tetech said:

@aveline said:
We may have what you need. But the only available filters in production are country, continent and ISP for now.

Our AnyCast DNS is tuned properly and have a better latency ins some locations, you can check 188.244.98.1 or our domain misaka.io for testing.

Have been too busy working on other projects recently :-(

Thanks for the link! Good to see more providers with competitive offerings, and nice list of POPs. Probably I need sub-country for US (60% of my traffic is US and I have POPs in West/Central/East, so not necessarily state but at least region). Any timeframe for adding that? Would also be good to get some API info and details about the company - I'm a bit wary of "opaque" companies who I've never heard of and don't even give an address on their web page. Generally though pricing seems like it might be similar to Constellix.

We do have a filter based on ISO3166-2 but wasn't tested yet, should have time to do it next month.

API document is working in progress but we're a API-first service, our web console is also built on the API ... so don't worry, we don't want to use bad API to annoy ourselves.

tetech · July 2020

@tetech said: Another option I forgot is Oracle. Their "cloud DNS" has no per-domain charge, $4/M for "traffic director" queries (so 0.7M = $2.80) and their 30 sec health checks are a comparatively reasonable $0.30/month. So 2.80 + 6x0.30 = $4.60 (like AWS their API is a nightmare and it wouldn't be fun to integrate Hetrix).

I put in an hour figuring out the Oracle DNS API and integrating Hetrix. Not fun, but better than I thought it would be. Price-wise it is thus a flat $4/M for "traffic director" (i.e. GeoDNS) queries.

This might be a short-term solution, but the cost doesn't scale up very well, so I'm still contemplating what to do in the longer term.

tetech · July 2020

@tetech said: For Route 53, it is more like $0.50 for the domain + $0.49 for geoproximity queries, or $0.99. Sounds good, but a real pain to integrate Hetrix health checks, so if I have 6 POPs and AWS charge $0.75 each it works out to $4.50 just for the health checks and I might as well go with Constellix.

Today I integrated Hetrix with Route 53. It wasn't too bad, and looks like I've managed to do it on the AWS "free forever" tier (not the "free 12 month trial"). But maybe I should give it a day for billing info to update and not be too confident on that last point.

Cost-wise, using external monitors makes Route 53 quite competitive: $0.50 for the domain + $0.70/M GeoDNS queries. That means it scales up better than Oracle or NS1. I'm considering continuing to use NS1 for my most complex filter chain and Route 53 for the rest. I'm estimating cost around $0.65/month.

tetech · July 2020

@tetech said: Today I integrated Hetrix with Route 53. It wasn't too bad, and looks like I've managed to do it on the AWS "free forever" tier (not the "free 12 month trial"). But maybe I should give it a day for billing info to update and not be too confident on that last point.

I re-checked AWS billing and seems good. By pushing the Hetrix updates to CloudFront metrics, the Route 53 health checks are considered 'AWS endpoints' and thus free of charge.

Amazon Route 53 Health-Check-AWS$0.00
First 50 Health Checks of AWS endpoints are free of charge0.477 Mo$0.00

Amazon CloudWatch$0.00
$0.00 per alarm metric month - first 10 alarm metrics0.921 Alarms$0.00
$0.00 per metric-month - first 10 metrics0.007 Metrics$0.00
$0.00 per request - first 1,000,000 requests14.000 Requests$0.00

Probably I go with the NS1 + Route 53 combination for now.

Still exploring anycast options. If Rage4 repeated the special offer and could make it annual, I'd probably try that out.

splitice · July 2020

We do mitigation on our Anycast network. Works well as Anycast as a service too.

Premium Network currently spans 9 cities over 3 continents.

tetech · July 2020

@splitice said:
We do mitigation on our Anycast network. Works well as Anycast as a service too.

Premium Network currently spans 9 cities over 3 continents.

I'm sure that's great for a lot of people! Unfortunately, lack of Australia kills it for me, and $30/month is way over my budget. But for someone who needs mitigation it would be good.

Neoon · July 2020

@aveline said:
We may have what you need. But the only available filters in production are country, continent and ISP for now.

Our AnyCast DNS is tuned properly and have a better latency ins some locations, you can check 188.244.98.1 or our domain misaka.io for testing.

Have been too busy working on other projects recently :-(

"Smart Domains"

aveline · July 2020

@Neoon said:

@aveline said:
We may have what you need. But the only available filters in production are country, continent and ISP for now.

Our AnyCast DNS is tuned properly and have a better latency ins some locations, you can check 188.244.98.1 or our domain misaka.io for testing.

Have been too busy working on other projects recently :-(

"Smart Domains"

It's just another name of GeoDNS enabled domains, like constellix's GTD enabled domains. Doing checks consumes more CPU time on edge nodes so we charge it separately.
I have no idea how to name it :-(

splitice · July 2020

@tetech said:

@splitice said:
We do mitigation on our Anycast network. Works well as Anycast as a service too.

Premium Network currently spans 9 cities over 3 continents.

I'm sure that's great for a lot of people! Unfortunately, lack of Australia kills it for me, and $30/month is way over my budget. But for someone who needs mitigation it would be good.

Good to know. We are an Australian company ourselves. Eventually an Australian PoP will be on the cards. Currently however we don't rank high enough in the cost benefit Analysis for new PoPs (at-least without a sponsoring customer). Unfortunately we have too small of a population, high cost and a minimal rate of domestic DDoS.

Unfortunately you arent going to get Anycast in premium locations without spending reasonable money. We are at the cheaper end of the Anycast as a Service range most of the commercial operators have at-least one more zero in their minimum monthly cost.

You will likely have to settle for GeoDNS in that price range. Rage4 is what we use but we get them a bit cheaper than their commercial rate as we are a reseller (we provide our customers with 2 Rage4 zones with their services). I've found their GeoDNS routing to be about as good as it gets (the kind of mistakes it makes are the same ones made by other GeoDNS providers). Their closest first mode however seems to be better. I believe it may use the serving node location, therefore as long as the Anycast network is balanced it would be higher accuracy.

tetech · July 2020

@splitice said:

@tetech said:

@splitice said:
We do mitigation on our Anycast network. Works well as Anycast as a service too.

Premium Network currently spans 9 cities over 3 continents.

I'm sure that's great for a lot of people! Unfortunately, lack of Australia kills it for me, and $30/month is way over my budget. But for someone who needs mitigation it would be good.

Good to know. We are an Australian company ourselves. Eventually an Australian PoP will be on the cards. Currently however we don't rank high enough in the cost benefit Analysis for new PoPs (at-least without a sponsoring customer). Unfortunately we have too small of a population, high cost and a minimal rate of domestic DDoS.

Unfortunately you arent going to get Anycast in premium locations without spending reasonable money. We are at the cheaper end of the Anycast as a Service range most of the commercial operators have at-least one more zero in their minimum monthly cost.

You will likely have to settle for GeoDNS in that price range. Rage4 is what we use but we get them a bit cheaper than their commercial rate as we are a reseller (we provide our customers with 2 Rage4 zones with their services). I've found their GeoDNS routing to be about as good as it gets (the kind of mistakes it makes are the same ones made by other GeoDNS providers). Their closest first mode however seems to be better. I believe it may use the serving node location, therefore as long as the Anycast network is balanced it would be higher accuracy.

All of that makes perfect sense. I'm doing the same type of cost-benefit on a much more lowend scale. For reference, this was my starting point: https://talk.lowendspirit.com/discussion/comment/8981.

To step through my thought process, if I'm going to spend around $5-8 per month on a decent GeoDNS then I'd be happy to spend $10/month and run it myself on Anycast, since this could give us an incremental benefit in some other areas. The above-referenced Rage4 offer for €10/month is getting pretty close (maybe paying annually would get us there?), and I didn't know if there were other similar providers that might come in slightly lower (considering I don't need Africa/South America).

However, if the above deal (no longer available) is as good as it will ever get and the currently-available options are in the €20/month range, then cost-benefit says that I'd be better off paying for GeoDNS, some BunnyCDN edge storage, and a couple of extra VPS, at least for now. That's perfectly OK as an answer, this thread was really about surveying the options.

splitice · July 2020

The LES offer Piotr has made is many multiples below industry standards for pricing. If it's what you need (technology, locations, performance, support) I would take it. The closest we can even get to that is our current sale ($14) so I know that's a tight margin offer from him.

You arent going to build a comparable Anycast network for $10/month or even $20/month. Best case you use $5 Vultr nodes that gets you 4 PoPs. And that's before purchasing / renting IPv4 subnets & LIR/RIR membership fees (costs there are significantly higher for a single IP).

Lowend anycast-as-a-service or geo+anycast+API DNS

Comments