Python DNS resolver and original TTL
Question:
I need to get original TTL for dns record on each query.
DNS resolver shows original ttl only at first query. It shows time to reset cache on each next query.
>>> answer = dns.resolver.query('www.stackoverflow.com')
>>> print answer.rrset.ttl
300
>>> answer = dns.resolver.query('www.stackoverflow.com')
>>> print answer.rrset.ttl
292
How can I get original TTL at any query?
Answers:
You can only get the original TTL by directly querying the authoritative server. This is not Python-specific.
- Find out what the set of authoritative nameservers is by querying for
NS records for the desired name. If you find no NS records for the name then remove the first label and query again (query the parent domain). Recursively repeat until you get some NS records.
- Once you have NS records, query those nameservers directly for the originally requested name. In case one or more of these nameservers doesn’t respond, query the next one in the list.
This is basically equivalent to doing part of the job of a recursive resolver.
To answer the question as stated in the original post: Yes, Python can indeed be used to obtain the original TTL of a domain, easily so if done via DoH, or slightly less so via plain old DNS (UDP/53).
A recursive resolver is not needed if the TTL provided by the network resolver is sufficient for your use case. In most cases, it is.
Here is a code snippet that obtains DNS records over DoH taken from a real product I wrote:
import urllib.request
import urllib.parse
import json
domain = "example.com"
doh_ep = "https://1.1.1.1/dns-query?"
params = urllib.parse.urlencode({'name':(domain.encode("idna")).decode(), "type": "A"}, quote_via=urllib.parse.quote_plus)
req = urllib.request.Request(doh_ep+params, headers = { "accept" : "application/dns-json" })
res = urllib.request.urlopen(req)
dns = json.loads(res.read().decode('utf-8'))
for answer in dns["Answer"]:
print("{domain} {ttl} {ipv4}".format( domain = answer["name"], ttl = answer["TTL"], ipv4 = answer["data"]))
This code prints a series of line containing the domain, it’s original TTL value, and it’s associated address.
Footnotes:
- This code makes use of standard library modules only. No third party modules or dependencies are required.
- This code is able to handle IDNs correctly if the DoH provider supports it. I used cloudflare’s 1.1.1.1 service in this example which does support IDNs. I’m not aware of any DoH provider that doesn’t.
- This code only obtains TTL values for IPv4 (
A) records. You can change the record type to any other type: AAAA for IPv6, NS for nameservers, …etc.
- To the best of my knowledge, Google’s DoH API is compatible with cloudflare’s. You should be able to switch the URI in
doh_ep with https://dns.google/resolve? or any other DoH endpoint of your choosing as long as it implements the DNS-JSON API.
- Using plain DNS is similarly straight forward but a bit more cumbersome to implement: Create a UDP socket, write a DNS query question according to the relavent RFC, then parse the response.
I need to get original TTL for dns record on each query.
DNS resolver shows original ttl only at first query. It shows time to reset cache on each next query.
>>> answer = dns.resolver.query('www.stackoverflow.com')
>>> print answer.rrset.ttl
300
>>> answer = dns.resolver.query('www.stackoverflow.com')
>>> print answer.rrset.ttl
292
How can I get original TTL at any query?
You can only get the original TTL by directly querying the authoritative server. This is not Python-specific.
- Find out what the set of authoritative nameservers is by querying for
NSrecords for the desired name. If you find no NS records for the name then remove the first label and query again (query the parent domain). Recursively repeat until you get some NS records. - Once you have NS records, query those nameservers directly for the originally requested name. In case one or more of these nameservers doesn’t respond, query the next one in the list.
This is basically equivalent to doing part of the job of a recursive resolver.
To answer the question as stated in the original post: Yes, Python can indeed be used to obtain the original TTL of a domain, easily so if done via DoH, or slightly less so via plain old DNS (UDP/53).
A recursive resolver is not needed if the TTL provided by the network resolver is sufficient for your use case. In most cases, it is.
Here is a code snippet that obtains DNS records over DoH taken from a real product I wrote:
import urllib.request
import urllib.parse
import json
domain = "example.com"
doh_ep = "https://1.1.1.1/dns-query?"
params = urllib.parse.urlencode({'name':(domain.encode("idna")).decode(), "type": "A"}, quote_via=urllib.parse.quote_plus)
req = urllib.request.Request(doh_ep+params, headers = { "accept" : "application/dns-json" })
res = urllib.request.urlopen(req)
dns = json.loads(res.read().decode('utf-8'))
for answer in dns["Answer"]:
print("{domain} {ttl} {ipv4}".format( domain = answer["name"], ttl = answer["TTL"], ipv4 = answer["data"]))
This code prints a series of line containing the domain, it’s original TTL value, and it’s associated address.
Footnotes:
- This code makes use of standard library modules only. No third party modules or dependencies are required.
- This code is able to handle IDNs correctly if the DoH provider supports it. I used cloudflare’s 1.1.1.1 service in this example which does support IDNs. I’m not aware of any DoH provider that doesn’t.
- This code only obtains TTL values for IPv4 (
A) records. You can change the record type to any other type:AAAAfor IPv6,NSfor nameservers, …etc. - To the best of my knowledge, Google’s DoH API is compatible with cloudflare’s. You should be able to switch the URI in
doh_epwithhttps://dns.google/resolve?or any other DoH endpoint of your choosing as long as it implements the DNS-JSON API. - Using plain DNS is similarly straight forward but a bit more cumbersome to implement: Create a UDP socket, write a DNS query question according to the relavent RFC, then parse the response.