weird DNS answers

hello, I was attempting to SSH into an instance which was timing out. I also have a wireguard tunnel up, and ran a dig query for the internal hostname. The response started with stale records and had a bunch of duplicates, along with some cutoff answers:

$ dig AAAA fe-anchor-meta-anchordotdev.internal

; <<>> DiG 9.10.6 <<>> AAAA fe-anchor-meta-anchordotdev.internal
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52402
;; flags: qr rd ad; QUERY: 1, ANSWER: 39, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;fe-anchor-meta-anchordotdev.internal. IN AAAA

;; ANSWER SECTION:
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:9cc1:ea2d:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:9cc1:ea2d:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2

;; Query time: 15 msec
;; SERVER: fdaa:0:3723::3#53(fdaa:0:3723::3)
;; WHEN: Mon Nov 08 16:32:30 EST 2021
;; MSG SIZE  rcvd: 1157

After a minute or so, it corrected itself:

dig AAAA fe-anchor-meta-anchordotdev.internal

; <<>> DiG 9.10.6 <<>> AAAA fe-anchor-meta-anchordotdev.internal
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41035
;; flags: qr rd ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;fe-anchor-meta-anchordotdev.internal. IN AAAA

;; ANSWER SECTION:
fe-anchor-meta-anchordotdev.internal. 5 IN AAAA fdaa:0:3723:a7b:21e0:314d:86ae:2

;; Query time: 14 msec
;; SERVER: fdaa:0:3723::3#53(fdaa:0:3723::3)
;; WHEN: Mon Nov 08 16:40:51 EST 2021
;; MSG SIZE  rcvd: 93

This was right after I deployed the app, but more than 5 seconds (the TTL in the answers) had passed so I wouldn’t expect the stale records at the top to be in the answer.

We pushed out a fix to address this on the gateways yesterday; can you let me know if you’re still seeing weird duplicate AAAA records?

Yep, looks good now, thanks!