Hello
I have a memcached app and periodically the app crashed and I don’t have any details. Could you please check the app and tell me what’s going on sxt-dev-memcached ?
While you wait for someone from Fly, some possibly helpful commands which may shed some light:
fly status
… which will show the current status, but more importantly, a list of the VMs the app uses. The first column having an ID. Like abcde1234. Armed with that ID:
fly vm status THAT-ID-HERE
… will show a bit more detail about it, like recent events and checks. May show the cause of any issue.
Failing that, you could SSH in using
fly ssh console
… and then look for any logs. With Linux those are generally in /var/log e.g
tail -f /var/log/syslog
tail -f /var/log/messages
… which, if they are available, will show a whole load of technical stuff, but buried within there may be some revealing message. Like out of memory errors, or something.
1 Like
In addition to what @greg said, fly status --all will show you VMs that have failed. Look for any in a failed state or with restarts, then fly vm status <id> should show you exactly what happened.
Is that the only vm fly status --all shows? That doesn’t look like it crashed, it seems like it’s been running since January just fine?
Try running fly dig aaaa sxt-dev-memcached.internal -a sxt-dev-memcached. If there’s only one IP address listed, that hang might indicated a problem. fly vm stop b5eb7c30 might help in this case.
can’t connect to lhr and ord regions but syd works as properly
when I try to see logs I don’t get any details - empty results
Hmm … well as @kurt says, if the vm has been running since January, that suggests the vm itself should have a record of what caused memcache to crash, likely within /var/log.
Not sure why you can’t connect (see @kurt 's reply to investigate that) however for the vm you say you can connect to, is there anything in /var/log (or equivalent)? e.g a /var/log/memcached.log?
cd /var/log
ls -l
?
Hmm … nothing stands out there as being wrong, so not sure what to suggest. 
The only other thought could be to look in the Fly dashboard/metrics so see the app’s memory usage, as memcached stores data in RAM. So check you aren’t hitting any vm memory limits which could cause a crash. Seems unlikely, but not sure what else to suggest to retrospectively debug what may have happened.