Wildcard *.mydomain.com certificate stuck in `Awaiting certificates` mode after remove/add

Similar issue, but non-wildcard domain. In our case the steps were:

  1. Provision cert in dashboard
  2. Wait some time
  3. No cert issued. Delete cert, goto 1