Yes, you can. It might be a bit of work, though. See also: IPv6 WireGuard Peering · The Fly Blog (esp, the part about RDS connectors).
If you desire fancy routing stuff, Kurt recommends tailscale: WireGuard peering without proxy - #2 by kurt | Edit wireguard allowed ips of an app - #2 by kurt
flyctl proxy is another handy little command, if you have the liberty to install flyctl on the remote host: wireguard tunnels from userland - #2 by kurt
Prior art: Can I use fly + WireGuard to expose an internal web interface?