Fixed it.
COPY --chown=nextjs:nodejs --from=builder /app/ ./
Turns it out with some permission issue indeed.