gRPC service behind Envoy proxy

Hi,

I’m currently trying to set up a gRPC service behind an Envoy proxy.

Here’s my config:

envoy.yaml:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 8080 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                codec_type: auto
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ['*']
                      routes:
                        - match: { prefix: '/' }
                          route:
                            cluster: grpc_service
                            timeout: 0s
                            max_stream_duration:
                              grpc_timeout_header_max: 0s
                      cors:
                        allow_origin_string_match:
                          - prefix: '*'
                        allow_methods: GET, PUT, DELETE, POST, OPTIONS
                        allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
                        max_age: '1728000'
                        expose_headers: grpc-status,grpc-message
                http_filters:
                  - name: envoy.filters.http.grpc_web
                  - name: envoy.filters.http.cors
                  - name: envoy.filters.http.router
  clusters:
    - name: grpc_service
      connect_timeout: 1s
      type: logical_dns
      http2_protocol_options: {}
      lb_policy: round_robin
      load_assignment:
        cluster_name: cluster_0
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: my-grpc-service.internal
                      port_value: 8080

fly.toml (Envoy):

app = "my-envoy-proxy"

[[services]]
  internal_port = 8080
  protocol = "tcp"

  [[services.ports]]
    handlers = ["tls"]
    port = "443"

  [services.ports.tls_options]
    alpn = ["h2"]

fly.toml (Service):

app = "my-grpc-service"

# There's nothing else in here, the app is listening on `0.0.0.0:8080`

I can’t connect to my service via the proxy.

This setup works locally in docker-compose.

If I expose the service itself directly I can successfully call service methods on it with grpcurl but not through the Envoy proxy. Envoy is seemingly not able to connect to the service.

Is there a problem with my setup e.g. due to TLS termination via fly.io’s built-in proxy > envoy > service.

The apps (envoy & my service) are deployed in the same organization and DNS resolution works (envoy also finds it and properly resolves the IPv6 address of the service).

I found the gRPC example you guys built a while ago (using grpcwebproxy instead of Envoy). Is there any chance you could also provide such an example using Envoy <3? :slight_smile:

Can you enable logging on both services to stdout and see if anything shows up using fly logs?

Ok so in order to get it to work I had to define the bind address as ipv6 wildcard (::) in my services instead of 0.0.0.0. I also went ahead and did the same in the envoy proxy and also enabled ipv4 compat there.

      address:
        socket_address: { address: '::', port_value: 8080, ipv4_compat: true }
              - endpoint:
                  address:
                    socket_address:
                      address: my-grpc-service.internal
                      port_value: 8080
                      ipv4_compat: true