Errors querying metrics, 301 and 401

danthegoodman · May 23, 2023, 12:55am

Following the example at Metrics on Fly.io · Fly Docs, I get a 301 according to curl.

Adding the -L option to allow curl to follow the redirect, I then get a 401 from https://api.fly.io/prometheus/api/v1/query that I was redirected to

301:

curl -vvv https://api.fly.io/prometheus/$ORG_SLUG/api/v1/query \                                8:54PM
  --data-urlencode 'query=sum(increase(fly_edge_http_responses_count)) by (app, status)' \
  -H "Authorization: Bearer $TOKEN"

*   Trying 77.83.143.220:443...
* Connected to api.fly.io (77.83.143.220) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Client hello (1):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Server hello (2):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Unknown (8):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Certificate (11):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, CERT verify (15):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Finished (20):
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=api.fly.io
*  start date: Mar 30 11:19:17 2023 GMT
*  expire date: Jun 28 11:19:16 2023 GMT
*  subjectAltName: host "api.fly.io" matched cert's "api.fly.io"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: POST]
* h2h3 [:path: /prometheus//api/v1/query]
* h2h3 [:scheme: https]
* h2h3 [:authority: api.fly.io]
* h2h3 [user-agent: curl/7.87.0]
* h2h3 [accept: */*]
* h2h3 [authorization: Bearer ]
* h2h3 [content-length: 81]
* h2h3 [content-type: application/x-www-form-urlencoded]
* Using Stream ID: 1 (easy handle 0x7fc87f011400)
> POST /prometheus//api/v1/query HTTP/2
> Host: api.fly.io
> user-agent: curl/7.87.0
> accept: */*
> authorization: Bearer
> content-length: 81
> content-type: application/x-www-form-urlencoded
>
* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS == 32)!
< HTTP/2 301
< location: /prometheus/api/v1/query
< date: Tue, 23 May 2023 00:54:30 GMT
< server: Fly/8bd687f2 (2023-05-18)
< via: 1.1 fly.io, 2 fly.io
< fly-request-id: 01H131KE48NPZ3AB40MDG6FG4Q-lga
< content-length: 0
<
* Connection #0 to host api.fly.io left intact

With redirect:

curl -vvv -L https://api.fly.io/prometheus/$ORG_SLUG/api/v1/query \                             8:54PM
  --data-urlencode 'query=sum(increase(fly_edge_http_responses_count)) by (app, status)' \
  -H "Authorization: Bearer $TOKEN"

*   Trying 77.83.143.220:443...
* Connected to api.fly.io (77.83.143.220) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Client hello (1):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Server hello (2):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Unknown (8):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Certificate (11):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, CERT verify (15):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Finished (20):
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=api.fly.io
*  start date: Mar 30 11:19:17 2023 GMT
*  expire date: Jun 28 11:19:16 2023 GMT
*  subjectAltName: host "api.fly.io" matched cert's "api.fly.io"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: POST]
* h2h3 [:path: /prometheus//api/v1/query]
* h2h3 [:scheme: https]
* h2h3 [:authority: api.fly.io]
* h2h3 [user-agent: curl/7.87.0]
* h2h3 [accept: */*]
* h2h3 [authorization: Bearer ]
* h2h3 [content-length: 81]
* h2h3 [content-type: application/x-www-form-urlencoded]
* Using Stream ID: 1 (easy handle 0x7fb4cd011400)
> POST /prometheus//api/v1/query HTTP/2
> Host: api.fly.io
> user-agent: curl/7.87.0
> accept: */*
> authorization: Bearer
> content-length: 81
> content-type: application/x-www-form-urlencoded
>
* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS == 32)!
< HTTP/2 301
* Please rewind output before next send
< location: /prometheus/api/v1/query
< date: Tue, 23 May 2023 00:54:52 GMT
< server: Fly/8bd687f2 (2023-05-18)
< via: 1.1 fly.io, 2 fly.io
< fly-request-id: 01H131M3F950SMYQNEVGDJ4105-lga
< content-length: 0
* Keep sending data to get tossed away
<
* Connection #0 to host api.fly.io left intact
* Issue another request to this URL: 'https://api.fly.io/prometheus/api/v1/query'
* Switch from POST to GET
* Found bundle for host: 0x60000270c900 [can multiplex]
* Re-using existing connection #0 with host api.fly.io
* h2h3 [:method: GET]
* h2h3 [:path: /prometheus/api/v1/query]
* h2h3 [:scheme: https]
* h2h3 [:authority: api.fly.io]
* h2h3 [user-agent: curl/7.87.0]
* h2h3 [accept: */*]
* h2h3 [authorization: Bearer ]
* Using Stream ID: 3 (easy handle 0x7fb4cd011400)
> GET /prometheus/api/v1/query HTTP/2
> Host: api.fly.io
> user-agent: curl/7.87.0
> accept: */*
> authorization: Bearer
>
< HTTP/2 401
< date: Tue, 23 May 2023 00:54:52 GMT
< server: Fly/8bd687f2 (2023-05-18)
< via: 1.1 fly.io, 2 fly.io
< fly-request-id: 01H131M3H4H4GNF7517MWRJBHN-lga
< content-length: 0
<
* Connection #0 to host api.fly.io left intact

danthegoodman · May 23, 2023, 12:56am

FWIW it also seems like the example should be using the /query_range endpoint instead of /query based on the linked prometheus docs

wjordan · May 23, 2023, 3:10am

Hi @danthegoodman,

Your output is showing the required $ORG_SLUG shell variable is empty, so the request path is /prometheus//api/v1/query instead of /prometheus/<org-slug>/api/v1/query. Find your organization slug using flyctl orgs list and then set it by entering ORG_SLUG=org-slug into your shell, or just manually replace the $ORG_SLUG part of the url with the correct value.

As for /query (instant query) vs /query_range (range query), both types are supported, instant query just makes for a simpler example.

danthegoodman · May 23, 2023, 10:35am

Oh wow… guess I was just coding too late into the night… thanks

system · May 30, 2023, 10:36am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Authenticated requests for health_checks and Prometheus Metric Scraping? Questions / Help metrics	7	935	November 29, 2022
Error: Exceeded maxRedirects (Docker Container and fly.dev domain + IP) Questions / Help	5	2331	January 22, 2022
Hiding metrics from the public internet with Cloudflare help-me-help-you	1	291	November 9, 2023
redirect_uri is http instead of https Questions / Help	6	831	August 28, 2022
Only expose /metrics for Fly internal scraper Questions / Help elixir	1	378	January 19, 2022

Errors querying metrics, 301 and 401

Related Topics