# How to debug why Squid Proxy delivers 500 error codes intermittently?

**URL:** <https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917>\
**Category:** General\
**Tags:** login-only\
**Created:** [September 13, 2025, 7:41am UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917 "2025-09-13T07:41:25Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![erlangga](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@erlangga](https://community.fly.io/u/erlangga)\
**Post date:** [September 13, 2025, 7:41am UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/1 "2025-09-13T07:41:25Z")

</div>

Tbh im not really sure how to ask this

i have 1 nodes using squid (that have static egress ip)  
and my other 4 nodes will connect via proxy to that node to call third party app

this is related to [Egress IP sometimes not working 100%?](https://community.fly.io/t/egress-ip-sometimes-not-working-100/24887)  
now suddenly after 3 AM Jakarta time the proxy is not working , i try to change the squid config but still no result

atp im not really sure why and how because at that time im not changing anything and suddenly receive alert error that the api of third party cannot be called in my email

---

<div class="post-metadata">

**Author:** ![erlangga](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@erlangga](https://community.fly.io/u/erlangga)\
**Post date:** [September 13, 2025, 7:56am UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/2 "2025-09-13T07:56:38Z")

</div>

so after i try create simpe code that chek my endpoint 1000 times wether it fails or not

Summary:  
200: 815  
403: 0  
500: 65  
other:0  
error:120

500 = My Proxy Fails

atp im not sure the error is because the squid that cant handle the connection or something

---

<div class="post-metadata">

**Author:** ![halfer](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@halfer](https://community.fly.io/u/halfer)\
**Post date:** [September 13, 2025, 12:27pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/3 "2025-09-13T12:27:36Z")

</div>

I doubt there is enough information here, or enough clarity, for readers to help you.

Consider adding a diagram of your apps/machines, so readers can see where traffic is coming in. In general if your Squid Proxy is giving 500 errors intermittently, it is possible that your app is failing, and thus Squid is fine. What HTTP status code is “error”?

Can you run your 1000-check script inside your app, using `flyctl ssh console`? This will skip the proxy, and you can then see if your app is to blame.

Also check your app logs and your proxy logs.

---

<div class="post-metadata">

**Author:** ![erlangga](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@erlangga](https://community.fly.io/u/erlangga)\
**Post date:** [September 13, 2025, 2:56pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/4 "2025-09-13T14:56:47Z")

</div>

I think you were right im sorry for the lack of information

for now i will try to make produceable scenario but the hard thing is this issue only occur once a month maybe for whole day suddenly the squid proxy is act weird even tho i already restart

like right now everything works fine again even tho i done nothing besides eating my ramen 🥰

i create mini project [https://reproduce-proxy.fly.dev/test-proxy?num\_requests=250](https://reproduce-proxy.fly.dev/test-proxy?num_requests=250)  
to check proxy failing or not maybe the next time i see the error i can give more information

---

<div class="post-metadata">

**Author:** ![halfer](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@halfer](https://community.fly.io/u/halfer)\
**Post date:** [September 13, 2025, 3:47pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/5 "2025-09-13T15:47:14Z")

</div>

No worries. You should find that, even if the problem can no longer be reproduced, you can look at your app or proxy logs for the time the error happened, and the relevant logs will still be there. Is it worth having a look now?

---

<div class="post-metadata">

**Author:** ![erlangga](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@erlangga](https://community.fly.io/u/erlangga)\
**Post date:** [September 13, 2025, 4:37pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/6 "2025-09-13T16:37:26Z")

</div>

The only logs i see is from my python django app that requests to payment gateway service got rejected (timeout) because cannot connect to proxy

`except requests.exceptions.ConnectTimeout as e: raise HTTPException(status_code=502, detail=f"Proxy connection timeout: {str(e)}")`

the thing is only happen very very randomly sometimes once a month but sometimes full month is okay  
i create custom grafana monitoring in may because i want to monitor this issue

 ![Screenshot 2025-09-13 at 23.22.28](https://global.discourse-cdn.com/fly1/original/3X/f/c/fcc5efe97b735870fb7f0475ac1c3dd22507c4fe.png)

what usually happen is

> I woke up  
> check my message 50+users complain in our whatsapp group that they cannot withdraw their money from our site  
> check the logs  
> says proxy timeout  
> try to restart the squid server  
> no luck  
> 1 hour later suddenly working again

this already happen like 5-8 time already since last year we migrate to fly

the issue was today the withdraw service down for like 8 hour 3-AM Jakarta - 12 AM Jakarta  
and then recover again but with low error rate (some user still got the error but most are fine)  
but during 15:00 / 3PM Jakarta time the error occur again  
an today 6 PM jakarta time while i build the reproduce. everything is fine again

 ![Screenshot 2025-09-13 at 23.30.09](https://global.discourse-cdn.com/fly1/original/3X/d/8/d8880825df0d58d22fc20940402430a752150a2b.png)

currently my mitigation is i increase the proxy memory which currently only had 512mb of ram because i saw some spike on the memory which im not sure why

this is my squid config

`# Minimal permissive Squid forward proxy

# WARNING: This is an open proxy configuration. Restrict access before exposing publicly.

# Listen on port 8080

http\_port 3128

# Standard ACLs for allowed ports

acl SSL\_ports port 443  
acl Safe\_ports port 80 # http  
acl Safe\_ports port 21 # ftp  
acl Safe\_ports port 443 # https  
acl Safe\_ports port 70 # gopher  
acl Safe\_ports port 210 # wais  
acl Safe\_ports port 1025-65535 # unregistered ports  
acl Safe\_ports port 280 # http-mgmt  
acl Safe\_ports port 488 # gss-http  
acl Safe\_ports port 591 # filemaker  
acl Safe\_ports port 777 # multiling http  
acl CONNECT method CONNECT

# Allow everything (OPEN PROXY). Harden before production use.

http\_access allow all

# Optional: Disable on-disk caching

cache deny all  
cache\_mem 0 MB  
maximum\_object\_size 0 KB  
cache\_dir null /tmp

# DNS and forwarding behavior

dns\_v4\_first on  
forwarded\_for on  
via off

# Keep Squid quiet-ish

shutdown\_lifetime 1 seconds`

and the architech is look like this  
the reason i need to use proxy is because the payment gateway only allow me to register limited number of ips (4 max i think)

 ![Screenshot 2025-09-13 at 23.35.21](https://global.discourse-cdn.com/fly1/original/3X/a/1/a1ecbfeade651c4cb7a9741cd19b235b586262b7.png)

since i know most of our users (just small apps with thousand of users)

i just tell them okay wait lemme fix this (doing absolutely nothing other than restart and pray) and ask them to use the feature again after couple of hour

my guess is the ram but spending another 10$/mo for ram on proxy server little bit ….

---

<div class="post-metadata">

**Author:** ![halfer](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@halfer](https://community.fly.io/u/halfer)\
**Post date:** [September 14, 2025, 9:01am UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/7 "2025-09-14T09:01:18Z")

</div>

Ah, could you be affected by the current network outage? You mention users in India.

> We are continuing to work with our external connectivity providers to address the service degradation caused by multiple submarine cable outages. Traffic between India and Europe/US East is impacted. Since this is an external incident, some degradation may still occur. We’ll provide another update as soon as we have more information.

Did you have this problem before 6th September?

---

<div class="post-metadata">

**Author:** ![halfer](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@halfer](https://community.fly.io/u/halfer)\
**Post date:** [September 14, 2025, 9:59am UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/8 "2025-09-14T09:59:31Z")

</div>

(The diagram is very useful. Consider adding that sort of thing in your first post, where you have a question of this type.)

I would put a direct uptime monitor on the payment gateway, and also I would maybe add a mode in one of the Django apps to contact the payment gateway directly, to see if the success rate is better in that one app.

In relation to logs, I would be certain that if you are getting proxy timeouts, there will be logs at the proxy. Keep looking for relevant logs: they do exist, and you’re just not finding them.

---

<div class="post-metadata">

**Author:** ![erlangga](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@erlangga](https://community.fly.io/u/erlangga)\
**Post date:** [September 14, 2025, 1:21pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/9 "2025-09-14T13:21:54Z")

</div>

> [@halfer](#):
>
> Ah, could you be affected by the current network outage? You mention users in India.

My users is from Indonesia and we deploy it on region SIN (Singapore)

> [@halfer](#):
>
> In relation to logs, I would be certain that if you are getting proxy timeouts, there will be logs at the proxy. Keep looking for relevant logs: they do exist, and you’re just not finding them.

you were probably right but right now i think my squid log is not enabled

> [@halfer](#):
>
> I would put a direct uptime monitor on the payment gateway, and also I would maybe add a mode in one of the Django apps to contact the payment gateway directly, to see if the success rate is better in that one app.

tbh this is good idea, ill try that but since it was IP whitelisted only prolly ill just get 403 access but i still able to compare it , thanks

---

<div class="post-metadata">

**Author:** ![halfer](https://avatars.discourse-cdn.com/v4/letter/h/df705f/32.png) [@halfer](https://community.fly.io/u/halfer)\
**Post date:** [September 14, 2025, 1:31pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/10 "2025-09-14T13:31:12Z")

</div>

> [@erlangga](#):
>
> right now i think my squid log is not enabled

OK, that’s the first thing you can fix then, so you can understand the problem next time it happens. 🏆

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/fly1/original/3X/6/e/6e8955d00cb218e186ebbe3a6ab06fefdce15290.svg) [@system](https://community.fly.io/u/system)\
**Post date:** [September 21, 2025, 1:31pm UTC](https://community.fly.io/t/how-to-debug-why-squid-proxy-delivers-500-error-codes-intermittently/25917/11 "2025-09-21T13:31:38Z")

</div>

Closed on September 21, 2025, 1:31pm UTC

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.
