I hope you are having a good end to the week. Me, on the other hand, I went on an S3 scavenger hunt
For context, my production code works fine but I haven’t ran any deployments or dependency upgrades in a few days. Today, I was working on a feature on my local machine and after upgrading my deps (go get -u ./… and go mod tidy) all my requests to Tigris started failing with the following error:
operation error S3: PutObject, https response error StatusCode: 501, RequestID: 1737729871240111208, HostID: , api error NotImplemented: A header you provided implies functionality that is not implemented”
I tried connecting to a different bucket, changing permissions etc. but with no luck - and the error message isn’t very clear as to what is not implemented. However, I came across the AWS S3 Go SDK changelog for v1.73.0 (15/01/2025):
Feature: S3 client behavior is updated to always calculate a checksum by default for operations that support it (such as PutObject or UploadPart), or require it (such as DeleteObjects). The checksum algorithm used by default now becomes CRC32. Checksum behavior can be configured using when_supported and when_required options - in code using RequestChecksumCalculation, in shared config using request_checksum_calculation, or as env variable using
So it seems that the Checksum Calculation is causing errors on Tigris’ side? I ended up fixing the error by requesting for the checksum to be calculated only when required:
svc := s3.NewFromConfig(config, func(o *s3.Options) {
o.BaseEndpoint = aws.String(u)
o.Region = r
o.RequestChecksumCalculation = aws.RequestChecksumCalculationWhenRequired // new line <----
})
I am not sure if this is a robust solution as I haven’t tested DeleteObject yet. But it’s something that I wanted to bring to the community’s attention as it might start affecting people upgrading their deps.
Edit: I will mark your answer as the solution, so that people that come across this post know that there’s a solution and that you guys are looking into it
I get a similar 501 error when using PutObject Kotlin API but only when my file is larger than 1024* 1024 bytes. I’m using aws-sdk-kotlin 1.4.9 (Release v1.4.9 · awslabs/aws-sdk-kotlin · GitHub) which just came out and supposedly contains a fix for a header signing issue but I don’t know if this is the same issue.
When uploading the same file via the aws s3 CLI everything works, but running with --debug shows that it’s doing a multipart upload.