I have a shadow bucket setup that we are using to mirror objects from S3. I need to download all objects under a specific prefix but I get a 404 unless I download a specific object. Is it possible to do this with a shadow bucket?
This is a great ask Cullen, but unfortunately, it’s not currently supported. I’ll have a feature request for this on our side. To unblock for now, one option is to perform a list
on that prefix from s3 and then perform get
so that automatically migrated to Tigris.
Thanks for the response. Listing does not seem to show objects other than those that have already been copied into tigris with an explicit cp
. Could you show an example of this working?
Currently, it is only supported for the shadow buckets that are configured in “Write Through” mode where list
results are fetched from the shadow source. Otherwise, you’re right. The results are returned from Tigris.
I was suggesting if you explicitly perform list
on s3 and then based on the results that you get from s3 then you perform get
on tigris. This will trigger the migration of these objects to Tigris.
Let me know if this helps, happy to help you more on the migration. Or if you want you can also reach out to us directly to help@tigrisdata.com.
Added wishlist
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.