Hi I have an application where I am storing photos onto AWS. It works well, except I'm not sure how to let the URL be public. I know I can go into the object itself on AWS and change it's permissions to allow the public to read the file. Is there a way using this library that I can do the same thing for each object I'm putting in to my bucket?
Hi Spencer,
the most convenient way of making some objects in a store public is to create a bucket policy.
In the AWS console switch to the S3 dashboard, select your bucket and go to the Permissions tab.
Click on Edit for "Edit block public access (bucket settings)" and remove the checkbox for "Block all public access" then click on Save changes.
Next scroll down to Bucket policies and click edit. Here is a policy that allows access to objects in a bucket that are prefixed with "images" (folder) and only those.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::<YOUR BUCKET NAME>/images/*" } ]}
This is just a simple one, more complex permissions can be granted based on a conditions like a metadata value asf.
you might also add a CORS configuration depending on how you want to access the objects.
Create a folder images and upload some images. Try to access a image by its https path. You should be able to access it (try from a InPrivate browser instance). Upload another image to different location and try to access it. It should return an access denied.
For making S3 objects available at scale and secure you may find my article https://itnext.io/serve-files-at-scale-with-outsystems-and-aws-s3-and-cloudfront-6f0b11a37866 helpful.
Best
Stefan
I appreciate your response. I think you're right in using a bucket policy is the way to go. I'm fairly. new to AWS and want to make sure that it is getting set up to be secure as possible. Can I make a bucket policy where the images are only public when getting accessed from a specific website? Or would that be a combination of a bucket policy and CORS?
Or would I be able to use the CloudFront approach?
mh. I havent used that yet, but you could try adding a referer condition to your policy. Referer is not really secure as it can easily be manipulated.
{
"Version": "2012-10-17",
"Statement": [
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<YOUR BUCKET NAME>/images/*",
"Condition": {
"StringLike": {
"aws:Referer": "https://www.example.com/*"
}
]
The other option is to set it in the CORS policy (AllowedOrigins). Like this
[
"AllowedOrigins": [
"https://www.example.com"
],
"AllowedMethods": [
"GET"
"AllowedHeaders": [
"*"
"MaxAgeSeconds": 3000,
"ExposeHeaders": []
CloudFront I would only recommend when you have large volumes of public images and need high image loading performance across regions.
Our use case is each night hundreds of photos are taken and submitted to AWS. Then in the day those on the internal system are looking at those photos and deciding if a job needs to be created from what they see on the photo. So we are processing a large amount of photos each day.
I see. But the processing of photos (by the way, can I get a large OutSystems sign for my house? ;-) ), takes place centrally right? In that case there is no need for CloudFront as the "clerks" working on the photos are not spread across the planet. In that case your good to go with the S3 bucket alone. CloudFront would not add any benefits but just makes the architecture more complex. Even if you later on discover the need for a CDN you can attach it any time.
Haha ya how big you want the sign? But you're right all the processing is just done in the US. Well I'll play around with CORS and with the bucket policies as well. The less complicated but still secure the better. Thank you for your help!
Twice as big as my house :-)
Regarding "secure": This is somewhat secure, but the referer and origin can easily be manipulated. So if the photos are confidental you are better off with the either the CloudFront approach for accessing private data i described in my article OR you create pre-signed GET Urls upon request of a picture at runtime (so when a user clicks on a link asf). Generating pre-signed Urls are part of the forge component.
I live 30 Kilometers outside of Munich. The idea would be that the sign is spotable from the Frauenkirche in Munich city center :-)
Consider it done
How's it going with S3 Spencer?
Hey Stefan,
I got a little sidetracked on other parts of a project and am now just coming back to the S3 stuff. After talking with my boss and colleagues we decided that the presigned URL is the way to go. That being said I'm about to refactor how we are currently doing it to match that. Maybe you can answer this for me as I prepare to refactor. I know that when you request the presigned URL you set the expiration time. Do I need to make logic that when that time is past I need to request another presigned URL? Or does the component just allow me to use the client action of retrieving a presigned URL and it determines if it can grab the current URL that is not expired vs calling for a new one?
I hope I got that right.
A pre-signed URL is self-contained. It is not requested from the S3 service but computed locally (in that case in an external logic function) with your AWS credentials. That said you can have multiple generated signed urls targeting the same resource.
The best option is to generate the pre-signed URL the moment you want to request the object in S3.
If you can wait for another week then you will find some best practice patterns on my blog https://without.systems regarding S3. I already wrote 3+1 Integration patterns in preparation for my session at the Lisbon Developer Day. But they will be published on the 19th and not earlier.
Oh I'll be watching for that article and try to implement the suggestions. The use case I'm trying to explain is when somebody clicks on the link for a photo it will generate that pre-signed URL as I'm requesting the object in S3. I know I need the client action to that. What I'm not sure of is if they go and click it again do I need to have logic that says "oh hey you just did this and it hasn't expired look at this pre-signed URL" or will it just request another pre-signed URL?
Just request a new one. Keeping track of the expiry is not worth the effort 😏
Haha so I have your permission to do the easier approach? perfect ;). I guess if it becomes a problem I'll rework it to where I keep track of the URL to save on calls, but for now the cost is so minimal I don't see it being a problem. Thanks for your help
@Stefan Weber I just read your articles about ODC and AWS. Well done, very informative and helpful. I wanted to pick your brain a little bit. As discussed above I ultimately landed on using presigned URLs for my mobile application, which has worked great. Once the photos are uploaded to S3 a user in a web application will be able to query for them also using presigned URLs. They have a longer expiration time than the mobile app and I cache the server action where they generate the URL to avoid frequent calls to S3 where I can avoid it. (These images don't really change so I don't need to worry about accessing an outdated photo.) All this is also working.The only problem I run into is this: When a user on the web application decides ok I want to email these photos out to a customer they can do so. The email gives them an option of attaching the images as attachments, but it will always embed the images into the email as well. When I try to embed the images into the email with a presigned URL it appears the URL gets encoded and ultimately breaks. It won't even send the email because it can't successfully fetch the image. I'm assuming that OutSystems will try and download the image to embed it onto the email, though I'm not entirely sure.The other concern I have is once that expiration time is hit on the image embedded in the email will the customer no longer be able to see that image that was embedded because the URL expired? I know that isn't a problem for the attachments. What would your suggestions be for working around these two problems?
Thank you Spencer. Always nice to get some feedback :-)
So far I havent had the requirement to send emails with attachments. Or lets I could prevent sending large emails :-) and instead redirected the receiver to a secure screen where he can also view the "attachments".
If you need both embedded pictures and the attachments I would assume that the embeds do not need to have the same size and quality like originals. A possible solution could be (but I havent tried it) to generate a thumbnail from the original (I guess there should be something on the forge that can help) and then convert the thumbnail binary to base64 and use the base64 as a custom img tag src.
Unfortunately it doesn't seem that gmail allows you to embed Base64 images into the email itself. The attachment stuff again isn't a big deal and I can do that fine. But inside the email body itself is the problem. So now I'm torn between just making everything public again, which I don't love or having to find some other solution for emails.
You could create a screen that accepts a token you validate and displays a gallery with the images?
@Stefan Weber Have you played around with sending emails through AWS rather than OutSystems?
Yes. In close to all our projects we send thru Amazon Simple Email Service.