How to Deploy a Docker App to AWS using Elastic Container Service (ECS)
Вставка
- Опубліковано 27 чер 2024
- In this step by step tutorial, I show you how to deploy a Flask based Docker app to AWS. We start by building a local docker image and uploading it to Elastic Container Registry. We create an ECS Cluster, and configure a task to run our docker image. Finally, we launch a Task into the cluster and expose the app to the internet via VPC.
Looking to get hands on experience building on AWS with a REAL project? Check out my course - The AWS Learning Accelerator! courses.beabetterdev.com/cour...
00:10 Example Overview
02:15 Application Walkthrough
04:34 Creating a ECR Repository
05:16 Uploading your Image to ECR
06:28 Creating an ECS Cluster
09:10 Creating a ECS Task Definition
11:35 Running a Task on your Cluster
13:03 Testing our app
Want the docker file used in this video? Check out docker-curriculum.com/ Tutorial (about a third down the page). docker-curriculum.com/
How to install the AWS CLI - • How to install and con...
Source Code + Policy Template - gist.github.com/awssimplified...
🎉SUPPORT BE A BETTER DEV🎉
Become a Patron: / beabetterdev
📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚
Clean Code - amzn.to/37T7xdP
Clean Architecture - amzn.to/3sCEGCe
Head First Design Patterns - amzn.to/37WXAMy
Domain Driver Design - amzn.to/3aWSW2W
Code Complete - amzn.to/3ksQDrB
The Pragmatic Programmer - amzn.to/3uH4kaQ
Algorithms - amzn.to/3syvyP5
Working Effectively with Legacy Code - amzn.to/3kvMza7
Refactoring - amzn.to/3r6FQ8U
🎙 MY RECORDING EQUIPMENT 🎙
Shure SM58 Microphone - amzn.to/3r5Hrf9
Behringer UM2 Audio Interface - amzn.to/2MuEllM
XLR Cable - amzn.to/3uGyZFx
Acoustic Sound Absorbing Foam Panels - amzn.to/3ktIrY6
Desk Microphone Mount - amzn.to/3qXMVIO
Logitech C920s Webcam - amzn.to/303zGu9
Fujilm XS10 Camera - amzn.to/3uGa30E
Fujifilm XF 35mm F2 Lens - amzn.to/3rentPe
Neewer 2 Piece Studio Lights - amzn.to/3uyoa8p
💻 MY DESKTOP EQUIPMENT 💻
Dell 34 inch Ultrawide Monitor - amzn.to/2NJwph6
Autonomous ErgoChair 2 - bit.ly/2YzomEm
Autonomous SmartDesk 2 Standing Desk - bit.ly/2YzomEm
MX Master 3 Productivity Mouse - amzn.to/3aYwKVZ
Das Keyboard Prime 13 MX Brown Mechanical- amzn.to/3uH6VBF
Veikk A15 Drawing Tablet - amzn.to/3uBRWsN
📚 References:
Getting started with AWS: • Introduction to AWS | ...
☁Topics covered include:
Docker
Flask App
Elastic Container Registry
Elastic Container Service
🌎 Find me here:
Twitter - / beabetterdevv
Instagram - / beabetterdevv
Patreon - Donations help fund additional content - / beabetterdev
#SoftwareEngineer
#SoftwareDeveloper
#ElasticContainerService
#AWS
For anyone getting this error: CannotStartContainerError: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:338: getting the final child's pid from pipe caused: read init-p: connection reset by peer: unkno
The cause is there's not enough memory assigned for your Task. If you followed along to the video, in task definition, change memory from 1 to 512 and CPU to 512 as well. Hope it helps
Thanks for pointing this out Alexey! I've had a few folks comment on this same issue and wasn't sure of the reason. Glad you were able to figure this out and share with others.
I'm going to pin this to the top so others can benefit. Cheers!
It doesn't work for me.
I am using GitLab Container registry. ARN seems not working as well.
GitLab returned an error "unauthorized: HTTP Basic: Access denied
You must use a personal access token with 'api' scope for Git over HTTP.
You ca". Not sure how to set GitLab personal access token in AWS secret used for ARN.
I just made it work. I need personal access token as password. :)
Man! I wanna hug you! I suffered for about a month through bullshitery guides from amazon on how to do it! But you explained it so simply! Now I will need to make a pipeline with autodeploy. I hope you'll have videos on that topic. THANK YOU once again for such clear explanation.
You're very welcome !!!
This was amazing, I've ben looking for this exact kind of tutorial for ages and it really cuts through the verbose aws documentation! Thank you so much!
Glad it helped!
Thanks for this awesome tutorial!
For anyone actually trying to deploy some kind of code that needs to be built on the machine, please take EXTRA care in making sure that the instance type has more than enough memory to actually build the app and start it properly. Figured this out only after ssh-ing into the EC2 container and interactively running commands on the docker image to try and manually build the project
You saved my day! Thanks. Specially when you showed that the public IP wasn't accesible because of the default security group that the instance had attached.
Glad I could help!
straight forward and quick, thank you!
Thank you so much saviour you made my task like flawless. huge respect
Have been looking for such a tutorial since two months now
Glad you enjoyed!
Great video - really helps with understanding ECR, ECS and EC2. 😀
Thank you so much, I have a container enabled on port 9000 but didn't know why it wasn't working, this is a life saver.
Thanks again.
You're very welcome Juan!
Love your video!! Thanks!!..quick and easy !...with no garbage! God Bless you!
You won one subscriber!
Thanks so much Israel and welcome to the channel!
How could one figure all this out without this tutorial... the amount of steps are insane and the AWS CLI is so not intuitive! thanks for demystifying it for us
Thanks so much ad ma! I feel your pain - doing this the first time I almost wanted to rip my hair out :P
This is a great tutorial! Thank you very much.
It'll be great to see this being done using a CI/CD pipeline!
Thank you for the consistence aws videos.
You're very welcome!
Thanks lot for your wonderful videos .. content are sharp and crisp
Thanks Sridhar!
Well explained, thanks for good tutorials.
Glad you like them!
Precise and on point. Thank you.
Glad it was helpful!
Great video Thanks a lot. I tryed many times to do this but only with your video I solved the problem very very great thanks
Man, congrats!!! This video helped me a lot! Very nice!!
Thanks Vinicius! Glad you found it helpful.
You make it seem easy, thank you
Only after hours of struggles ;)
The cat gifs make everything worth it
Haha I agree!
Thank you for the video, I was about to give up trying to run a container from a Docker image because I could not understand Amazon's documentation. This video was incredibly helpful.
Glad I could help!
Thank you. Extremely helpful info.
Thanks for this awesome overview
thanks man, great tutorial !
Glad you liked it!
Super nice....thanks, vey useful, especially the task definition part.
Great tuturial. Just a side note - if you just crated your account, it can take a while before you're able to create EC2 instances in all regions, so if no EC2 instance shows up, you just have to wait a few hours - that was the case for me at least. Also if you get memory problems when trying to run the task with 1024 memory set for the task with a t3.micro it's becaues the t3.micro only has 1gb (1000mb) in memory, so try 512 and you sohuld be good.
1GB is 1024MB. 1GB is NOT 1000MB
Great tutorial, thank you.
Thanks a lot, great video, great explaining
Glad it was helpful!
Thanks for this helpful tutorial, it got me up and running! However, one issue that I ran into was trying to run multiple clusters (one for a staging release of my app, one for production) - I found that if I put both clusters on the same subnet, the second cluster wouldn't start an EC2 instance; putting them on different subnets resolved that. This is my first foray into AWS and I have about zero experience with networking, so maybe that's totally expected and normal - but if you're like me and you're just starting to learn your way around, maybe this'll be helpful for you.
Hi Darren, I can't say I've personally experienced this issue but thank you for sharing your knowledge! I'll keep this in mind next time I set up ECS. Cheers!
Thanks for this it was exactly what I was looking for to learn about ECS. Great tutorial!
Very welcome.
wonderful tutorial!!!!!!
you are a life saver! thanks!
Happy to help Carlos!
Part about VPC very helpt, thank you very very much!
Glad it was helpful!
Thank you Sir Very much for your guidance
Thanks a lot. Nice tutorial.
Glad it was helpful!
Thank you for this !
You're very welcome!
Awesome!
Great one Thanks!
You're very welcome Maor!
Thank you man!
Happy to help!
Thaks! nice video!!!
Great intro for me
Good job man.
Thanks eric!
1:30 *task*
3:33 *login to ECR*
4:18 create repository in ECR
4:46 upload the built image onto ECR repo
4:52 tag image: docker tag
5:17 push
6:24 create a cluster
Thanks mate!!!
You're very welcome Sebastian!
thanks man!
You bet!
nice, thanks!
You're very welcome!
Mahnn, you was dishing out the info hitting me like arrows, had a take nap to recover from the information overload.
Haha I hope it was useful!
Awesome video ++++++++++++ 🙂
Thank you!
You're very welcome!
Great tutorial! Thank you!
You're very welcome!
Thanks
thank you very much
You're very welcome!
Thanks a lot.
You're very welcome!
luv it
Thanks!
Thank you for sharing. This is was greatly done, straightforward and so much easy to understand.
Thinking on the monitoring, What are some things that you'd recommend to be monitored for apps running on ECS, whether it's setup using EC2 or Fargate?
The video was very helpful. Thanks for sharing such a great content. But it is confusing the part you call the image and the repository 'test', having them with the same name makes the watcher confused if they are using other names.
thx
The Flask application works without any problem . Thanks a lot for this important example.then I have tried to deploy a django application on a ecs container following your instructions.
I am afraid that the T2micro is too small to load a imagine of 1.24 Gb .
Nice. You've explained in less than 20 minutes what takes hours of crawling over terrible AWS documentation.
Great this tutorial worked for me! Question in mind tho, do i need to change SG and network mode when deploying to ElasticIp and Route53?
Great to hear Barry! In order to make your instance publicly accessible, you would need to make your instance able to receive traffic from the public internet (inbound rule with 0.0.0.0/0). Hope this helps.
4:10 If you cannot login, you need to allow access for "AWSAppRunnerServicePolicyForECRAccess" at AWS IAM console. I watched AWS CLI tutorial, but I struggled because I didn't know about "AWSAppRunnerServicePolicyForECRAccess".
also, I allowed access for "EC2InstanceProfileForImageBuilderECRContainerBuilds". I'm not sure which is necessary to login.
Just want to give a heads up to anyone running a react container and have issues with your task exiting after starting.
The error I'm talking about it "essential container in task exited". If you are facing this issue you need to turn on the pseudoTerminal within your task so it doesn't exit immediately while react is starting up.
1. Find your task and click "create new revision"
2. Scroll all the way down until you see "volumes" and click "Configure via JSON"
3. Find "pseudoTerminal" and set the value to "true"
4. Assign the task again and everything should be working fine now
thank you for this i had this problem and i tried the solution but it still exists even after modifying the json file
edit: i created another revision with 512mb and that seems to work
Thank you
It worked! Can you pls explain why this happened?
How would we go about connecting the flask app to a db like mongodb with persistent storage?
Thank you for the great tutorial! Based my observation and the time of this video, I guess you were using an Intel Mac?
For anyone also on M1 chip Mac: I use Apple M1 chip Mac and it creates ARM64 image by default. It’s not runnable on EC2 t3.micro instance.
If you still want to run the an ARM64 image with EC2, you’ll need to select instances powered by Graviton, for example t4g.micro, which can run ARM64 workload. OR you need to run `docker buildx build` to build an amd64 image at the beginning.
Great Video Thanks! Quick question... Is this video a part of some series ..I would like to know how/what docker files are.
Hi Veens! This video was meant to be a standalone one, however I do have a video explaining what Docker is. Check it out here: ua-cam.com/video/oyD5yIEAeoA/v-deo.html
Cheers
Tutorial also works for running serverless Fargate tasks (at least in Nov'22 🙂).
very helpful. do you have any videos showing how to deploy images on ecs from local docker CLI? I'm getting this error:
pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
To anybody using an M1 Mac and getting "(Essential container in task exited)" error, I realized I was building with arm64 and the architecture of the ECS Cluster is AMD64. To solve this issue, use "docker buildx build --platform linux/amd64 -t app ." and then tag it again and publish it to the repository. Wasted one hour on this.
I stuck for 1 day. You saved my day!
Wow great catch. I was stuck on this for so long. Huge con of using M1 is that most platforms/software are not compatible yet, and it's hard to realize sometimes until after countless troubleshoots.
how'd be the process for a multi container application ?
Hi great video. I followed this along but got stuck where the task was pending status. Can you show me how you fixed it exactly? I was following exactly what you were doing.. I don’t know what typo you made.. is that container name ?
Nice and simple as always. I don't see the Dockerfile dan Python code on the Gist. Please advice.
A video to make it HTTPS would be very useful
Now do it through cloudformation and link it up to a code pipeline for a full CICD automated deployment!
This would be epic :)
Do you have a video that shows how to add --env to the docker run command? I need to do this although we use terraform to actually provision the fargate container. Maybe terraform adds those env vars?
Hey Mike,
Unfortunately I've never done this with Terraform before. However I found this link that you may find helpful: stackoverflow.com/a/40784106/13872863
Thanks,
Daniel
Excellent..Thank you. How much do you think it would cost for the application you wrote to run on ecs for a month?
Hi James, if you use this guide the costs zero. Be sure to use the Free Tier eligible EC2 instances (provided that fits your use case) and you can try this out with minimal costs. Also, be sure that your docker image is below 500mb in size so that you can stay in the free tier.
Hope this helps,
Daniel
@@BeABetterDev hey if image size exceeds 500 mb is there any workaround to still deploy it under free tier, like resize image, etc
Great tutorial, I have a question. I have a similar set-up and wonder if there is any way to handle clients loosing connection each time you make a deployement? E.g every time I deploy the users that were authenticated and signed in to the site gets kicked out since the container with redis gets cleared
Hi Tolga, this is an interesting problem. Is it possible your Redis instances can dump state onto disk so the caches can be recovered on startup? Else, you may want to use the dedicated AWS ElasticCache service with Redis.
Hope this helps
@@BeABetterDev Hmm that could be possible, I will have to look into it. Thanks for the tip!
How did you end up fixing the issue with image uri being incorrect format?
Hi, what was the mistake you defined the task definition? I dont know what you mean by the wrong url. The wrong url of the registry/image? I copied pasted that. Please help. Not sure whats wrong
Please do one showing how to run a private registry.. :(
Nice and helpful - around 7:46, I get an error at this step saying '
For
container instances to receive the new ARN and resource ID format, the
root user needs to opt in for the container instance IAM role. Opt in
and try again.
I made the role, but it doesn't show up in the drop down, and I can't find where to attach it to root. Any one have an idea?
Does this tutorial works if I try to deploy a SB application with MySQL using docker compose?
Hey man! Excellent tutorial. However, I am struggling when trying to use a private image (stored in ECR). Documentation is a bit confusing and I cannot seem to find aby good resource for dealing with that. Wondering if you or someone else might help ?
Thanks
Hi Farid. I haven't dealt with this personally. Can you share what kind of error your are running into? I may be able to help.
@@BeABetterDev thanks. it looks like if you are using private ECR images when setting up your containers on ECS (and the ECR image is from the same AWS account), no additional setup is needed for ECS to function properly, yet your Docker image is still private. sweet!
Hello. what if i have a database that i want run in a docker container but don't want it to scale? Do i put in in a separate cluster with 1 desire & maximum capacity?
You sir are a lifesaver! I've been trying to figure this out for a week.
I do have a question. I created the ECS cluster but i don't see any ECS instances, even though i have an image pushed to ECR. Any idea what i'm missing?
Hey...same here. Do let me know if you find a solution.
@@krishnans1665 I ended up deleting the cluster and then recreating it. I think the only thing i did differently was add a different VPC and then followed the video from there.
It showed up this time.
@@XpanderTN Weird, I had the same problem, I deleted the cluster and did exactly the same setup (not change in VPC) and now the instance shows up. 😕
@@mathematica7 Super weird. I wonder what determines if the cluster is able to see the instance? They were both valid ec2 instances and i built the first one from the cluster creation screen.
Is there a way you can do it as an auto deploy to EC2?
If I use fargate, does it mean I don't need to define any EC2/ECS ? Thanks.
Tasks are similar to deployment in k8s
Load balancer target group?
how to deploy multiple docker containers on a single ec2 instance with the load balancer and assign a custom route 53 dns to it. Please help to sort it out
Can you please share the flask/python code so we can completely follow along? That's be super helpful.
@ 2.50 the command typed is docker built
I was looking off screen when this happened I didn't even notice. Luckily I had built the project prior so it didn't affect the video.
What did you change at 12.33 to make your task run
Hey, I followed your tutorial all the way through. I made sure to copy the image URI correctly and use that value when I set up the container. However, I too am also getting an error:
OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:319: getting the final child's pid from pipe caused \"read init-p: connection.
Would you have any idea what might've happened?
I received the same error
did you figure out the solution?
I'm having same issue. Anyone know how to fix this?
I worked out the issue. The dockerfile was not running the command to start the process
@@Bobifier how did you fix this?
At 10:54 the port mapping in the user interface has both a container port and a host port, however when I look at my console there is no host port. I can no longer map the container port 5000 to port 8888.
How should you go about that now?
there is a way to do the same, but with docker compose?
I’m guessing that using a nginx proxy would be declared within the task definition?
What was the policy document for?
I didn't need to use it ... ?
i am getting this error
Stopped reason
Essential container in task exited. Do u know about this error.
Where can I get the flask app code
if i wanna use fargate, should i choose everywhere fargate option instead of ec2?
I mean
cluster template : networking only 6:32
task : fargate 9:17
run task : fargate 11:44
Hi Zhdan,
To ensure compatibility (and not run into any headaches / configuration problems), I would definitely suggest clicking the fargate option.
Cheers