Shout out to @IvanPopov for taking this video to the next level in this blog article: polystack.blogspot.com/2024/10/passing-credentials-to-gcp-cloud-run.html In this article, Ivan shows a simple technique for passing ENV vars/secrets to Cloud Run containers in CICD. 🔥
Very nice. I only wish you addressed one the trickiest parts that may arise in that process: passing the JSON (through a secret) of another service account to the container through the --set-env-vars property.
Great to meet you @IvanPopov! I can include a NOTE in the video description and/or GitHub repository and shout you out with this extra tip you are recommending Can you include the command and a little note of why it's so useful?
@@cmd_labs Generally the command is just like yours (the last command in your cicd.yaml file) except that at the end it uses a few consecutive --set-env-vars flags. In those flags I'm passing the environment variables required by the application. For the values of these variables I'm passing some secrets. Now for the regular one-line variables (like bucket names, volume mounts, etc.) the values are being passed just fine. But one of the values is the key that represents a service account, i.e. that value is a multi-line JSON data (exactly like the one you defined in the video). The Github environment (the ubuntu-latest runner) has a problem with that for parsing it in yaml and is giving me hell for the past two days. I tried many sorts of escaping. Today I tried the flags file approach with the same results. It's all about multi-line secret JSON value not being parsed properly into the yaml. Next I'm trying (as much as I wanted to avoid it because I don't want to touch the application code) is the base64 encoding approach. For brevity I'm pasting here the relevant part of my workflow (the variant with flags file) although I doubt the format will be preserved in the comment ... # Setup Google Cloud SDK (no need for additional authentication here) - name: Setup Google Cloud SDK uses: google-github-actions/setup-gcloud@v2 with: project_id: ${{ secrets.GCP_PROJECT_ID }} - name: Create Flags File run: | cat
@@cmd_labs OK. Let's see ... my command is not too different than the last one you're using in your cicd.yaml descriptor. I'm adding a several --set-env-vars flags for the variables I want to pass to the container. The values of those variables come from the secrets I defined, the same way you did for the service account you're using for the authentication. Most of the variables are short strings and they're fine. The problem comes when I try to pass a service account with a secret - it's a multi-line JSON data in effect and the bash inside runner (the ubuntu-latest OS of the runner) fails to interpret it as such and rewrite it to proper yaml equivalent due to the special characters (the ,s and :s that delimit the JSON data). I've tried a bunch of methods of escaping but everyone failed for its specific reasons. The transition to key.json files failed for similar reasons. Then I tried with flags file (with the file being generated within the action just before the deployment step. Again - that failed because of the multi-line nature of the service account data. Now I'm forced to try the bease64 encoding method. I was avoiding it because I didn't want to touch the application code. But if it does the trick I can live with that. Cheers!
Beautiful video. I was following the official method through the gcp yt vid and the cloud run gh action but it was convoluted and didn't talk about how to set the permissions. I tried getting my gha ction authenticated through their recommended federated identity method but it didn't work. Your service account key method is much simpler to configure. I found your vid and after 3 hours, got my deployment working. Thank you sir! Btw at the very end, "--platform" is a nonexistent argument and the gh action throws an error about it.
@skyhappy Great to meet you! Can you share what line of code in the linked repo you are referring to? github.com/thaddavis/how-to-deploy-a-dockerized-fastapi-to-google-cloud-run/tree/part_4 Hopefully I can fix it : ) Also can you share the timestamp in this video where this error is mentioned
So happy you shared this Also @skyhappy, what version of the gcloud cli are you using? ie: gcloud --version Google Cloud SDK 470.0.0 I see in this version does have a CLI argument option for `--platform`... gcloud run deploy [[SERVICE] --namespace=NAMESPACE] [--async] [--concurrency=CONCURRENCY] [--container=CONTAINER] [--ingress=INGRESS; default="all"] [--max-instances=MAX_INSTANCES] [--min-instances=MIN_INSTANCES] [--platform=PLATFORM; default="managed"] [--service-account=SERVICE_ACCOUNT] [--tag=TAG] [--timeout=TIMEOUT] [--no-traffic] [--args=[ARG,...] --command=[COMMAND,...] --cpu=CPU --depends-on=[CONTAINER,...] --memory=MEMORY --port=PORT --[no-]use-http2 --clear-env-vars | --env-vars-file=FILE_PATH | --set-env-vars=[KEY=VALUE,...] | --remove-env-vars=[KEY,...] --update-env-vars=[KEY=VALUE,...] --clear-secrets | --set-secrets=[KEY=VALUE,...] | --remove-secrets=[KEY,...] --update-secrets=[KEY=VALUE,...] --image=IMAGE | --source=SOURCE] [--clear-labels | --remove-labels=[KEY,...] --labels=[KEY=VALUE,...] | --update-labels=[KEY=VALUE,...]] [--connectivity=CONNECTIVITY --clear-config-maps | --set-config-maps=[KEY=VALUE,...] | --remove-config-maps=[KEY,...] --update-config-maps=[KEY=VALUE,...]]
@@cmd_labs 1 - timestamp is 2:52 I believe 2 - I cant link the repo because of spam filter, the file path in that repo is how-to-deploy-a-dockerized-fastapi-to-google-cloud-run/.github/workflows /cicd.yaml, line 47. I'm not sure if the gcloud cli version is relevant as it runs in a github action runner, which should download the latest version? line 31 in the above file uses "'google-github-actions/setup-gcloud@v2'".
@@cmd_labs Man, I replied to you but my comment got delted by spam filters. didnt even give a link. in that repo's github workflow folder, in cicd.yaml, line 47 is what I'm talkign about. the cli version is from whats downloaded from the github action step in line 31? the gcolud sdk library? I got a error in gh action saying that --platform argument wasnt valid so I removed it and the error was gone.
@@skyhappy The internet ahh : ) Interesting cause I don't get any errors thrown by this line 🤔 Yes I was wondering if something about the environment used by your GitHub account for running the action was different What region of the world are you in btw? Sounds like you got through it tho... I'll keep an eye out to see if I can reproduce this Great to meet you and I greatly appreciate the feedback : )
I find this to be a vague question Can you provide more details? I would recommend you test what you are trying out and let us know how it goes Learn by doing If you want to share the link to your GitHub with your code please do ; )
Shout out to @IvanPopov for taking this video to the next level in this blog article: polystack.blogspot.com/2024/10/passing-credentials-to-gcp-cloud-run.html
In this article, Ivan shows a simple technique for passing ENV vars/secrets to Cloud Run containers in CICD.
🔥
My man don't even care his key got public because he understands
Very nice. I only wish you addressed one the trickiest parts that may arise in that process: passing the JSON (through a secret) of another service account to the container through the --set-env-vars property.
Great to meet you @IvanPopov!
I can include a NOTE in the video description and/or GitHub repository and shout you out with this extra tip you are recommending
Can you include the command and a little note of why it's so useful?
@@cmd_labs Generally the command is just like yours (the last command in your cicd.yaml file) except that at the end it uses a few consecutive --set-env-vars flags. In those flags I'm passing the environment variables required by the application. For the values of these variables I'm passing some secrets. Now for the regular one-line variables (like bucket names, volume mounts, etc.) the values are being passed just fine. But one of the values is the key that represents a service account, i.e. that value is a multi-line JSON data (exactly like the one you defined in the video). The Github environment (the ubuntu-latest runner) has a problem with that for parsing it in yaml and is giving me hell for the past two days. I tried many sorts of escaping. Today I tried the flags file approach with the same results. It's all about multi-line secret JSON value not being parsed properly into the yaml. Next I'm trying (as much as I wanted to avoid it because I don't want to touch the application code) is the base64 encoding approach.
For brevity I'm pasting here the relevant part of my workflow (the variant with flags file) although I doubt the format will be preserved in the comment ...
# Setup Google Cloud SDK (no need for additional authentication here)
- name: Setup Google Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
project_id: ${{ secrets.GCP_PROJECT_ID }}
- name: Create Flags File
run: |
cat
@@cmd_labs Weird - I wrote a detailed answer about two hours ago but it's gone now ...
@@IvanPopov Rewrite please if you like : )
Maybe make it a bit more concise
Maybe it was meant to be this way : )
@@cmd_labs OK. Let's see ... my command is not too different than the last one you're using in your cicd.yaml descriptor. I'm adding a several --set-env-vars flags for the variables I want to pass to the container. The values of those variables come from the secrets I defined, the same way you did for the service account you're using for the authentication. Most of the variables are short strings and they're fine. The problem comes when I try to pass a service account with a secret - it's a multi-line JSON data in effect and the bash inside runner (the ubuntu-latest OS of the runner) fails to interpret it as such and rewrite it to proper yaml equivalent due to the special characters (the ,s and :s that delimit the JSON data).
I've tried a bunch of methods of escaping but everyone failed for its specific reasons. The transition to key.json files failed for similar reasons. Then I tried with flags file (with the file being generated within the action just before the deployment step. Again - that failed because of the multi-line nature of the service account data.
Now I'm forced to try the bease64 encoding method. I was avoiding it because I didn't want to touch the application code. But if it does the trick I can live with that.
Cheers!
Beautiful video. I was following the official method through the gcp yt vid and the cloud run gh action but it was convoluted and didn't talk about how to set the permissions. I tried getting my gha ction authenticated through their recommended federated identity method but it didn't work. Your service account key method is much simpler to configure.
I found your vid and after 3 hours, got my deployment working. Thank you sir! Btw at the very end, "--platform" is a nonexistent argument and the gh action throws an error about it.
@skyhappy Great to meet you! Can you share what line of code in the linked repo you are referring to?
github.com/thaddavis/how-to-deploy-a-dockerized-fastapi-to-google-cloud-run/tree/part_4
Hopefully I can fix it : )
Also can you share the timestamp in this video where this error is mentioned
So happy you shared this
Also @skyhappy, what version of the gcloud cli are you using?
ie: gcloud --version
Google Cloud SDK 470.0.0
I see in this version does have a CLI argument option for `--platform`...
gcloud run deploy [[SERVICE] --namespace=NAMESPACE] [--async]
[--concurrency=CONCURRENCY] [--container=CONTAINER]
[--ingress=INGRESS; default="all"] [--max-instances=MAX_INSTANCES]
[--min-instances=MIN_INSTANCES]
[--platform=PLATFORM; default="managed"]
[--service-account=SERVICE_ACCOUNT] [--tag=TAG] [--timeout=TIMEOUT]
[--no-traffic]
[--args=[ARG,...] --command=[COMMAND,...] --cpu=CPU
--depends-on=[CONTAINER,...]
--memory=MEMORY --port=PORT --[no-]use-http2 --clear-env-vars
| --env-vars-file=FILE_PATH | --set-env-vars=[KEY=VALUE,...]
| --remove-env-vars=[KEY,...]
--update-env-vars=[KEY=VALUE,...] --clear-secrets
| --set-secrets=[KEY=VALUE,...] | --remove-secrets=[KEY,...]
--update-secrets=[KEY=VALUE,...] --image=IMAGE | --source=SOURCE]
[--clear-labels | --remove-labels=[KEY,...] --labels=[KEY=VALUE,...]
| --update-labels=[KEY=VALUE,...]]
[--connectivity=CONNECTIVITY --clear-config-maps
| --set-config-maps=[KEY=VALUE,...] | --remove-config-maps=[KEY,...]
--update-config-maps=[KEY=VALUE,...]]
@@cmd_labs
1 - timestamp is 2:52 I believe
2 - I cant link the repo because of spam filter, the file path in that repo is how-to-deploy-a-dockerized-fastapi-to-google-cloud-run/.github/workflows
/cicd.yaml, line 47. I'm not sure if the gcloud cli version is relevant as it runs in a github action runner, which should download the latest version? line 31 in the above file uses "'google-github-actions/setup-gcloud@v2'".
@@cmd_labs Man, I replied to you but my comment got delted by spam filters. didnt even give a link.
in that repo's github workflow folder, in cicd.yaml, line 47 is what I'm talkign about.
the cli version is from whats downloaded from the github action step in line 31? the gcolud sdk library?
I got a error in gh action saying that --platform argument wasnt valid so I removed it and the error was gone.
@@skyhappy The internet ahh : )
Interesting cause I don't get any errors thrown by this line 🤔
Yes I was wondering if something about the environment used by your GitHub account for running the action was different
What region of the world are you in btw?
Sounds like you got through it tho...
I'll keep an eye out to see if I can reproduce this
Great to meet you and I greatly appreciate the feedback : )
thank you and have a doubt do we need to use docker or it just works without it ?
I find this to be a vague question
Can you provide more details?
I would recommend you test what you are trying out and let us know how it goes
Learn by doing
If you want to share the link to your GitHub with your code please do ; )