Really nice video! As you say, in a big company, most of the obstacles are access related. In my case, I wasn't able to create a feed or even upload to an existing one due to not having permissions 🤣, but up until that point I got everything else working just fine with your help! when I get unstuck I'm sure I can continue with the rest of the pipeline
Do you know how I would you set up that pip.ini on the azure pipeline build server? I want to deploy my custom python library to my azure function app. I have a azure build and release pipeline to push my function apps to the server but it would obviously fail if i try to pip install a custom python library without the pip.ini.
When using poetry to install dependencies from an artifact feed, I get authentication failures, even when adding the feed with a PAT. Permissions are also set correctly, as far as I can tell. Any tipps?
I got the error 403 error and tried to add project-scoped build but I couldn't find the three dots option in the feed settings. Any way I can resolve this issue? Thanks :)
Hello! Thank you for the video, it was very useful. I have a question: If my repo contains many packages and sometimes packages that use other packages. How do I build them into artifacts using one pipeline?
This video is great - thank you for the clear explanation and actual walkthrough. Would you do a video or give instructions of how to add the credentials (min 23:05) into a conda.yaml or requirments.txt file that is then pushed to Azure ML with Environment.from_conda_specification to create an AMLS environment? Thanks in advance.
Thanks for the video mate, very useful!! I have one question though, how do you manage automating the version number of your package, without manually modifying it each time in the setup.py? I have seen that it's possible to do it with bump2version, but I didn't find a way to integrate it to my CI Pipeline.
I am facing the issue below when running the pipline ##[error]No hosted parallelism has been purchased or granted. To request a free parallelism grant, please fill out the following form (I removed the link because youtube keeps deleting my comment) Pool: Azure Pipelines Image: ubuntu-latest Started: Today at 2:20 PM Duration: 12m 5s According to stackoverflow, this is due to a restriction imposed by Microsoft on both private and public projects. Nevertheless, I am seeing some recent comments here from people who seem to have not faced this issue. I am not sure if you came across this issue before or if you have a fix for it (other than contacting Microsoft's team directly)
@@RawCoding I think you meant a private project. If so, I do in fact have a private project. This StackOverflow link (a/66791725/1831518) , mentions that private projects might face this issue as well. As a temporary workaround, I built the package locally (like you did at the beginning of the video) then used twine to upload to Azure artifacts. After that, I followed along your tutorial and it worked like a charm so many thanks. Now I am going to try to import the library into Azure Functions and see if it works
Hi Raw Coding, thank you for the post. I am having some issues with CmdLine. I keep getting the following error: Forbidden - User '6aad9f02-8bee-4eac-98d3-76f400db356c' lacks permission to complete this action. You need to have 'ReadPackages' or 'AddPackages'. Your assistance will be greatly appreciated.
Thank you so much. I found the issue. Limit job authorization scope to current project for non-release pipelines was disabled. I enabled it and now I am was able to deploy pipelines. I really find your tutorial very rich in content. Greatly appreciated by all here I am sure.
@Raw Coding - its really a nice video explaining a lot of details , but in the scenarios where we need to install the python package directly on build server in Azure DevOps , how can we achieve that ?
@@RawCoding how can we create a virtual environment on MS provided build server and then create/update an ini file over there and install it . Apologize to put this direct questions but I am searching this for a while and didn't got any corect answer. I want to install python package from Azure Artifact on build server . And then use it for further processing.
@@RawCoding sorry I am not getting it , can you help me with an example or an article which I can follow .. I am new to this stuff . Its an MS build agent.
Thank you for a very informative video. I am stuck at the uploading stage. My artifacts are very small in size: 16-18K, there's two of them in my dist folder, one is the actual tar.gz and the other one is ending in py3-none-any.whl. I am doing twine upload --verbose but it is not giving me much in terms of extra info. It is just spinning forever trying to upload these two small files. I was wondering if you could suggest anything. Thanks!
@@RawCoding Thanks, will do. But earlier I was getting a 404, now I am not, which sort of suggests it finds the URL ok now. Permissions need to be checked, yes.
Hi, I confirm that this was a permission issue. In the pypirc file in the section where I point to the feed as per video, I had to specify on a separate line username: and on the next line password: . Great stuff, thanks very much, Raw Coding!
In CmdLine he asks me to authenticate myself. "To log in, use a web browser to open the page..." How can I fix this so that in CmdLine does not ask for authentication?
@@RawCoding my file name is always the same: , and that is referenced in pyproject.toml as: name = "package-name" and version = 1.0.1, which I have changed from 1.0.0. I can't see any other places where you should change the file name.
@@fc1984fc I had a similar issue when I released a version as an artifact and attempted another release. In my case, I had to bump the version number in the setup.cfg, so check you haven't also defined a version number elsewhere that might be accidentally getting picked up.
if you set the path for azure artefacts globally, will Python look for the package in the public pip index first? I am wondering whether it makes for me for all packages to always be downloaded from Azure and if not present, then failed.
Hello, Good video explanation. I have one question here, I have a requirement in my project that we publish python packages like pandas, NumPy for ML projects, can I make a bundle of all these modules and publish it as a single artifact? I am new to python as well. Need some help here. Thanks
Hi @bhubenp5215, did you find the way to achieve this requirement. Actually I am struggling to achieve this currently in my project. please let me know if you the answer. I will connect with you. Thanks!
Really nice video! As you say, in a big company, most of the obstacles are access related. In my case, I wasn't able to create a feed or even upload to an existing one due to not having permissions 🤣, but up until that point I got everything else working just fine with your help! when I get unstuck I'm sure I can continue with the rest of the pipeline
Thanks bro, 2 years old video, but still helpfull!
Amazingly clear and to the point, have my thumb up you good man.
thank you for watching )
Clear & concise!
This helped me a lot. Thanks!
Thanks
Thank you so much for teaching this kind of topic!
amazing video, so clear in each and every step.
Cheers
Very nice and superhelpful. And very funny at the end with the "pring" ;)
Nice. Had some change on Azure Portal but the example work yet!!! Tks.
Hi @davidsonsilva7636 please can you tell me I am stuck where dependencies are failing
Thanks for the video ! Very clearly explained and a lot more helpful
Thank you for watching
Thanks a lot !!!!
Clear and to the point
Thank you
this helped me a lot, thanks. You should do more Python tuts
What would you like to see about python?
@@RawCoding Please make videos on stuff related to azure devops using python..😊
Thank you very much! this video has been very useful, great job! :)
Do you know how I would you set up that pip.ini on the azure pipeline build server? I want to deploy my custom python library to my azure function app. I have a azure build and release pipeline to push my function apps to the server but it would obviously fail if i try to pip install a custom python library without the pip.ini.
MORE PYTHON :) great video
Keep an eye out )
thank you! it is a great manual
thank you for watching
Amazing !!
Super helpful video, thanks for explaining! Quick question, did you ever explain why you set the scope to organization instead of project at 15:10?
Amazing! Thankyou.
One question, how can we have automatic versioning for the build artifacts?
You could use something like gitversion.net/
Best video ever!
glad you liked it!
When using poetry to install dependencies from an artifact feed, I get authentication failures, even when adding the feed with a PAT. Permissions are also set correctly, as far as I can tell. Any tipps?
I got the error 403 error and tried to add project-scoped build but I couldn't find the three dots option in the feed settings. Any way I can resolve this issue? Thanks :)
Hi, Nice video, But do you have any idea on 400 Request - Could not parse PyPI version ?
No clue
Very clear thank you !
Super helpful! Tanks!
I'm assuming that all the shenanigans happening before the install is successful need to happen for any user wanting to install this package. Correct?
Yes, you are connecting to a private feed (private server)
Hello! Thank you for the video, it was very useful. I have a question: If my repo contains many packages and sometimes packages that use other packages. How do I build them into artifacts using one pipeline?
You’ll have to specify those local packages as dependencies in the toml or setup file.
This video is great - thank you for the clear explanation and actual walkthrough. Would you do a video or give instructions of how to add the credentials (min 23:05) into a conda.yaml or requirments.txt file that is then pushed to Azure ML with Environment.from_conda_specification to create an AMLS environment? Thanks in advance.
Heya not sure what conda is but you’d place you credentials (secret) in the azure library variables. And then you can use that variable in a pipeline
Thanks for the video mate, very useful!!
I have one question though, how do you manage automating the version number of your package, without manually modifying it each time in the setup.py?
I have seen that it's possible to do it with bump2version, but I didn't find a way to integrate it to my CI Pipeline.
There are azure pipelines parameters that you can use, for example the build number
Good one
Cheers
after restart the server got an issue like import error? and need some external command in SSH then it works so please guide me
Where would you like to go?
@@RawCoding got failed to building wheel for dlib.
what i need to do if I need to add other dependencies for ex: azure packages/requirement.txt file which i have stored inside src/example.
Sorry I don't quite understand, you just want a requirement.txt for dependencies in you package?
Nice video.
Спасибо!)
И тебе )
Why did you build the package locally when the code you committed does not contain the dist-info folder? Was it just to show how it's done?
It’s a demonstration, if you can do something locally you can make a remote computer do it as well.
I am facing the issue below when running the pipline
##[error]No hosted parallelism has been purchased or granted. To request a free parallelism grant, please fill out the following form (I removed the link because youtube keeps deleting my comment)
Pool: Azure Pipelines
Image: ubuntu-latest
Started: Today at 2:20 PM
Duration: 12m 5s
According to stackoverflow, this is due to a restriction imposed by Microsoft on both private and public projects.
Nevertheless, I am seeing some recent comments here from people who seem to have not faced this issue.
I am not sure if you came across this issue before or if you have a fix for it (other than contacting Microsoft's team directly)
Check that you do NOT have a public repository.
@@RawCoding I think you meant a private project. If so, I do in fact have a private project. This StackOverflow link (a/66791725/1831518) , mentions that private projects might face this issue as well.
As a temporary workaround, I built the package locally (like you did at the beginning of the video) then used twine to upload to Azure artifacts. After that, I followed along your tutorial and it worked like a charm so many thanks.
Now I am going to try to import the library into Azure Functions and see if it works
I’ve only seen that issue with public repos if they happen for both that’s a bit of a bummer
@@RawCoding ok that sucks, I have it in a private repo
Hi, I guess python packaging documentation has changed, and there is no need to create a setup file anymore, right?
Could you tell which agent are you using? is it a self hosted or MS agent?
Think it was ms can’t remember
Hi Raw Coding, thank you for the post. I am having some issues with CmdLine. I keep getting the following error:
Forbidden - User '6aad9f02-8bee-4eac-98d3-76f400db356c' lacks permission to complete this action. You need to have 'ReadPackages' or 'AddPackages'.
Your assistance will be greatly appreciated.
Looks like the token you generated for the login doesn’t have the permissions. Create a new token
Thank you so much. I found the issue. Limit job authorization scope to current project for non-release pipelines was disabled. I enabled it and now I am was able to deploy pipelines. I really find your tutorial very rich in content. Greatly appreciated by all here I am sure.
@@RawCoding
You are a life saver :)
No worries, consider saying thank you by supporting me on patreon.
Can you please explain in detail what exactly you did in Limit authorization scope to current project for non-release pipeline.
PRING!!!!! 😂
@Raw Coding - its really a nice video explaining a lot of details , but in the scenarios where we need to install the python package directly on build server in Azure DevOps , how can we achieve that ?
We do it the same way we did with packages required to build our package.
@@RawCoding how can we create a virtual environment on MS provided build server and then create/update an ini file over there and install it . Apologize to put this direct questions but I am searching this for a while and didn't got any corect answer.
I want to install python package from Azure Artifact on build server . And then use it for further processing.
You do it the same way you do it locally
@@RawCoding sorry I am not getting it , can you help me with an example or an article which I can follow .. I am new to this stuff .
Its an MS build agent.
sudo apt-get -y install python3-virtualenv
virtualenv -p python3.7 venv
source venv/bin/activate
Thank you for a very informative video. I am stuck at the uploading stage. My artifacts are very small in size: 16-18K, there's two of them in my dist folder, one is the actual tar.gz and the other one is ending in py3-none-any.whl. I am doing twine upload --verbose but it is not giving me much in terms of extra info. It is just spinning forever trying to upload these two small files. I was wondering if you could suggest anything. Thanks!
If it’s spinning I’d expect it to be a connection issue. Double check url/credentials
@@RawCoding Thanks, will do. But earlier I was getting a 404, now I am not, which sort of suggests it finds the URL ok now. Permissions need to be checked, yes.
Hi, I confirm that this was a permission issue. In the pypirc file in the section where I point to the feed as per video, I had to specify on a separate line username: and on the next line password: . Great stuff, thanks very much, Raw Coding!
In CmdLine he asks me to authenticate myself. "To log in, use a web browser to open the page..." How can I fix this so that in CmdLine does not ask for authentication?
you should either pass the credentials in the pypirc file or use the default pypirc created by twine authenticate
permission block
If you want to release a new version of the package, can you use the same feed, or do you need another one?
Same feed, just change the name of the package you publish.
@@RawCoding I get this error: Conflict - The feed already contains file
'-1.0.0-py3-none-any.whl' in package '
File name
@@RawCoding my file name is always the same: , and that is referenced in pyproject.toml as: name = "package-name" and version = 1.0.1, which I have changed from 1.0.0. I can't see any other places where you should change the file name.
@@fc1984fc I had a similar issue when I released a version as an artifact and attempted another release. In my case, I had to bump the version number in the setup.cfg, so check you haven't also defined a version number elsewhere that might be accidentally getting picked up.
if you set the path for azure artefacts globally, will Python look for the package in the public pip index first? I am wondering whether it makes for me for all packages to always be downloaded from Azure and if not present, then failed.
Exactly this happens if you follow the example. However, there is a way using the extra index config instead of index.
Anton began to choke python 😱
I choke many things ;)
Hello, Good video explanation. I have one question here, I have a requirement in my project that we publish python packages like pandas, NumPy for ML projects, can I make a bundle of all these modules and publish it as a single artifact? I am new to python as well. Need some help here. Thanks
Hi @bhubenp5215, did you find the way to achieve this requirement. Actually I am struggling to achieve this currently in my project. please let me know if you the answer. I will connect with you. Thanks!
Hi @bhumibenp5215
4:20 why you speak about java and other non python? video about python. probably java user look to different videos. with "java" in name
thank you
The concept of a “project” exists in all languages hence I mention it here in case you are aware of them.