Combining Rust and Python: The Best of Both Worlds?
Вставка
- Опубліковано 22 тра 2024
- In this video, I'll show you how to seamlessly integrate Rust with Python using Pyo3. This library allows you to write Python modules with Rust. This means that we get the speed and safety of Rust along with Python's easy-to-use features!
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
🔥 GitHub Repository: git.arjan.codes/2024/rust_bin...
💻 ArjanCodes Blog: www.arjancodes.com/blog
✍🏻 Take a quiz on this topic: www.learntail.com/quiz/euyfat
Try Learntail for FREE ➡️ www.learntail.com/
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
Social channels:
💬 Discord: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
📱Instagram: / arjancodes
♪ Tiktok: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
- Kit Hygh
- Alexander Milden
- Bean
🎥 Video edited by Mark Bacskai: / bacskaimark
🔖 Chapters:
0:00 Intro
1:20 What is Pyo3?
3:56 What is Maturin?
5:14 Building and running the Pyo3
5:45 Writing our Rust code
8:06 Publishing our module
8:22 Bonus
10:03 Final thoughts
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
Babe, wake up. Arjan uploaded a new tutorial
Fair enough
I used Rust pyo3 to expose some validation tool logic our team wrote in Rust (well, me) to our Python scripts. A useful tip is you can specify a stub pyi file at the top level (alongside your toml files) to provide better type hint and editor support when you import them. Maturin picks these up when it builds the package as long as it's called [module name].pyi
I was going to ask about stub files, glad to know this is supported!
Nice! Gonna push my team to write some Rust around our python projects I guess
This is mind boggling I'm enthralled. The publishing seems so incredibly easy wow
Side note: In a way, it's actually Poetry which is non-compliant here. The standard format for pyproject.toml nowadays is defined by PEP 621, which seems to be used by pyo3, while Poetry still clings to their own legacy format.
poproject.yaml is the new generic format, so Arjan could've simply disabled Poetry plugin and look for a generic one for pyproject files.
That's right! It always bothered me with poetry. According to Maturin project page they comply with PEP 621. But TBF Arjan actually has incorrectly specified dependencies, they should be an array of strings. Maybe Maturin doesn't care and still reads it correctly but that's why Python VS Code extension is showing an error
This is awesome! I’ve been waiting for a video like this. Thanks for the great overview 🎉
Im glad you liked the content!
Great video! A follow-up discussing/showing real life problems that can solved with combing Python and Rust together. Boosting performance, increasing safety etc. Personally I'm currently working a lot with async, websockets and JSON messages in python, any significant and potential performance rewards would be great to see.
Really great video and easy to follow guide! When i used pyo3 i decided to make a proc macro the help make wrappers for struct impls so you could basically just add it for python support with little to no hassle. Can definitely say that there are ALOT of limitations on pyo3 that aren't directly visible, maybe a video on these things would be awesome and help some newer people to pyo3!
Thanks so much! Very clear! Will definitely give it a try!!
Glad the video helped you!
Hey this is awesome. I actually picked up Rust recently and it’s been going great. Great timing to see this a month later when I’ve gotten a little more experienced. Great video!
Glad you enjoyed the video!
Amazing video as usual! :)
What do you use to have auto-completion on your terminal? :D
Nice video thanks, I used maturin but didn't know about rustimport. Python and Rust are my favorites language, I also think that they fit well together.
Is there some direct benefit of defining your python classes as Rust structs? I can understand that doing number crunching would be useful but not sure about the defining of objects (only). Would be interesting to hear your thoughts on what gives the most “bang for the buck” when combining the languages like this.
Thanks for interesting videos.
this is very nicely explained! Question how do we add python typehints to rust functions?
Looks promising. But how about the interaction between Python and Rust regarding memory management? 🤔
Maybe show an example where you parse a numpy array and perform some transformation on it.
I'm just using the standard `difflib.SequenceMatcher` and for sentences and snippets of code it was just fast enough. But atm I'm trying to throw some 1000s lines of JSON against it and want to display it in some Qt text fields. It's already MUCH faster in Python 3.11 and 12 but might still be a couple seconds! I can imagine that this would be a breeze to do with a little Rust plugin!
Ive been slowly incorporating rust through polars which uses pyo3. Im pretty convinced at this point replacing pandas for polars is just better. The api handles the same, its faster and infers column types (this is huge). The fact that these apis just copy sql operations and facilitate working on dataframes make the switch to polars all the more obvious.
How do cross-platform builds work when you publish the project to pypi?
If rust is more efficient, I might put some bigger/slow projects into rust and see if I can trim the time down. Thanks for the info!
You’re welcome!
This is GREAT! That's why I love Python. It's so easy to integrate with other stuff to overcome limitations of Python. Thank you for this video
Glad you enjoyed t!
Love it .. going back to c after learning python from you.. today i learned how to cheat c compiler.
I was coding for luhn check for three days. Today i made a dumb change in the code by suppying an argument which i assigned to something but not used. The compiler wasnt happy and error parameter not in use. So what i did was i just make a variable int error=0; and then error += s;
The error was gone programe compiked 🎉🎉🎉🎉🎉. Tomorroelw i will work on how to get rid of the error variable 😂😂😂😂3
Thank you....
Yaaay! :) Rustimport is pretty interesting for cross platform support... publishing compiled binaries for many platforms is a bit hectic
Hey Arjan, I'm curious if you see yourself writing rust as a day-to-day? what areas would you apply it to? love the rise of rust tutorials on your channel :)
I feel the same way. With Go or similar languages it's easy to find use cases, but Rust ... not so much
@@LieberLois I just used Rust for an embedded project using an E-Ink display on an ESP32, but I've also used Bevy to make games, Leptos for websites, and Clap to create command-line tools.
@ArjanCodes does the original module written in Rust can still be used as a rust library as well or now after importing py03 it can only be used by Python?
Also does it give the same performance benefit of rust when we ran it as a module from Python?
Have you looked into Julia Programming Language? What do you think about the multiple dispatch?
Wow, great tutorial, I always wanted to rewrite the core routines in rust for my project "pyTermTk" (a TUI Lib)
What about cross compilation?
Does Maturing take care of it?
One of the things that prevented me to use rust or c/c++ is that I don't know if I would be able to publish a package compatible with different architectures.
I can't speak for C/C++ but in Rust you publish your package as a "crate" which can then be freely used by other devs. It will be compiled for their architecture when they download and use it (usually with Cargo).
Can you make one for Go and Python too?
Please do more rust tutorials 🙏
Is it possible to run python code from rust as well?
what is the advantage over c++ modules
Are you going to create a rust programming course?
what about Cython? i think is less complex and better in performance terms
This adds a significant amount of complication to a python project. What I did not see is any measurement at all of how much speed up inserting less modules could achieve! How do I know if I should bother with all this, if I have no idea how much speed up it might get me?
I'd be curious to see how Cython compares to this
Time to have chat gpt write me rust modules and scripts to import into my projects, perfect
Whatever they do, name the result "Rython". Silly as that name may sound, it sure as hell beats programming in "Pust".
I would call it: Ruthon
a litlle more credit for Rust 🤣😂😁
does this give python the performance of rust?
Yes, but with certain caveats. Moving objects back and fourth into python can be "slow"-ish. The more/bigger chunks of work you can shift over the boundary into rust, the bigger the benefits :)
no
Julia combines both in one language
this would create a dependency on an extra toolkit i.e. rust
It looks great! But as an amateur python programmer I don't think I am going to use it. I don't know anything about rust so I have to learn a new language from scratch. When I have done that, I have to understand when it is usefull to to use rust. So I stick by python, if you don't mind....🙂😇
I will use this for sure. Do you need to think about accidentally spawning new processes?
Make armorers great again
If C++ can be pronounced as Sepples
C++-ians can also be called Sepplesians
mojo is an existing superset of python which is already 50% faster than rust
I haven’t tried rust yet. But I might give it a go. But,I haven’t tried go yet but I will wait and see. I haven’t tried c yet.
I’d rather just call Rust libraries (like polars) from Python 😂😂
Nah, I'll wait for Mojo.
the thing i hate about rust is that you can only get blockchain job with it
We want the two decade experiance of " How would I do it if i start to learn software engineering today " video, Am sure the internet will expolde if you make such video because of the experiance you have.
You should start when you’re 10
I was thinking how you can reverse the rusty python pun. Like pythony rust or pythoned rust but it won`t make any sense
Read "maturin init" in a Bri'ish accent for extra lols.
Nice but not convinced. It you really want Python + Performance - you are better off giving Mojo language a go..
I agree that in general this would be the better approach, problem is mojo is missing a lot of features and is atm just a proof of concept, but not a full language.
Also you should stop thinking in just performance, rust offers memory safety.
Fancy words like memory safety, superman performance blah blah are delusional. The lucky enough people who work at netflix level software - you know, the 0.0 something percent - need to worry about memory safety and are normally not watching introduction videos on UA-cam. The rest of us - mental model shift is a necessary consideration for our day to day jobs.
@@squishy-tomatoIf you write something in rust vs python, you need less resources, if you are in a cloud, could save you some money and the less power we use, the less carbon we put into the air. ;-)
Why would we use rust when we can do all of this in Python?
More like the worst of both languages
I am just not sold on rust... Its likely that AI will just be able to read your C code and find menory vulnerabilities, use after free, buffer overflows... When that happens, i think rust will just fade a lot...
If you haven't already, give Rust a try! It's a lot easier (and more fun) writing code that just doesn't have those memory vulnerabilities in the first place.
Arjan simply does not explain things clearly or coherently. People just like his voice.
It smells bad to me if it uses a non compliant pyproject.toml
Why not use another name?
Babe, wake up. Arjan uploaded a new tutorial