You missed that Julia can use all the Python libraries just like there were native Julia libraries, but it goes further and can also use R, C++, Fortran, and Java libraries too as easily as Python in the case of C++ and almost as easily for the other languages. So, one of your big selling points for Python is not actually a selling point versus Julia.
@@MrEo89 its not moot, if python is slower than Julia. Calling Python from Julia will not slowdown Julia as much, but calling Julia from Python WILL incur a performance penalty.
As a pure science background (biology), I like programming language with high level performance in any OS and also have strong types system. That's why I choose Julia. Julia combine both concept of Java (JIT) and C/Rust (LLVM Compiler). I think Julia has a bright future for scientist. Great video. Thank you🙏
it's interesting how Julia on board looks so modern and python's defense was "it has a huge community and lot of packages". Check Julia 1.9 first the plot is fixed :D
Julia is like a Python 4.0. Many libraries in Python are needed to fix the lacks of Python. Julia doesn't need libraries like Numpy, Numba, Cython, Cpython. Matrix multiplication? Already built in. And there are a lot of things I like in Julia, but you'll never get in Python. I like that every block is terminated by an end. In Python I get often confused by nested blocks. I like Julia's support of Unicode and Latex. Formulas are more like they look in math. Python is easy to learn, but if performance is needed, you have to do some extra mile. Julia, which is as easy to learn, write and read, doesn't need this effort. Therefore there are just 2 advantages of Python over Julia: much bigger community and much more written code. I hope these advantages will decrease in the future.
In fairness Python also has much better documentation. It is harder to learn Julia, not because it is more complex, but for bad documentation that is too abstract and lacks step-by-step examples.
Just think when Ai LLM's move from Python to Julia you will no longer be waiting 20 seconds for a response, it will be like 1.6 seconds. Plus, Julia works natively inside GPUs without issues or library calls.
A “problem” that prevents Julia to grow is the lack of support for…newbies! Python is taught for kids, teens. It’s everywhere. It’s used for practical things like office automation, web scraping, solve problems for regular people and daily needs. Julia’s developers seems to focus only on big tasks, math, engineering…but dont evangelize young people, they don’t advertise Julia as a fantastic general purpose tool. If they want to make Julia get users, get traction, they should invest more in the “grassroots”, not only the math/engineering/computing elites.
@@valdemarkatayamakjaer9304I agree that python is a good language to start with, python was one of my first languages. Though, python being an all purpose language is one of multiple reasons that makes it slow. Unless you're working in a low(er) level language (C, C++, Rust) you cannot expect performance and all purpose functionality. Julia focusing on maths, AI, engineering etc is the way to go. All in all, community and/or funding is the main driving factor for getting a language off the ground, not ease of use. Take Rust as an example, it is difficult to learn and I would not reccomend a beginner learn it as their first language, but rust has become quite popular.
Great video! Thank you for mentioning multiple dispatch, a real game changer. I think your comparison is fair to both languages. Looking forward to upcoming videos.
You make it sound like Julia cannot use 3rd party libraries for various tasks. I don't believe this is true, but someone naive to coding would think that was the case from what you said.
Maybe I was not paying attention but I don't remember any reference to the main difference between Python and Julia: namely that Python is interpreted and Julia is compiled just-in-time.
I just want to point out that Julia has a lot libraries. I learned Julia before python. Python's syntax can be confusing. Also, Julia does not depend on white spaces so you can see the formatting. I have used Julia for signal processing for my acoustics work. Julia does not have a fragmented packaging system like python and can be ran only basically any desktop operating system.
@@ClariNerd Languages that iterate like mathematics does, instead of thinking in terms of memory offsets, iterate from 1. that's called "abstraction" - forget about machine representation.
Totally agree. I hate Python just for that. You really need to use an IDE or you'll lose your mind. Julia is easy to program with any editor. It's going to be my goto language.
It not really true if you say that Julia is “dynamically typed language” because Julia in contrast to Python can be use as “dynamically and statically typed language”. It depends on the user which approach he will choice. Special for the large scale Application it is the key difference.
Nice video. But I think you have the point about extensibility backwards. Python’s lack of extensibility has forced, for example, multiple projects to write their own custom NumPy implementations from scratch (e.g. jax.numpy). The basic problem is that in Python there is no way to extend a given function onto a type that you don’t own. This extensibility is one of the prime selling points of Julia. In fact, the most recent criticism of Julia that sparked a lot of discussion was that Julia is _too_ extensible and that the culture of extensibility has led to correctness bugs in popular third party libraries. I think the point you were making wasn’t necessarily about extensibility but how “batteries included” each language is. On the continuum of “all functionality needs to be imported” (Python lives around here) to “all functionality is in the base language” (MATLAB lives around here), Julia is pretty decidedly on the Python side. The one notable difference is that Julia’s arrays have a concept of dimensionality baked in, so matrices and higher-dimension arrays are part of the core language. Personally, I’m a fan, because NumPy is hamstrung into some pretty clunky syntax around matrices. And it’s nice to not have to care about the difference between a list and a numpy.array. But I think those are more personal preferences. People seem to like NumPy, so I won’t tell them they’re wrong. I’ll note one last difference in that community in Python and Julia mean different things. In Python, community is generally a one-way street. You can search a question and almost always find an answer someone gave at some point in time. In Julia, you won’t always find a lot of answers by searching, but you can hop on the Discourse, Slack, Zulip, etc. and very quickly get an answer, as the community is very active. There are advantages and disadvantages to both approaches. Julia’s approach certainly isn’t going to scale well as the user base grows, but for now it feels kinda nice. The people who most often answer your questions tend to be working on similar problems as you, so industry-specific connections kinda happen naturally.
I realized something while watching this. Chat GPT is trying to mimic answers like these. Not sure if it will ever be as fun as people can make the experience. Thanks to IBM for all the efforts you make in educating the general public of new technology people like myself may never get a chance to experience or even hear about, much less the inner workings. Just had to add my sincere appreciation and couldn’t imagine a more trustworthy source of truth than the Experts at IBM. Thank you for believing in your audience. Shows I’m not alone in believing this with the numbers of views.
Python is "do anything" because it has, for some strange reason, accumulated enormous number of additional libraries. In most cases Python programs are little more than glue code for those libraries (usually written in C/C++ or more and more Rust.) Julia also has many good libraries available for it. Also, perhaps with a bit of glue code ("wrappers") one can call virtually anything from anything else, although having to rely on different virtual machines and/or runtimes will complicate things.
That's the beauty of Python, I can go from an idea to the first implementation in one evening, without spending an entire evening debugging the program. I can start with Idea, develop web server on Raspberry Pi in my home, make it do things, pickup data from net, process it with tensorflow lite model, connect that server with additional sensor from outside environment, and in the end display data on small custom screen from AliExpress 😄 - everything in day or two of hobby programming!! This is way Python rules, and it is irreplaceable!!
I think Julia is definitely the future of ML. It can do anything python can do at twice (or more) the speed, including python via ffi. It also doesn't need extensions for what should be basic functionality.
Scalability in terms of networking is the biggest issue in Python. Serializing / deserializing data and passing it to individual processes / nodes, across a distributed environment such a pain.
Also, I would add, that numpy is not as intuitive as the in-built matrix operators in Julia. Plotting in Julia is so much easier than with matplotlib. JAX which is the new package for ML by DeepMind in python is almost a clone of the paradigm of FluxML in Julia. They even have their own version of JAX numpy which is made to resemble the matrix operations in Matlab and Julia. Some quirky things in the Python syntax do not exist and in general that 'pseudo code abbreviations' in python are not always obvious, like in python it is 'len' but Julia it is simply 'length' spelling it out completely
I'm not a big fan of Python, but I much prefer the code abbreviations. Shorter is better when the meaning is clear. In any language it is necessary to learn the identifiers, and I would never assume 'length' rather than 'len' or 'size' or whatever. I would rather have 'enum' than 'enumerate' in Python, for example, and it would be more consistent with 'len'. There is no difference in clarity, but one is easier to type and keeps the code more compact, which I find easier to read. I think languages should be written for continuous use, not just first time use or teaching use. Perhaps starting in C on Linux/Unix made me prefer brevity.
The argument of extensibility listed under Python is very much also a Julia advantage. The ecosystem is not yet as large, but you may need only a small part of the ecosystem. For me it's been quick to start with CSV, NCDatasets, Dates, Makie, Images packages. There is a good deal of plotting options, but it was a bit hard to decide which one to learn first. The syntax and layout of Julia look a lot cleaner to me than Python. I have been using Scilab (a free Matlab alternative) for years, and IDL/PVWAVE before that. I am finding the transition to Julia very smooth. I actually don't mind the type annotations, especially for the speed gained. It is pretty much mandatory if you wish to input data into certain functions. In Julia you need to be careful about assigning variables to other variables, if you don't want to change those as well - this can lead to unexpected bugs you can lose hours with :)
I think the issue is the same in Julia and Python. a = b means a is just another label, not a different piece of data in a different portion of memory. This is a good thing. If you want you can do a = copy(b).
Mmmm Julia is dynamically typed by default but unlike Python real static typing is an option. Also, Python libraries are usable in Julia by default, that's not an advantage of Python
The real problem with Julia is the lack of a good debugger. I was using it a couple years ago and had to add print statements all over the place to trace down issues. It's like I was living in the 80's. Not good.
It was an excellent explanation on the application of Python vs Julia! Since, Julia is used for applications such as Artificial Intelligence and Machine Learning. Is it possible to make an video on how it is applied in these areas? I will be waiting for the Video!
Great video! Learning about the differences between Python and Julia is a great way to expand your knowledge and become a more well-rounded programmer.
Python is a lot of fun for hobbies and experimenting with new technologies because of the huge user community. But if there is any serious number crunching, it is so slow that it will make you commit suicide. It sounds like Julia is better for that. (But every language is faster than Python)
Python does support single and multiple dispatch. Singe dispatch is part of the Python distribution and has to be imported from functools; multiple dispatch has to be installed from the repositories.
In Julia you have full flexibility where you can annotate the type like x::Vector{Float64} = ... and get all the benefits of static typing (speed, input/output security, functions tailored to the input type) - where it is needed, and use dynamic typing for other things, as you wish.
The review is flawed, as Julia is capable of utilizing all Python libraries. For instance, if you run a Python-based LLM (AI) locally on an i7 8-core processor, it may take around 13 seconds to generate a response. In contrast, the same LLM implemented in Julia could potentially provide an answer in just 1.6 seconds. This performance advantage is one reason why organizations like the DOD and NASA are adopting Julia for various applications. Additionally, projects like the Next Generation Air Dominance System (NGADS) and future fighter aircraft featuring integrated AI piloting systems are also being considered towards a transitioning towards Julia or another in-house language built from Julia. Rewriting the James Web Telescope from C++ to Julia could yield significant benefits, particularly due to Julia's ability to leverage unlimited core and device access unavailable in other programming languages. Given 100% flex control on every moving piece through the actual manipulate of power flow cut off as fractional and part cycles. With that, Python has no advantage over Julia, which was original built around having its future adopters coming from the Python programming community.
Python is strong when there is an existing performant library for your use case. If this is not the case, custom Python code is very slow compared to Julia (often a factor of 100 slower in runtime). Optimizing Python code with Cython, Numba, etc. is possible, but rather cumbersome. In such a use case, Julia is a much better fit.
There's no 'replacing' in any concrete sense of the term when it comes to open languages like Python or C. It's mostly super niche proprietary langs that get 'replaced' at one point or another. And sometimes the real friction facing these migrations isn't from the actual use case at all. Like, a data engineer might have a hard time migrating to Julia because its web development scene is nowhere near that of Python. Your question was about Data Science specifically, but Data Science doesn't exist in a vacuum so there's that.
#Example, if you want to use #matplotlib directly using PyCall pl = pyimport("matplotlib") xs = -2pi:0.01:2pi ys = sin.(xs) pl.plot(xs.ys) #But in this case, you would prefer to call PyPlot
Is Julia ready for use in a production environment? Have used it years ago (v0.4/0.5), and contemplating on using it for analysis. How well are packages maintained and how often will implementations break?
Not only is it at version 1.8 with a best in class package manager. It has almost 9,000 packages downloadable. It supposedly has great threading capabilities. Currently, there is a big effort to make it support HARD real-time execution with the ability to control garbage collection scheduling so GC doesn't prevent responding to events in a specified number of milli or maybe micro seconds. This is coming incrementally in versions 1.8 and 1.9 and possibly some final work in 1.10. Part of this is Julia Computing, now Julia Hub, is giving abilities for static compilation so that you can do all the hard compilation work once and then when it runs, you will be hopefully able to eliminate the long "time to first plot", and instead have, basically a shipable .EXE file (or a few). Current talk seems to be about fully compiling and in 1.9 there will be caching of compiled instances of each module (package), but by version 1.10 Julia being able to have a single .EXE file (or an EXE plus one compiled library object). The details escape me and seem to be evolving. The beauty of all this EXE and real-time work is to get around Julia's famously slow time-to-first-plot startup time, measured in seconds or sometimes even minutes for code that relies on dozens of packages. Everything else is looking production ready. No breaking features after 1.5 and none planned.
I would say yes, but only very recently. Julia reached v1 (i.e. no break promise) in 2018 but most large packages (DataFrames, JuMP,...) did that only 1-2 years ago....
I am still trying to break into the IT even after my Btech graduation I am still struggling to get a good IT job. I have done certification in Data Science and business analytics just to find out that I have to be a python developer or a good statistian to become a data scientist. I am good at SQL and have created some projects in data science in python. Should I learn Julia too?
Once Julia is used in University textbooks, this will stsrt to grow. So, need to write a very good programming, web application and data science books with lots of excercise for students, then Julia will get popular
Really strange when a very simple program runs differently from Jupyter and the command line -- both Julia 1.10.3. x = 10 # x in the global scope for i = 1:5 x = i # x in the soft scope (inside the for loop) end println(x) # prints: 5 for Jupyter and 10 for command line
...I don't know much but, I'm just going to throw this out there: NumPy is a wrapper around Fortran code ...your not going to get faster than that, are you? what kind of performance hit do you take from the 'wrapper?' Nice video, thanks!
You've actually alluded to another issue quite nicely; if you learn Python and want to get into the nuts and bolts of the libraries you're using quite often you discover that you would have to have a working knowledge of Cpp or Fortran. Part of the motivation for developing Julia was to solve the, 'two language', problem.
While numpy is largely Fortran code, the API is pretty bad for performance. Numpy code ends up making a ton of copies and doing a bunch of redundant loops over the same memory. It also has a pretty high per call overhead (roughly 1us) which makes it incredibly slow for small array operations.
I really wanted to use Julia but it's packages were convoluted. Conflicts. Some are not build without any understandable reason. Had to abandon idea to use Julia. To my sorrow. Switched to Python. And it worked out of the box. Maybe will try Julia once more in a year ot two.
It's never important to me that my algorithm gets there quickly, I love slowness. I think it's great that IBM had this video, but it was more of a round about way to say yeah Julia is fast but python can do more.
When you try to calculate a numerical integral over 1M points, try to like such slowness. That's why I prefer Julia over Python. Julia is faster and can be parallelized easily.
Its funny how Amazon and their AWS company debunked that Julia was fast. As Python was tested against Julia in AWS and AWS said Python was much faster and even more economical when doing extensive tests with them. Julia takes up most electricity than python when tested in the AWS, which means AWS seen that Julia cost more to operate than Python. AWS and Amazon even created a 5 page website extensively going through each and every language and how fast and feasible they were to run within AWS Amazon services. Python won and beat every language including Java and C#
Dont forget that Guido Guido van Rossum is with Microsoft now and has enormous resources to acomplish whatever plans he has for Python . I would place my bet on Python primarily because of its extensive libraries. A long time ago, IBM developed PL/1 that is upposed to combine the best features of Fortran and Cobol but it never gained traction. Will Julia suffer the same fate? Time will tell.
Pretty sure they included some compile time in their Julia benchmarks. Short-running scripts are rather slow in Julia because they are compiled every time a new Julia instance is started. This may be an issue for some kind of webservices.
I've been programming Python since it was first released. I'd suggest the Amazon engineers are hacks applying a great language within a crappy framework designed for Python.
I'd take those benchmarks with a grain of salt. You can precompile code and libraries so that there is not much start up time and the code will run much faster than python. I'll bet that the Julia Jit compiler time got baked into those benchmarks.
I tried studying Python and my vision started to blur when the Walrus operator was introduced. I doubt I could debug my own Python program after a year. I really hate the forced indentation means of packaging as much as I hate C's curly braces. Sounds like what was said for these languages was also touted for Java. I guess I'll stick to REXX or Forth. I used REXX to rename files and the performance was dismal with each "address DOS ... " statement. So nowadays, my REXX programs write BATCH programs which are executed as fast as CMD can clear them. Not greased lightning like the big guys but I get the job done under my terms and control. I mainly use these to organize JPGs from digital cameras into a single folder/subdirectory. My programs rename each file with a 4 hex fingerprint prefix and an optional time-date-stamp infix into the filename. Another program traverses the directory tree and moves the files to the parent folder/directory. Someday they will write the HEX file directly which can be burnt into an EEPROM of a microcontroller. Good for implanting a secret copyright notice bypassing the IDE.
I love the simplicity of Forth. Julia has really good string processing capabilities and lots of file IO tricks as well. Your REXX jobs would run pretty fast under Julia, outside of the normal bottlenecks with disk IO.
With sustainability in mind it's nice to see an alternative for Python with high performance. The "It's slow, yet fast enough" sratement I often hear from Python developers starts to annoy me a bit. This 'slow-yet-fast-enough'-mindset is killing our planet. Though responsive enough, in a lot of 'sluggish' languages (like also Perl, Ruby and many, many more) too many processor cycles are used and this costs an unnecessary amount of energy and time, and therefore physical hardware.
LOL. You've probably used it dozens of times whenever you've done anything in numpy. It's ubiquitous for dense linear algebra. Though perhaps you have MKL libraries installed without knowing it.
I can tell you first hand, if you are doing computationally heavy things, e.g., simulating agent-based models, Julia just simply wins against python. It takes far less effort to crank out very heavy scientific computing. Python is what's used when you want to make science to a proof of concept (POC), but isn't even the best for real-world production code. It just ends up taking more effort to just get it to be as performant as other languages. Great for that POC area of work, but falls short everywhere else. Even MOJO sucks in my experience, with even their previous benchmarks proving a lie once you decompile them. Once the ML/AI hype hits, it's gonna be really rough for those who pigeonholed themselves as python monkeys, rather than people are doing serious work with C++, SQL, etc.
@@pedromendozaaristegui5974 Also multimethods can do pretty cool stuff. I wonder if in Julia you can reprogram the semicolons and assignment like you can in Lisp. Also hmm imagine using lenses(profunctor optics; "purely functional pointers") like in Haskell to edit your data and code, then writing a macro/compiler to turn your code into low level pointers hmm.
The way I see it is that, I have been using computers since the days of Atari, and commodore, I have programmed them and fix them too, and all I’ve experienced is pain, and more pain, beginning and ending with, limitations, it is like living in a communist country, with communist programmers, I remember how painful was programming with basic, qbasic and Visual Basic, and with the ridiculous and stupid, and illogical, no purpose limitations Microsoft set with the 64k limit memory access crap. Every single programming language that I have used, is full of limitations, ridiculous error messages, etc, etc, so maybe now with AI will have AI design programming languages on the fly, we also need a new hardware design, a flexible hardware, more like software, programming languages are like religious denominations, and we need to move on with a true programming language. My prediction is this, engineers and programmers days are numbered, THATS just the way it is, AI is moving so fast, there is no escape, my advice to software engineers and programmers is this, find a new skill, cause you/we cannot beat computers at their own game. Why the hell would anyone, knowing what is going in, want to learn another programming language, when programmers will be out of jobs in less that’s, I would say, 2 years. That’s my take. Thanks for the video.
I have a background similar to yours. I think at the age we probably both are now, no worries about AI replacing us much in what time we have left. Just better tools to look forward to so we can save more precious time.
I literally hate python. I find refuge in julia. But julia made me upset in some ways. One always pays price at initial run, since it it can not be precompiled even though there are several efforts for precompilation.
I just hope they stop inventing more new languages. There are so many different syntax to remember. Just stick on the language and keep improving will be much easier.
New languages are necessary and sometimes amazing! Improving is a must, but making something ground up with a completely different ideology is way too important to forget, because" learning syntax is hard"..
@@maxesit_tutor We need a Python -> Julia compiler, but there will always be weird programs that use dynamic manipulation of the stack that can't be compiled to Julia.
That is not true. Since Julia does Just In Time compilation it may be a bit slower the first time you call a function, which may be creating a plot. The second time you run it the function has been compiled and runs at full speed. Even this minor problem has been largely eliminated in version 1.9.
A “problem” that prevents Julia to grow is the lack of support for…newbies! Python is taught for kids, teens. It’s everywhere. It’s used for practical things like office automation, web scraping, solve problems for regular people and daily needs. Julia’s developers seems to focus only on big tasks, math, engineering…but dont evangelize young people, they don’t advertise Julia as a fantastic general purpose tool. If they want to make Julia get users, get traction, they should invest more in the “grassroots”, not only the math/engineering/computing elites.
Would you elaborate, i.e., which videos and what is the problem? We know about the screechy markers in videos 2-3 years back, but we've taken steps to avoid that.
I used to do all my scientific programming in Python. Invariably I'd have to write cython code or C code to speed things up. Now I thank God for the invention of Julia. I've been using it for 5 years and have never looked back.
You missed that Julia can use all the Python libraries just like there were native Julia libraries, but it goes further and can also use R, C++, Fortran, and Java libraries too as easily as Python in the case of C++ and almost as easily for the other languages. So, one of your big selling points for Python is not actually a selling point versus Julia.
So i should learn Julia too.
That and the fact that Julia has absolutely insane ffi speeds. So insane in fact that the overhead is barely noticable for most intents and purposes
Probably one of the most useful comments on YT I have seen. Thanks.
You can call Julia from Python and Python from Julia. So the point is rather moot.
@@MrEo89 its not moot, if python is slower than Julia. Calling Python from Julia will not slowdown Julia as much, but calling Julia from Python WILL incur a performance penalty.
As a pure science background (biology), I like programming language with high level performance in any OS and also have strong types system. That's why I choose Julia. Julia combine both concept of Java (JIT) and C/Rust (LLVM Compiler). I think Julia has a bright future for scientist. Great video. Thank you🙏
it's interesting how Julia on board looks so modern and python's defense was "it has a huge community and lot of packages". Check Julia 1.9 first the plot is fixed :D
Julia is like a Python 4.0. Many libraries in Python are needed to fix the lacks of Python. Julia doesn't need libraries like Numpy, Numba, Cython, Cpython. Matrix multiplication? Already built in.
And there are a lot of things I like in Julia, but you'll never get in Python. I like that every block is terminated by an end. In Python I get often confused by nested blocks. I like Julia's support of Unicode and Latex. Formulas are more like they look in math.
Python is easy to learn, but if performance is needed, you have to do some extra mile. Julia, which is as easy to learn, write and read, doesn't need this effort.
Therefore there are just 2 advantages of Python over Julia: much bigger community and much more written code. I hope these advantages will decrease in the future.
In fairness Python also has much better documentation. It is harder to learn Julia, not because it is more complex, but for bad documentation that is too abstract and lacks step-by-step examples.
Well i prefer Julia's doc to Python's, my preference tho.
Just think when Ai LLM's move from Python to Julia you will no longer be waiting 20 seconds for a response, it will be like 1.6 seconds. Plus, Julia works natively inside GPUs without issues or library calls.
More videos on Julia would be really interesting.
I’ve heard of Julia but this really has opened my eyes. Thanks, I’m going to learn it.
*The real problem of Julia is that Julia did not find a big tech company to spend money on it, just like how Google spent money on Python.*
all in due time m8
Julia can use all of pythons libraries.
A “problem” that prevents Julia to grow is the lack of support for…newbies! Python is taught for kids, teens. It’s everywhere. It’s used for practical things like office automation, web scraping, solve problems for regular people and daily needs. Julia’s developers seems to focus only on big tasks, math, engineering…but dont evangelize young people, they don’t advertise Julia as a fantastic general purpose tool. If they want to make Julia get users, get traction, they should invest more in the “grassroots”, not only the math/engineering/computing elites.
@@valdemarkatayamakjaer9304I agree that python is a good language to start with, python was one of my first languages. Though, python being an all purpose language is one of multiple reasons that makes it slow. Unless you're working in a low(er) level language (C, C++, Rust) you cannot expect performance and all purpose functionality. Julia focusing on maths, AI, engineering etc is the way to go. All in all, community and/or funding is the main driving factor for getting a language off the ground, not ease of use. Take Rust as an example, it is difficult to learn and I would not reccomend a beginner learn it as their first language, but rust has become quite popular.
@@valdemarkatayamakjaer9304but this will make the language lose it's essences
Great video! Thank you for mentioning multiple dispatch, a real game changer. I think your comparison is fair to both languages. Looking forward to upcoming videos.
You make it sound like Julia cannot use 3rd party libraries for various tasks. I don't believe this is true, but someone naive to coding would think that was the case from what you said.
Maybe I was not paying attention but I don't remember any reference to the main difference between Python and Julia: namely that Python is interpreted and Julia is compiled just-in-time.
I just want to point out that Julia has a lot libraries. I learned Julia before python. Python's syntax can be confusing. Also, Julia does not depend on white spaces so you can see the formatting. I have used Julia for signal processing for my acoustics work. Julia does not have a fragmented packaging system like python and can be ran only basically any desktop operating system.
What were you programming in before Julia? You seem to have an interesting background.
“Python’s syntax can be confusing”
Saying that while defending a language that iterates from 1 instead of 0 is one hell of a hot take.
@@ClariNerd Languages that iterate like mathematics does, instead of thinking in terms of memory offsets, iterate from 1. that's called "abstraction" - forget about machine representation.
Totally agree. I hate Python just for that. You really need to use an IDE or you'll lose your mind. Julia is easy to program with any editor. It's going to be my goto language.
@@biskitpagla I learned C first. The package manager was a real game changer.
It not really true if you say that Julia is “dynamically typed language” because Julia in contrast to Python can be use as “dynamically and statically typed language”. It depends on the user which approach he will choice. Special for the large scale Application it is the key difference.
Julia is (can be) as fast as C and as easy as Python...it's an incredible achievement.
Nice video. But I think you have the point about extensibility backwards. Python’s lack of extensibility has forced, for example, multiple projects to write their own custom NumPy implementations from scratch (e.g. jax.numpy). The basic problem is that in Python there is no way to extend a given function onto a type that you don’t own. This extensibility is one of the prime selling points of Julia.
In fact, the most recent criticism of Julia that sparked a lot of discussion was that Julia is _too_ extensible and that the culture of extensibility has led to correctness bugs in popular third party libraries.
I think the point you were making wasn’t necessarily about extensibility but how “batteries included” each language is. On the continuum of “all functionality needs to be imported” (Python lives around here) to “all functionality is in the base language” (MATLAB lives around here), Julia is pretty decidedly on the Python side. The one notable difference is that Julia’s arrays have a concept of dimensionality baked in, so matrices and higher-dimension arrays are part of the core language. Personally, I’m a fan, because NumPy is hamstrung into some pretty clunky syntax around matrices. And it’s nice to not have to care about the difference between a list and a numpy.array. But I think those are more personal preferences. People seem to like NumPy, so I won’t tell them they’re wrong.
I’ll note one last difference in that community in Python and Julia mean different things. In Python, community is generally a one-way street. You can search a question and almost always find an answer someone gave at some point in time. In Julia, you won’t always find a lot of answers by searching, but you can hop on the Discourse, Slack, Zulip, etc. and very quickly get an answer, as the community is very active. There are advantages and disadvantages to both approaches. Julia’s approach certainly isn’t going to scale well as the user base grows, but for now it feels kinda nice. The people who most often answer your questions tend to be working on similar problems as you, so industry-specific connections kinda happen naturally.
What a great way to present by writing mirrored against the screen of the viewer! 💡😃
I realized something while watching this. Chat GPT is trying to mimic answers like these. Not sure if it will ever be as fun as people can make the experience. Thanks to IBM for all the efforts you make in educating the general public of new technology people like myself may never get a chance to experience or even hear about, much less the inner workings. Just had to add my sincere appreciation and couldn’t imagine a more trustworthy source of truth than the Experts at IBM. Thank you for believing in your audience. Shows I’m not alone in believing this with the numbers of views.
I don't know man this is pretty surface level stuff that you will probably pick up just by going to each languages' website.
Thankful for such awesome recommendation!
Sticking around for more of such content!❤
Python is "do anything" because it has, for some strange reason, accumulated enormous number of additional libraries. In most cases Python programs are little more than glue code for those libraries (usually written in C/C++ or more and more Rust.) Julia also has many good libraries available for it. Also, perhaps with a bit of glue code ("wrappers") one can call virtually anything from anything else, although having to rely on different virtual machines and/or runtimes will complicate things.
That's the beauty of Python, I can go from an idea to the first implementation in one evening, without spending an entire evening debugging the program.
I can start with Idea, develop web server on Raspberry Pi in my home, make it do things, pickup data from net, process it with tensorflow lite model, connect that server with additional sensor from outside environment, and in the end display data on small custom screen from AliExpress 😄 - everything in day or two of hobby programming!!
This is way Python rules, and it is irreplaceable!!
That's was a very fine point that both of them are open-source.
I’m going to make a bet in Julia for ML / DS and Rust for system programing. Lets see if they’ll have a kind of QT…
I think Julia is definitely the future of ML. It can do anything python can do at twice (or more) the speed, including python via ffi. It also doesn't need extensions for what should be basic functionality.
Scalability in terms of networking is the biggest issue in Python. Serializing / deserializing data and passing it to individual processes / nodes, across a distributed environment such a pain.
Use pyspark or ray or one of the other gazillion distributed frameworks available
Also, I would add, that numpy is not as intuitive as the in-built matrix operators in Julia. Plotting in Julia is so much easier than with matplotlib. JAX which is the new package for ML by DeepMind in python is almost a clone of the paradigm of FluxML in Julia. They even have their own version of JAX numpy which is made to resemble the matrix operations in Matlab and Julia. Some quirky things in the Python syntax do not exist and in general that 'pseudo code abbreviations' in python are not always obvious, like in python it is 'len' but Julia it is simply 'length' spelling it out completely
I'm not a big fan of Python, but I much prefer the code abbreviations. Shorter is better when the meaning is clear. In any language it is necessary to learn the identifiers, and I would never assume 'length' rather than 'len' or 'size' or whatever. I would rather have 'enum' than 'enumerate' in Python, for example, and it would be more consistent with 'len'. There is no difference in clarity, but one is easier to type and keeps the code more compact, which I find easier to read. I think languages should be written for continuous use, not just first time use or teaching use. Perhaps starting in C on Linux/Unix made me prefer brevity.
@@podunkis auto-completion coming on a white horse...
The argument of extensibility listed under Python is very much also a Julia advantage. The ecosystem is not yet as large, but you may need only a small part of the ecosystem. For me it's been quick to start with CSV, NCDatasets, Dates, Makie, Images packages. There is a good deal of plotting options, but it was a bit hard to decide which one to learn first. The syntax and layout of Julia look a lot cleaner to me than Python. I have been using Scilab (a free Matlab alternative) for years, and IDL/PVWAVE before that. I am finding the transition to Julia very smooth. I actually don't mind the type annotations, especially for the speed gained. It is pretty much mandatory if you wish to input data into certain functions. In Julia you need to be careful about assigning variables to other variables, if you don't want to change those as well - this can lead to unexpected bugs you can lose hours with :)
I think the issue is the same in Julia and Python. a = b means a is just another label, not a different piece of data in a different portion of memory. This is a good thing. If you want you can do a = copy(b).
Mmmm Julia is dynamically typed by default but unlike Python real static typing is an option.
Also, Python libraries are usable in Julia by default, that's not an advantage of Python
The real problem with Julia is the lack of a good debugger. I was using it a couple years ago and had to add print statements all over the place to trace down issues. It's like I was living in the 80's. Not good.
Isn't Julia a Lisp though, you could do "eval hacking" to do traces.
the biggest plus for me in julia: static typing support, and broadcast operator
It was an excellent explanation on the application of Python vs Julia! Since, Julia is used for applications such as Artificial Intelligence and Machine Learning. Is it possible to make an video on how it is applied in these areas? I will be waiting for the Video!
Thank you, I am goin' for Julia, as I do like the beauty of Matlab
MY MIND IS BLOWNWNWNNW..... HOMEBREWER MARTIN KEEN IS AN IBM MASTER INVENTOR..... I SHOULDVE KNOWN
Your videos are always insightful. Thanks!
Great video. Love the content you guys put out.
Great video! Learning about the differences between Python and Julia is a great way to expand your knowledge and become a more well-rounded programmer.
Python is a lot of fun for hobbies and experimenting with new technologies because of the huge user community. But if there is any serious number crunching, it is so slow that it will make you commit suicide. It sounds like Julia is better for that. (But every language is faster than Python)
Great video. Please make a video like that java vs csharp
Python does support single and multiple dispatch. Singe dispatch is part of the Python distribution and has to be imported from functools; multiple dispatch has to be installed from the repositories.
If you like Julia or Python but you prefer static typing then you may like the Nim programming language, a little known gem!
Does Nim has something like Pandas?
In Julia you have full flexibility where you can annotate the type like x::Vector{Float64} = ... and get all the benefits of static typing (speed, input/output security, functions tailored to the input type) - where it is needed, and use dynamic typing for other things, as you wish.
Would be nice to see actual benchmarks
Thank you, I love you ❤
The review is flawed, as Julia is capable of utilizing all Python libraries. For instance, if you run a Python-based LLM (AI) locally on an i7 8-core processor, it may take around 13 seconds to generate a response. In contrast, the same LLM implemented in Julia could potentially provide an answer in just 1.6 seconds. This performance advantage is one reason why organizations like the DOD and NASA are adopting Julia for various applications. Additionally, projects like the Next Generation Air Dominance System (NGADS) and future fighter aircraft featuring integrated AI piloting systems are also being considered towards a transitioning towards Julia or another in-house language built from Julia. Rewriting the James Web Telescope from C++ to Julia could yield significant benefits, particularly due to Julia's ability to leverage unlimited core and device access unavailable in other programming languages. Given 100% flex control on every moving piece through the actual manipulate of power flow cut off as fractional and part cycles.
With that, Python has no advantage over Julia, which was original built around having its future adopters coming from the Python programming community.
What about cython? By adding types performance can be enhanced 20x.
A small question:
Will Julia replace Python in performing some data science tasks in the near future? If yes, then what are those tasks?
Anywhere where you apply math extensively.
Python is strong when there is an existing performant library for your use case. If this is not the case, custom Python code is very slow compared to Julia (often a factor of 100 slower in runtime). Optimizing Python code with Cython, Numba, etc. is possible, but rather cumbersome. In such a use case, Julia is a much better fit.
There's no 'replacing' in any concrete sense of the term when it comes to open languages like Python or C. It's mostly super niche proprietary langs that get 'replaced' at one point or another. And sometimes the real friction facing these migrations isn't from the actual use case at all. Like, a data engineer might have a hard time migrating to Julia because its web development scene is nowhere near that of Python. Your question was about Data Science specifically, but Data Science doesn't exist in a vacuum so there's that.
but you can call Python from Julia. Does that help with extensibility? or not really?
Yes, there is an easy to use interface to call Python
#Example, if you want to use
#matplotlib directly
using PyCall
pl = pyimport("matplotlib")
xs = -2pi:0.01:2pi
ys = sin.(xs)
pl.plot(xs.ys)
#But in this case, you would prefer to call PyPlot
Is Julia ready for use in a production environment? Have used it years ago (v0.4/0.5), and contemplating on using it for analysis. How well are packages maintained and how often will implementations break?
Julia is now available in version 1.8 and a really mature environment in the meantime
Not only is it at version 1.8 with a best in class package manager. It has almost 9,000 packages downloadable. It supposedly has great threading capabilities.
Currently, there is a big effort to make it support HARD real-time execution with the ability to control garbage collection scheduling so GC doesn't prevent responding to events in a specified number of milli or maybe micro seconds. This is coming incrementally in versions 1.8 and 1.9 and possibly some final work in 1.10. Part of this is Julia Computing, now Julia Hub, is giving abilities for static compilation so that you can do all the hard compilation work once and then when it runs, you will be hopefully able to eliminate the long "time to first plot", and instead have, basically a shipable .EXE file (or a few). Current talk seems to be about fully compiling and in 1.9 there will be caching of compiled instances of each module (package), but by version 1.10 Julia being able to have a single .EXE file (or an EXE plus one compiled library object). The details escape me and seem to be evolving.
The beauty of all this EXE and real-time work is to get around Julia's famously slow time-to-first-plot startup time, measured in seconds or sometimes even minutes for code that relies on dozens of packages.
Everything else is looking production ready. No breaking features after 1.5 and none planned.
Julia Pro is designed for enterprise. In my personal experience I can't recall implementations breaking.
I would say yes, but only very recently. Julia reached v1 (i.e. no break promise) in 2018 but most large packages (DataFrames, JuMP,...) did that only 1-2 years ago....
Julia is in version 1.9 now and is very solid and stable.
I am still trying to break into the IT even after my Btech graduation I am still struggling to get a good IT job. I have done certification in Data Science and business analytics just to find out that I have to be a python developer or a good statistian to become a data scientist. I am good at SQL and have created some projects in data science in python. Should I learn Julia too?
I no have idea about a program called Julia. It's interesting video, I'm a beginner in that new world about programing.
Once Julia is used in University textbooks, this will stsrt to grow. So, need to write a very good programming, web application and data science books with lots of excercise for students, then Julia will get popular
Really strange when a very simple program runs differently from Jupyter and the command line -- both Julia 1.10.3.
x = 10 # x in the global scope
for i = 1:5
x = i # x in the soft scope (inside the for loop)
end
println(x) # prints: 5 for Jupyter and 10 for command line
Thanks for the video!
...I don't know much but, I'm just going to throw this out there: NumPy is a wrapper around Fortran code ...your not going to get faster than that, are you? what kind of performance hit do you take from the 'wrapper?' Nice video, thanks!
I did a quick internet search, it seems the 'debate' is on! julia vs fortran performance...
You've actually alluded to another issue quite nicely; if you learn Python and want to get into the nuts and bolts of the libraries you're using quite often you discover that you would have to have a working knowledge of Cpp or Fortran. Part of the motivation for developing Julia was to solve the, 'two language', problem.
While numpy is largely Fortran code, the API is pretty bad for performance. Numpy code ends up making a ton of copies and doing a bunch of redundant loops over the same memory. It also has a pretty high per call overhead (roughly 1us) which makes it incredibly slow for small array operations.
It's weird he didn't drink a beer at the end
Let's learn Julia
Could someone make a python variant having non-significant whitespace?
All I have to say, julia is hella faster than python for solving differential equations.
0:29 this thing writing on our screen could give heart stroke xD
I really wanted to use Julia but it's packages were convoluted. Conflicts. Some are not build without any understandable reason.
Had to abandon idea to use Julia. To my sorrow.
Switched to Python. And it worked out of the box.
Maybe will try Julia once more in a year ot two.
Hi I just curious what do you write the borad, with writing from left to right and display also left to right?
It's never important to me that my algorithm gets there quickly, I love slowness. I think it's great that IBM had this video, but it was more of a round about way to say yeah Julia is fast but python can do more.
I don't think Python can do more, since you can call Python from Julia.
When you try to calculate a numerical integral over 1M points, try to like such slowness. That's why I prefer Julia over Python. Julia is faster and can be parallelized easily.
Its funny how Amazon and their AWS company debunked that Julia was fast. As Python was tested against Julia in AWS and AWS said Python was much faster and even more
economical when doing extensive tests with them. Julia takes up most electricity than python when tested in the AWS, which means AWS seen that Julia cost more to operate than Python.
AWS and Amazon even created a 5 page website extensively going through each and every language and how fast and feasible they were to run within AWS Amazon services.
Python won and beat every language including Java and C#
Dont forget that Guido Guido van Rossum is with Microsoft now and has enormous resources to acomplish whatever plans he has for Python . I would place my bet on Python primarily because of its extensive libraries. A long time ago, IBM developed PL/1 that is upposed to combine the best features of Fortran and Cobol but it never gained traction. Will Julia suffer the same fate? Time will tell.
Pretty sure they included some compile time in their Julia benchmarks. Short-running scripts are rather slow in Julia because they are compiled every time a new Julia instance is started. This may be an issue for some kind of webservices.
I've been programming Python since it was first released. I'd suggest the Amazon engineers are hacks applying a great language within a crappy framework designed for Python.
@@benjlung Exactly.
I'd take those benchmarks with a grain of salt. You can precompile code and libraries so that there is not much start up time and the code will run much faster than python. I'll bet that the Julia Jit compiler time got baked into those benchmarks.
Wow he is really good at writing words backward.
Python: "Do anything" (slowly)
That is easy, modern c++ ;)
Julia vs Rust, which one do you prefer?
That is like comparing a hammer and a screwdriver.
I tried studying Python and my vision started to blur when the Walrus operator was introduced. I doubt I could debug my own Python program after a year. I really hate the forced indentation means of packaging as much as I hate C's curly braces. Sounds like what was said for these languages was also touted for Java. I guess I'll stick to REXX or Forth. I used REXX to rename files and the performance was dismal with each "address DOS ... " statement. So nowadays, my REXX programs write BATCH programs which are executed as fast as CMD can clear them. Not greased lightning like the big guys but I get the job done under my terms and control. I mainly use these to organize JPGs from digital cameras into a single folder/subdirectory. My programs rename each file with a 4 hex fingerprint prefix and an optional time-date-stamp infix into the filename. Another program traverses the directory tree and moves the files to the parent folder/directory. Someday they will write the HEX file directly which can be burnt into an EEPROM of a microcontroller. Good for implanting a secret copyright notice bypassing the IDE.
Interesting, another forth(er). Haven’t heard anyone else that’s used that in decades. Cool!
I love the simplicity of Forth. Julia has really good string processing capabilities and lots of file IO tricks as well. Your REXX jobs would run pretty fast under Julia, outside of the normal bottlenecks with disk IO.
Yes I also hate the indentation in python
With sustainability in mind it's nice to see an alternative for Python with high performance. The "It's slow, yet fast enough" sratement I often hear from Python developers starts to annoy me a bit. This 'slow-yet-fast-enough'-mindset is killing our planet. Though responsive enough, in a lot of 'sluggish' languages (like also Perl, Ruby and many, many more) too many processor cycles are used and this costs an unnecessary amount of energy and time, and therefore physical hardware.
how do u write backwards
Weird not hearing about a grain bill in this video
How the heck do you write backwards!
Search on "lightboard videos"
Why it's so quiet?
Is OpenBlas worth using? never heard of it before.
That makes at least two of us.
LOL. You've probably used it dozens of times whenever you've done anything in numpy. It's ubiquitous for dense linear algebra. Though perhaps you have MKL libraries installed without knowing it.
You're very good at writing backwards.
Search on "lightboard videos"
@@IBMTechnology Oh, I know :) I noticed there are an awful lot of left-handed people making these videos.
Why this audio is so low?
The comments are way more enlightening than this video.
I can tell you first hand, if you are doing computationally heavy things, e.g., simulating agent-based models, Julia just simply wins against python. It takes far less effort to crank out very heavy scientific computing.
Python is what's used when you want to make science to a proof of concept (POC), but isn't even the best for real-world production code. It just ends up taking more effort to just get it to be as performant as other languages. Great for that POC area of work, but falls short everywhere else. Even MOJO sucks in my experience, with even their previous benchmarks proving a lie once you decompile them.
Once the ML/AI hype hits, it's gonna be really rough for those who pigeonholed themselves as python monkeys, rather than people are doing serious work with C++, SQL, etc.
Seems weird not to include R in this conversation.
True, but R is interpreted and therefore slow like Python.
You forgot to mention Julia's array is 1-based but Python like most other languages, its array is 0-based.
Dude: Julia is way more extensible than Python.
Yeah, extend by macro system
@@pedromendozaaristegui5974 Also multimethods can do pretty cool stuff. I wonder if in Julia you can reprogram the semicolons and assignment like you can in Lisp. Also hmm imagine using lenses(profunctor optics; "purely functional pointers") like in Haskell to edit your data and code, then writing a macro/compiler to turn your code into low level pointers hmm.
Did the dude just write stuff on screen in reverse?
Search on "lightboard videos"
I'll stick with good old faithful C++
Julia all the way
cool video)
Can you also consider comparing Polar in your future video?
The way I see it is that, I have been using computers since the days of Atari, and commodore, I have programmed them and fix them too, and all I’ve experienced is pain, and more pain, beginning and ending with, limitations, it is like living in a communist country, with communist programmers, I remember how painful was programming with basic, qbasic and Visual Basic, and with the ridiculous and stupid, and illogical, no purpose limitations Microsoft set with the 64k limit memory access crap. Every single programming language that I have used, is full of limitations, ridiculous error messages, etc, etc, so maybe now with AI will have AI design programming languages on the fly, we also need a new hardware design, a flexible hardware, more like software, programming languages are like religious denominations, and we need to move on with a true programming language. My prediction is this, engineers and programmers days are numbered, THATS just the way it is, AI is moving so fast, there is no escape, my advice to software engineers and programmers is this, find a new skill, cause you/we cannot beat computers at their own game. Why the hell would anyone, knowing what is going in, want to learn another programming language, when programmers will be out of jobs in less that’s, I would say, 2 years. That’s my take. Thanks for the video.
I have a background similar to yours. I think at the age we probably both are now, no worries about AI replacing us much in what time we have left. Just better tools to look forward to so we can save more precious time.
Sir, 1 of those 2 years is almost over and no jobs were lost to ai bs, so please reconsider your opinion regarding the matter
I literally hate python. I find refuge in julia. But julia made me upset in some ways. One always pays price at initial run, since it it can not be precompiled even though there are several efforts for precompilation.
Largely fixed in version 1.9.
NICE!
Solskjær changed a lot after Man Utd
I just hope they stop inventing more new languages. There are so many different syntax to remember. Just stick on the language and keep improving will be much easier.
New languages are necessary and sometimes amazing! Improving is a must, but making something ground up with a completely different ideology is way too important to forget, because" learning syntax is hard"..
@@maxesit_tutor We need a Python -> Julia compiler, but there will always be weird programs that use dynamic manipulation of the stack that can't be compiled to Julia.
julia is all about performance unless you're doing something really esoteric such as making a plot
That is not true. Since Julia does Just In Time compilation it may be a bit slower the first time you call a function, which may be creating a plot. The second time you run it the function has been compiled and runs at full speed. Even this minor problem has been largely eliminated in version 1.9.
Have you tried the latest versions of Makie? It is surprisingly fast now.
👍👍👍
Being high level is not relevant.
C, Haskell, Prolog, Lisp, etc are all high level too.
wow the guy really writes backwards
Search on "lightboard videos"
What about with elixir?
Why make a video of 2 programming languages while showing no code
👍🏻
A “problem” that prevents Julia to grow is the lack of support for…newbies! Python is taught for kids, teens. It’s everywhere. It’s used for practical things like office automation, web scraping, solve problems for regular people and daily needs. Julia’s developers seems to focus only on big tasks, math, engineering…but dont evangelize young people, they don’t advertise Julia as a fantastic general purpose tool. If they want to make Julia get users, get traction, they should invest more in the “grassroots”, not only the math/engineering/computing elites.
Maybe if we ask Guido for a successor language.
How he is writing in reverse is making me dizzy 🥴
Search on "lightboard videos"
Julia is bad. Mojo is superior. Julia developers should give up and work on their python skills on prep for mojo.
Nope
IBM Technology should work on sound-audio technology. your youtube videos sound are terrible 😁
Would you elaborate, i.e., which videos and what is the problem? We know about the screechy markers in videos 2-3 years back, but we've taken steps to avoid that.
I think your speakers just suck lmao
Ji
Truth about julia is that it's a nice tool but it's not worth it. Python needs something way better go get dethroned
Yeah, time.
I used to do all my scientific programming in Python. Invariably I'd have to write cython code or C code to speed things up. Now I thank God for the invention of Julia. I've been using it for 5 years and have never looked back.
It's a very very easy choice! Python is in everywhere! Julia, well... is not!
For now.