Python vs Fortran vs Octave (Matlab) side-by-side performance comparison
Вставка
- Опубліковано 12 гру 2021
- Which free scientific computing language is the fastest to program and execute? The answer probably won't surprise you because it's obvious.
- Наука та технологія
You need to do Fortran vs C vs Rust. That'd be an interesting battle to see which comes out on top.
agreed. if anyone has some ideas for fair benchmarks, let me know. something that can be implemented relatively quickly without non-standard libraries
My thoughts exactly
@@xmdi0 No Ideias, but please do. Alredy clicked on the Bell Icon
@@xmdi0 Rust is meant to be used with non-standard libraries (=crates). My personal favorite for benchmarking is criterion.
@@xmdi0 Also include a simple I/O operation. I really wonder the capability of reading and writing capabilities.
Never have I come across a professional programmer that considered any technique that worked cheating.
It’s not cheating to use C binded libraries. Languages like python are designed to be used like that as glue languages. If you’re doing high performance computing in pure python you’re the one using it wrong.
I’d like to think he’s being sarcastic
Yeah, I write python, and I hit a math problem, I drop to C, write a lib, the expose the .so file in python and keep working. Simple, effective, and get to keep my high productivity in python.
You just confirmed that it would be cheating. If you're using it wrong for high performance computing (which I agree with) then it would also be wrong to measure it's performance to begin with.
@@Luxalpa I do agree measuring its preformace for algros is wrong. I dont know any devs who spend their time in python writing algros like that. We are building large applications with 100's of dependances and 100k+ loc. When you hit that size, what matters is developer read speed, not compile time, and honestly.. not even run time. I mean, we gotta keep the hot path fast, but again 99% of the time for me at least, its the network thats the bottleneck, not the application.
It's cheating if you're comparing Python as a language to another language like C or Fortran, which is what's happening in this video. Sure you might be able to get better performance using Python with libs written in a faster language than you could with straight Python but that doesn't make Python fast, you get good performance because the language used to write the library is faster than Python.
"Time to write code" is a really weird benchmark considering you keep erasing mistakes and probably already memorised certain lines in some languages but not others. The fortran code even has more lines than the python code but took less time to write?
Also the python and octave code isn't vectorised, which is the main way to write linear algebra code in these languages and basically solves your 'verbosity' problem. I mean, one advantage python and octave have over fortran is that you don't have to use for loops for everything.
Not even a chance. Fortran and C++, that's it. All the rest is a passing fashion.
@@pedromiguelareias lol do you have brain damage?
@@pedromiguelareias That's hilarious. Nvidia is building a multi-billion AI business that runs entirely on Python.
@@DoctorMandible Python just makes use of C, C++ and Fortran libraries. It's a wrapper and inefficient at that.
Just FYI in Octave you can write C++ files and compile them to oct-files which can be called as built-in functions. They will increase the speed of code with many loops. Of course there's no competing with Fortran or C++ but ease of use for scientists/engineers also has its merits.
So Octave is the fastest language if you write all of your code in C++?
@@danielthirtle666 hahaha
Thought Octave is just a library... xD
Is "using C++ is easy for scientists" some kind of joke, or this person is.
@@goose_clues you are talking about my comment? Because it says the opposite of what you are implying
FORTRAN was the second language I learned. Glad it's still a thing.
x2
I have no clue about Octave, but in regard to Fortran vs. Python you're comparing a highly optimized (-O3 flag) onetime compiled Fortran code to a Python script that keeps loading the NumPy module each time you run it. Probably 80 % of your Python code's execution time is due to the NumPy module being loaded over and over again. I've made similar speed comparisons between C, badly written Python, "pythonic" Python, Python + NumPy, Cython etc. If you load NumPy once and measure the net execution time (=the time the actual execution takes without loading the module), your code will be marginally slower than C, if at all. Obviously, if you repeat a vacuous comparison of gross execution time (loading module + executing the code) vs. net execution time of a precompiled optimized code ten times in a row, the gap between Python and Fortran grows bigger and bigger.
So, your point is python is slower?
You use numpy and then for-loop over the matrix, this is misusing of the library.
Hi Sataro, why for-loop over the matrix is misusing of the library? Is there another way to reach each component of the matrix with numpy?
I wrote a super scuffed “paper” comparing 2 ways of solving systems of linear equations for school and I used Python to solve all of the 15,000+ systems… thanks to this video I realize I should’ve just used Fortran.
Should've used c++
Fortran is a very challenging language. C/C++ would give you about equal results, with way less screaming into the void. There's a reason basically no one uses fortran anymore.
@@deidyomega C++ would probably give you much better results if you used multithreading and a sane linear algebra library which implements SIMD. It would probably solve all those equations within a second.
@@deidyomega yeah no
@@klittlet Are you disagreeing with the fact people dont use fortran anymore? Or that its challenging? Or that C/C++ have comparable results?
All of those can be backed up pretty easily.
You should include the Version of python and python libraries you used to allow for watchers far in the future to compare and benchmark against it so they can understand the progress made over the years.
You sir have done the needful
Cool content. Found you through my recommended videos. Earned my sub.
Interesing comparison clip. Fortran is also my favourite language to solve scientific and engineering problems. Partly becuase this was my first and continuously used computer language. In particular the modern enhancements make it an up to date high performanc e computation tool.
Motivated your comparison I tried to check what in practice (also using the python import libraries and having own code or slatec libraries in fortran) could be achieved by python (3.6) vs gfortran:
Problem 1: a nasty integration (using scipy.quad vs. adaptive integration in f90) of sin(1/sqrt(x)) from 0 to X with an accuracy level of 1e-9, X was varied form 1 to 100 in 0.01 steps to accumulate enough time: result python 44sec vs Fortran 0.8sec (corroborating your result).
However, employing numpy to solve a large linear equation problem (rand om coefficients, 100 realisations) compared to the old dgefs (slatec) routines yileds a mixed result:
for a 500x500 matrix fortran wins by a factor of about 2 (2.5sec vs 5sec), for a 1000x1000 matrix pyzhon is slightly faster 21sec vs 25sec and for a 2000x2000 Python (numpy) is faster
90sec vs 3.5 min. Obviously the numpy library uses some parallel processing as can be seen by the CPU usage which yields an advantge compared to the single threaded old dgefs routine. Using modern routines with parallelisation would probably restore the initial factor. Howver, with the compiled numpy code the disadvantage of the interpreted language is largely mitigated for problems where the heavy computing is absorbed by these function. Still I favour Fortran also because of other reasons as strict backward compatibility (can icorporate code from 50 years ago without
problems), clarity (if written properly) .
The latter comparison code fragmets if fortran and python are:
FORTRAN
program testsolve
implicit none
integer, parameter :: n = 2000
double precision :: A(n,n), B(n), X(n)
integer :: i
! for linalg solving: A * X = B
do i=1,100
call random_number(A)
call random_number(B)
call lesolve(A,n,B,n,n,X) ! wrapper for slatec dgefs
B = B - matmul(A,X) ! Check the solution
write(*,*)"Mean deviation(",i,") = ", norm2(B)/n
enddo
end program testsolve
PYTHON:
import numpy as np
import numpy.matlib as ml
import numpy.linalg as la
import random
import time
t1=time.perf_counter()
n=2000
A=ml.zeros([n,n])
B=np.zeros(n)
nr=range(n)
repeat=range(100)
for l in repeat:
A=np.random.rand(n,n)
B=np.random.rand(n)
X=la.solve(A,B)
BX=ml.matmul(A,X)-B
u=np.sqrt(ml.vdot(BX,BX))/n
print(f'try no {l} yields residual error = {u}')
t2=time.perf_counter()
print(' ')
print(f'Python(numpy): timer={t2-t1} sec with problem size={n}')
I got 3.54ms per loop for the scipy code. Also numpy code can be made to avoid for loops as much as possible, it gives 154ms per loop
Scipy code:
%%timeit
import numpy as np
import scipy.linalg as la
n = 200
A=np.random.rand(n,n)
P,L,U=la.lu(A)
print(np.max(np.abs(np.matmul(L,U))))
100 loops, best of 5: 3.54 ms per loop
Your results are meaningless without the other two solutions or at least the specs of the system.
It's like the cfop roux zz debate except there's objective benchmarks that you can hit (zz is Fortran in this example)
yes gen
It would be nice to include Python with Numba, which is not really hard to use. Then you might get faster code with Python than Octave and maybe even Matlab.
Man, the author thinks using numpy (or even importing it as np) is cheating, for Numba you go to jail lol
@@saitaro the test involves using the languages by themselves
@@klittlet I got it, but if he decides to use numpy (which is not a part of the Python stdlib), then it should be done properly, not for-looping over the np.array. It's just wrong to write Python code in the style of Fortran, and vise-versa.
IMHO, in the end, comparative tests should be about using tools from whole ecosystem of the language. It is especially important for scripting languages like Python.
as an embedded engineer, i'm curious for Rust vs C/C++ nowadays.
12 like almısın nıyazi
Fortran wins
I respect FORTRAN and you have amazing Vim skills but that is not the way to compare it. This is ok for 30 lines of code but what happens when you are working on a 300000 lines of code project where dozens of scientists need to collaborate? No one picks python due to its speed, but due to its easiness-to-code and readability, the very same reason that made FORTRAN popular back in the day.
Speed is still important. What could take days on FORTRAN may take weeks to compile and run. That's why it's still king in certain industries like Physics where you need to crunch high amounts of data.
Java vs C# vs python
wouldn't it be cooler to add compiling time into the trace ? like, a potential language which is written like python/octave but compiles to the same bytes as fortran would blow up this comparison..
Great channel! I think you have a good potential in growing the auditory.
Greetings from Russia)
С чего бы?
I would LOVE to see a comparison of Julia, Python, Fortran and C++ for this kind of application. Julia is the new kid on the block that people seems pretty excited about
There's Julia too, but it's JIT compilation takes a few seconds the first time you run a program.
Pretty sure Julia would win in programming time (assuming the author is similarly experienced in all languages). In run-time, Julia would be a close battle with Fortran if you include compilation time in both languages or none. Even if including compilation time, Julia should be faster than Python and Octave here.
AFAIK Matlab is faster than Octave. Also, I have a simple rule when it comes to Python, if I need to use many for loops to do it (and there is no other way, or it is too complex to implement), then I am better off just using a different language (like C or C++).
this 100%. in my experience matlab is usually 2-10x faster than octave, but there are ways to compile octave yourself to build in some of these (intel?) accelerations. something i have never needed to do, but i read about once
There is a technical term for that. Implementations in MATLAB/Python/Octave need to be "vectorized".
If you are writing C in MATLAB/Python/Octave then you shouldn't be using MATLAB/Python/Octave.
The loop should've been inside the program after imports.
I think it's unfair how you benchmarked octave and python in the second test. Both languages have the overhead of loading the interpreter and Python has also to import the modules which takes a comparatively huge amount of time compared to time it takes to get the actual task done.
That would be more representative of real world use as in real applications the time it takes to finish the task dwarfs that of loading.
time to import libraries is still time that the code takes to run. part of the beauty of a language like fortran is that it needs no libraries for mathematical computation no?
I used to program in Fortran and Cobol. Eventually, the SQL app tools started coming out and people would just laugh at the "old school" languages like Cobol.
Then they tried to use their fancy new db tools to process a few hundred thousand records. I could go through them in a few minutes. Their SQL just slowly chugged along.
There are some pretty fast ORMs out there (I have some experience with Diesel) but I usually return to using SQL 😁
This Keyboard ASMR is so satisfying xD
Nice comparison but isn't the whole point of vectorized data handling packages like numpy, scipy and pandas is to avoid for loops? I'm not sure how this particular example can be implemented otherwise but iterating over arrays or dataframes is basically going against the spirit of these libraries, something that you do as a last resort.
curious whether fortran libraries have caught up with current scientific computing... the speeds look compelling
The person clearly type much slower in python even for the same code and same amount of characters...
Octave is like that kid when he walks in "I CAN RUN FASTER" and ends up being in last place every year for track & field.
You code use elementwise function. Why you use for loops in python and fortran?
Python is good for prototyping, utils scripting, launch automatization and data pre/post manipulation.
Specific tasks in between must be done using specific tools.
Actually makes me happy I use NodeJS. It trades blows with Java when it comes to speed.
Very informative. Can you do similar video on C/C++ vs dart vs golang?
This is really quite interesting. As an engineer what do I want? Something that is easy to program first and foremost. That’s why I use MatLab and Octave. I don’t want to fight the programming language - I want to develop a reliable solution with the least amount of time and effort possible! If standard libraries improve performance and speed up the development of code who cares. I’m not creating code for commercial applications. I use programming to solve problems the way an accountant might use a spreadsheet.
Well, Python results like that because you did it wrong :)
Can you do that in C too? Compare?
What language is vim interface in?
one thing to add as well, is correct me if im wrong, but i believe writing to stdout can slow a program a decent bit if it's running in a single thread
Definitely true.
for both octave and python, you could do away without for loops and the code would run much faster. I'm not sure if you're trolling or if you just don't know how to use these languages
Im subbing, do more things like this please
I LOVE THESE
Demo 2 on Matlab 2021b, run time is 0.026 s.
tic
for i=1:10
n=200;
A=rand(n,n);
[L,U]=lu(A);
norm(L*U-A,Inf);
end
toc
why do you use A.__sub__(...) instead of A - ... ?
It was nice hearing the keystrokes.
Why not use both? I use Python in a Jupyter notebook and F2Py in Fortran code for CPU intensive calculations.
U need try compiler optimizations for fortran.
-O3 and -Ofast.
gfortran -Ofast nn.f94
and C, C++.
That looping over numpy array made my eyes 🩸
Can you do for Rust, Carbon Google, Bun, and Zig?
If I want to learn an auxiliary lang to python for scientific computing, should I go with C or Fortran?
C will be easier to learn for you because there are more resources, and it's more similar to Python. The main advantage to Fortran is that all this matrix algebra stuff is baked in, whereas you need a library or your own data structures to do that stuff in C.
@@xmdi0 Ty, indeed I did not find as good and many resources for FORTRAN, but I thought it had an inheritant advantage over C or something. But is reading files really that slow in FORTRAN? (which could be a different comparison vid)
I'd say Fortran, although the language is quite awful everybody in science (at least physics) uses it if they are not going for python (I never used it but I guess it must have tons of libraries?).
Both
@@Labicraft It really has a ton of well optimized libraries and is quite possible to understand if you come from other scientific background than computer science (which can't be said in all honesty about C or C++ when you need the amount of control typically needed in physics for huge calculations). Basically, it takes away some of the most notorious footguns from the coder.
That said, I'd also like to draw attention to Julia, which is a language specifically made to make the number crunching typically done in physics and statistics easier to both write and execute (similar to Fortran, but with a more modern look and options).
I don’t know, maybe there are beginners to whom such videos are somehow useful, but in fact a normal developer knows very well that each programming language has its own advantages, purposes of application and best coding practices.
What's the point of comparing airplane, lorry and tractor on what is the faster to go for grocery shopping to a nearby store?
Dislike from me.
At 1:54 what with python how do you get those parens to add themselves. You're obviously using vim, is that a plugin or a custom config thing?
lol
i'm pretty sure he's typing them manually, just looking at the parens highlight to know when to stop typing
Would be interesting to see R in there
you should also measure a time of other co-worker sit watching the code and thinking "what the heck it does?"
please make a epic fight beetween javascript and python or java vs C++*_*
No. Assembly was the first programming language. In fact assembly is the human readable alternative of machine code. It is machine code. Fortran is the first language to have a compiler that then converts to assembly.
You are missing an important metric, how much time you spent learning those languages to do useful work.
it is not the purpose of this video ;)
We need to see Haskell and Lisp, the father of all evil languages,
Love the attitude.
Python is by design just objects. Well, objects is a nice way of saying it is pointers to everything. (pointers to pointers to data or even more pointers) Then on top of that it isn't compiled and it's very rich in built in features.
Python is a nice scripting language; but i think it should have optional curly brackets by default. Why?
Cause the argument that you wouldn't use python in a situation where you cant easily control whitespace indent is only true because you can't use curly brackets.
It is precisely in small scripts i just want to send to an interpreter i could use something like python.
I know and have used semicolons to mark newline, f.ex if you ever need to use eval() , and it's possible but way to difficult.
But python does have some uses where speed isn't any major concern, altough i might change that statement if/when i get well into JavaScript. Still, python is a nice language to start with, it's sort of the modern basic.
I didn't know Fortran was much used anymore though, remember it was one of the first languages i dabbled with.
Octave is re-branded Basic?
The results are quite predictable. Interpreters work slower, but in general require less time for coding. Python unlike Octave/MATLAB or Fortran is a multipurpose language, so it may be slower in all aspects. The more interesting could be comparison of outputting a graph (Fortran has no intrinsic graph tools). I use Fortran for many decades, used MATLAB for some time and sometimes use Octave and gnuplot. I have some basic info about Python and responses from those how use it.
most biased video against python I've ever seen. especially when I saw A.__sub__. Not to mention np.eye and np.abs v.s. np.identity and np.absolute.
Are you coding on phone or something? Is cool if you coding in phone. what app you use
Just a vertical window on my desktop. Would be cool to have a phone app for that - especially for traveling. If you find a good one, let me know.
@@xmdi0
java n ide
I want learning java but i don't have PC so i learn in this app idk this is good? I think your knowledge more than mine
Julia would be interesting especially with more computationally intensive work.
Is there someone who knows here how to start programming in Fortran, I need it please. I mean, which compiler could you suggest me, thanks in advance.
You shouldn't print anything in the middle of the benchmark. IO is very slow and doesn't make sense during the calculation
numpy+for?
Fortran does produce the fatsest executable.
Sei italiano? Are you italian?
Python vs R vs Ruby
what is the name of the application
Try Fortran against C, C++ or even Pascal...
I like that last comment Don't mess with Fortran....🤩
Which language should I learn as an absolute beginner?
My opinion: pick Python if you want programming to be complimentary to your job, pick Java if you want your job to be a full-time software engineer. The most important thing is to just get started though, it doesnt really matter what you pick.
@@FR33Willi I have started with HTML and CSS for doing simple websites, once i end those i think im going with java or javascript
Can you do a Rust vs C or Rust vs C++ or C++ vs NodeJS vs Python?
C++ vs NodeJS vs Python? Why even make this comparison? We already know C++ is fastest, then NodeJS and slowest will be Python.
@@Luxalpa Unfortunately while I agree with you, I've had to debate with people on which was faster and I would like to set a hard benchmark to settle all debates.
@@shayhan6227 lmao
recommend learning fortran in 2022?
are you an engineer? are you a mathematician? If not fortran might be of little use for you. If Yes, Then you probably needs to buy Matlab after all...
@@justasydefix6251 I am studying computer engineering
Great code and when to start a Fortran, generally numpy array slow sadly no other reason using. Fortran is faster.
Hey I know its been a long time since you posted this. But i want to ask everyone here, is fortran a dying language right now? I am an engineering student, will learning fortran be a waste of time?
I can confirm for you that Fortran is still used in academia, particularly in aerospace engineering, but long gone are the days that engineers actually seem to care about writing fast programs themselves. I see more people either using scripting languages or specialized software to solve their problems, instead of writing the code themselves.
The fact that you absolutely suck at writing python code probably doesn't help your case. Each language has it's benefits. Python is the fastest for prototyping and rapid development because of dynamic typing and a huge base of scientific libraries. If you really care about runtime, you can create custom C extensions to "cheat". Gives you best of both worlds.
Julia is another good one for comparison.
got recommended py/c/asm, watching this, pressing bell, off you go
First of, did like the video
However, 1:20 rubs me the wrong way.
In the real world, the ecosystem of libraries at your disposal is likely more important the the language itself.
Any comparison where you are not taking that into account might be entertaining but is not really informative to make a conclusive decision of which is faster.
In the end languages are standards and what you are comparing are implementations of that standard, not the language.
If there is a specialized Linear Algebra library that shreds through your task, you use it (regardless it be Python or Fortran)
If using Python lowers the bar for a user to use said library, its fair game and should be concidered.
If fortran has a exotic but useful Lin Alg library with hand assembly optimization, its fair game and should be considered.
Maybe try making a video comparing these implementations where your leveraging these libraries?
That is no cheating, using scipy library is not cheating. Using cython, numba or pypy is. And numba is not C, but an LLVM compiler for python.
It technically is cheating because most of the heavy calculation functions in SciPy are written in C/C++/Fortran. And when you compare how fast a language can crunch numbers, making that language run a program written in another language to do the hard work is def cheating. But Python isn't made to be a fast number crunching programming language so the comparison is pretty pointless anyway.
"free knockoff matlib thing"
-xmdi 2021
So Fortran smokes modern languages for things that Fortran is good at doing. What about things that Fortran isn't good at doing? It seems like the speed and efficiency of Fortran wins where Fortran is able to do things, but I'm sure there are a myriad of use-cases where Fortran can't even do certain types of things.
Python isn't the most popular programming language due to efficiency and speed, but because it is easy and approachable to learn how to code in it, and it can do a whole lot of different things reasonably well. And with modern computer hardware, the differences in speed shown in these tests won't show up in many day-to-day tasks.
Just my 2c.
Fortran is very simple. Implementation times are unbeatable and compile times are way faster than those of C++.
Use Cython bro then see the speed ☣️
C vs Fortran vs Julia
How about Julia vs Fortran vs C++?? And of course we should agree upon which c++ version b/c there are a lot of them ... xD
Fortran vs c vs rust pls
Try to code in FORTRAN IV rather than FORTRAN 95, that would be an experience!
Why not look at Julia?
Yeah for math Fortran is still one of the best for performance
python vs c/c++ vs golang plz
isn't using numpy cheating according to your definition as well?
Not if you use it wrong and iterate through the values with a for loop, apparently.
I think the performance comparison is not meaningful unless you run a task that will at least take the fastest language a few minutes. Otherwise we are just comparing some constant overhead.
May be you are using Cpython as interpreter. It's much better use pypy for fast speed. It's can posible that with pypy this results were similiar to fortran. Peace ;).
C is no 1 !!!!!
It must include C++ and C in this comparison.
When include C++, use STL!
Matlab is not Octave. They are code compatible, but the engines will run differently on different tasks