Optimizing Serial Code in Julia 1: Memory Models, Mutation, and Vectorization
Вставка
- Опубліковано 12 лис 2024
- In Fall 2020 and Spring 2021, this was MIT's 18.337J/6.338J: Parallel Computing and Scientific Machine Learning course. Now these lectures and notes serve as a standalone book resource.
github.com/Sci...
Chris Rackauckas, Massachusetts Institute of Technology
Additional information on these topics can be found at:
sciml.ai/ and other Julia programming language sites
Many of these descriptions originated on www.stochastic...
Amazing video!!! Thank you so much for sharing this. I made my code about x10 times faster just by implementing what Chris says in this first lecture. I didn't know that heap allocation takes so much time. Looking forward to next lectures.
Amazing lecture! Very clear and concise, thank you for sharing this publicly.
Amazing, I can't thank you enough.
I am in awe of just how much you know! Looking forward to the future lectures.
this already super insightful! thank you very much :)
So say I have data corresponding to spatiotemporal states like u_i(x,t) where 1
Now that Juno is gone what is the best environment to get all these variable value previews?
1:05:05 Interesting. I myself prefer FF6.
Can you please post the reference material. This is a great talk, please continue making Julia videos.
The reference material can all be found on the course website: github.com/mitmath/18337
In reference to the example where we have `val = [ A[i, j] + B[i, j] ]` and then later only use val[1]. Would it be a good idea for the compiler to infer this stuff and automatically optimize it away making it equivalent to the StaticArrays case ?
The term that you're looking for is escape analysis (en.wikipedia.org/wiki/Escape_analysis), which would be a compiler pass to prove the lifetime of a potentially heap-allocated variable and, if the size and lifetime is then known, potentially move it to the stack for the user. Escape analysis is fairly hard: a lot of research went into the topic because of the rise of Java making everything an object, and it wanted to heap allocate objects but still be as fast as C when possible by avoiding the heap allocation if it can prove short lifetimes. Given the history of Java's performance... it's clear to say it's not always possible and you cannot always rely on it to get a high performance program (Java generally gets between 5x-10x the runtime of C IIRC, a lot having to do with the compiler not fully optimizing away this memory pressure in a lot of the benchmarks. OO programming is generally a nightmare for heap allocations!). There is talk of adding an array freezing/unfreezing mechanism along with sophisticated tools for escape analysis that utilize these compiler guarantees as part of Julia 2.0 (github.com/JuliaLang/julia/pull/31630) but that is highly speculative of course. That would solve this exact case, but there are deeper cases where it can be harder to prove the lifetimes.
Glad I chance upon this lecture series because it was very informative! But I can't believe you claim ff8 > ff7 😡
1:18:09 what are the dollar signs in front of variables e.g. $A?
@59:34 it's not true that MATLAB throws a bounds error if you write to an array element that doesn't exist -- it instead resizes the array to be that size.
Thank you very much!
I think there is some confusion with terminlogies that deserves clarification. Broadcasting, vectorization are being interchanged.
Where I come from from (deep learning), broadcasting refers to extending an operation to other dimensions of one of two operands that don't have same shapes.
Example:
a has shape [4, 1]
B has shape [4, 10]
Then, broadcasting *allows* adding / multiplying such objects. The rationale is: *a* is a vector, *B* is an array of vectors. The addition is to happen between *a* and *columns* of B resulting in a matrix that looks like *B* with all its columns being equal to *a* + original column value.
Vectorization: a function designed to work on scalars, give me a version of it that works fast on vectors by feeding the vectors directly instead of looping over their elements and calling the function just as many times.
They might be similar at core, but I have noticed that in some specific fields, the disinction is due.
Isn't vectorisation just a special case of broadcasting? In which case the usage seems correct to me.
@@torstenschenkel2110 After a second thought, I see you are right, or may be it is the other way around. I claim this because I see that vectorization seem to be something that you explicitly ask for via "." notation, while broadcasting happens spontaneously whenever the dimensions are condusive. This is the case Julia, Numpy, TensorFlow, Pytorch.
LOL 36m50s... can't redefine function g... no problem, just rename it gg.