The 100k Q and A form is here: forms.gle/dHnWwszzfHUqFKny7 Please consider sharing, liking, commenting on the video - this is probably one of my favorite videos on my channel. By the way, if you are wondering why (A^T)^T = A, this will require the double duals, or co-covectors and the "canonical isomorphism". Essentially, the co-covectors are machines that measures covectors, and they can be thought of as vectors themselves. The explanation is not that visual, and so I am probably not making videos on the double duals.
Standard linear algebra course should start from a very nice amateur 😂definition of vector as a co-covector - (1,0) tensor. Besides jokes if you could give some visual intuition for a general tensor definition - it would be cool !)
3b1b should be watching while learning each topic from your textbook or other video. Their main focus is granting you more intution and understanding. Not fully learning a concept.
This makes it a little more complicated then it actually is. Here it is simply - Every vector space V has a dual space V* - Elements of the dual space are called dual vectors or co-vectors. When you think of dual space, just think of this as the space of row vectors, while the original space is the space of column vectors. - Every linear map from V -> W induces a linear map between the dual spaces W* -> V* called the transpose. - Every finite dimensional vector space is isomorphic to its dual, so we can really view the transpose as going from W -> V One more point: Row vectors eat column vectors and return a scalar. In R^2, you can visualize row vectors as equally spaced stacks of planes with one through the origin, and the application of the row vector to the column vector is the number of planes that the column vector pierces. This is the intuition the video is trying to get at.
Thanks for clarifying! I had a big "sigh" while the chosen way to present the topic was the "physics-inspired" one, when the algebraic one would have been much easier like you wrote. Of course visualization is important, but sometimes it just clobbers things up IMHO.
I think there’s something worth to be mentioned about the second to last point. In general vector spaces, the isomorphism between a vector space and its dual is not natural. One would need to specify a basis for the vector space. Only when the vector space is endowed with an inner product is the isomorphism natural.
I would definitely call that a "simple but doesn't explain anything". It doesn't convey any meaning, you don't even say how these things exist, but more importantly why we would care about doing such things. That's probably how classic textbook math presents it, but it doesn't explain much, that's why visual explainatory math videos by the likes of him or 3B1B are amazing, they actually explain the intuition behind these ideas.
@@JthElement Eh, it really depends on the person. I actually found this comment easier to understand than the video, but different people prefer different methods, so I'm glad both of them exist. The problem to me is that I have to keep translating the geometric explanation to the algebraic one while watching the video. The problem is that this concept is just too complex for a geometric explanation to really explain it well, (at least for me, you may feel differently). And just so it's clear, I appreciated the video, but it made more sense after I read this comment. For example, the explanation of why A^T * A = I for orthogonal matrices was really insightful.
In Haskell matrix transpose is called `zip`, because of how it unzips a list of M N-tuples into a N-tuple of N lists of scalars. Very useful in general programming, even if you never think about it being linear algebra.
I've always had a hard time visualizing matrices as vector transformations, but you do a very good job with that here. You also do a terrific job of explaining the transpose as an inverse vector transformation rather than as a new matrix. Great video!
I commend you for a simple and overlooked process and revealing the hidden intricacies as to why it happens to look like ‘just switching rows and columns’. There’s alway a reason we do something in math and when we are first introduced to new concepts it are often give the how before we are given the why. Keep up the good work and please continue on this trend of explaining the hidden reasons why we some procedures that are surely taken for granted
In my opinion, the explanations are a bit ambiguous, at least to me. I am not a native English speaker, I don't know how other people feel, but I think what confuses me the most is how you chose very long, compound sentences, which made it hard for me to follow. I also feel like I was not explained the terms like "co-vector", "measurement",... and sometimes I have no idea what is being talking about. I really appreciate the hard work you put into this video though.
Okay i just watched the video a few more times. I think i should have focused more on the visualization. It is a lot easier to understand with the animation.
There is a difference between transpose and adjoint that should not be swept under the rug. It all boils down to the inner product. If the inner product is just multiply and sum, then yes, transpose is adjoint. If it's more complicated though, the adjoint follows suit.
To be more specific about the "more complicated", for example, the dot product in the complex vector space is not just a multiply and sum, since you have to take conjugates. But in this case, the adjoint is the conjugate transpose, so not too different.
This is why I said something like you have to restrict the space of covectors the "transpose" acts on. That refers to covectors of the form , where you can directly make a 1-1 correspondence with the vectors, even though in infinite dimensional spaces, there are covectors not of this specific form. In fact, the whole last chapter is actually about that inner product! (actually, more generally, a symmetric bilinear form) I just did not call it that way, and said something about the "pairing" between vectors and covectors, which is only doable with some notion of inner product anyway (or generally, a symmetric bilinear form).
@@mathemaniac Well I was referring to covectors of this form specifically, just with a different inner product. I didn't even think about the crazier stuff in infinite dimensions! I love these deep things lurking in corners of seemingly mundane stuff.
@@btnt5209 That's not coincidental, you can derive the adjoint from the inner product and it will follow its form. For instance, if = x^T Hy, then A* = H^-1 A^T H.
Agreed. Of all you want is the transpose, stay with duals (as row vectors) and not their adjoints: w=vA Then just transpose the whole equation. But better: keep the rows and just multiply backwards.
Could you please make a video about tensors? I don't like the usual introduction to it, and I feel that you could give a more intuitive and tangible approach. Thank you for your work, it is amazing. The 13:30 got me real good 😄
The main issue with learning about tensors is that the word means different things in different fields, so when you look for information on tensors you receive conflicting preliminaries and definitions. In physical science, tensors are considered a "generalization" of vectors in the sense that they have more indices, but their mathematical properties are identical to (mathematical) vectors. For mathematicians, contravariant tensors are best considered as the dual of multilinear forms: for a vector space V and field F, - A multilinear k-form or (covariant) k-tensor is a function f: Vᵏ -> F that varies linearly with respect to each of its components - A contravariant k-tensor is a member of Vᵏ, often considered as the input to a multilinear k-form (or covariant tensor). In mathematics, a tensor is *almost always* a covariant tensor, while in physical science, it is *always* a contravariant tensor.
Check out the Tensor Calculus series by Prof Pavel Grinfeld on his YT channel MathTheBeautiful. I deal with tensors since 2011 and seeing his lectures was totally eye-opening. I've never ever seen such a clear and didactic but also deep and rigorous approach. In all I do there's a clear distinction between before and after I came across his channel. Only now I know what Einstein meant by "horse of true mathematics" in a letter he wrote to Levi-Civita (as the tale tells, at least). I cannot recommend Prof Grinfeld's channel enough.
to me linearity and its duality between vector spaces is the most mesmerizing thing in all of math- especially because it is the motivation behind so many more complex entities like differential equations and laplace transforms which rely on those principles for their unshakeable power and accuracy for everything from interpolation to analysis
Oh boy you have no idea how much I’ve needed a video about this exact series of topics. Combining this with differential forms + knot theory and we about to *transcend.*
This was just a bit difficult to grasp at first, but then I got the feel for it. Linear Algebra never stops asking more and more questions, and no wonder why I like it so much. A video on metrics would really help, considering the fact that I have to tame Special Relativity next semester.
Covectors are, put it simply, coordinates on a vector space: this is, in my opinion, the simplest way to think about them. In matrix notation, if vectors are given by column matrices then covectors are given by row matrices, so that transposition stablishes the duality between vectors and covectors v |-> v^T. One can readily deduce the meaning of the transpose matrix from this: (Av)^T = v^T A^T
18:00 the fact that when refering to 'rotational' matrixes the transpose is the inverse implies that QtQ = I (if Q is orthogonal) because orthogonal matrixes can be viewed as rotations. Really great video, greetings from Spain
Keep up the great work Mathemaniac as an early undergrad your content is at a perfect level and it’s so interesting different interpretations of linear algebra from those I’ve already seen.
Try and look up a category theory approach if you want to learn a new perspective. There you could for example learn that what he this video is giving intuition for is the way the dual functor maps lineare maps to maps in the dual space. In general it is a very abstract viewpoint and can be incredibly beuatiful if you understand it.
Nice geometric interpretation of the transpose. Two comments: 1) you can get the components of the one form directly from the intercepts of the c=1 hyperplane. This can be used for a straightforward demonstration that the adjoint matrix is the transpose. 2) you mentioned linearity of the forms as functions, but they are also linear objects in the sense that they add like vectors. Both properties were used.
The time when i am looking for all resources available on internet for covectors , i am learning Special and General relativity kind of in transitioning between them , this video helped me🥰
These past two semesters at uni, we covered the book "Linear algebra done right", by Sheldon Axler (it is free, btw. def recommend it!). As suggested in the preface to the teacher at the beginning of the book, we skipped over the section detailing quotients and duality due to the limited time span. This concept of covectors and transposition gives off the vibes of adjoint, which I had trouble grasping. The mathematics is really interesting, especially in the context of inner products. But it is very difficult to digest. This is a good video and will undoubtedly benefit others in the future. I love algebra and hope to see more of it! Edit: I picked up my text, and I am thinking of the matrix conjugate transpose, which is also related to the adjoint. For a linear mapping T, with its adjoint T*, there exists a matrix M which is the matrix representation of T (to and from some bases). E.g, if we perform a change of basis from B to C, then the matrix representation of T is written M(T; B, C). Then, the matrix conjugate transpose is M(T*; C, B). This can be found on page 208 of Axler's third edition book.
The reason why you think of adjoint as conjugate transpose is only because in complex spaces, the usual inner product is = u(dagger)v. In real spaces, the adjoint is exactly the transpose!
We used all of axler except quotients in our second sem first year course. I thought the dual spaces chapter was the most fascinating and strongly recommend it. I have summarised it in the my reply to you elsewhere though. It was interesting watching this video as the scales analogy screamed dual map to me but the later video screamed adjoint map. From my other comment the matrix for the dual map with respect to the canonical dual basis is always the transpose even under C whilst the adjoint map’s matrix as you said is conjugate transpose
@@jacb2997 Yeah, same with us! Some of the later parts of the book are skipped for brevity, but we go through to SVD decomposition. I believe my uni introduces quotients in our standalone algebra course, but I am not too sure. I will certainly be giving the chapter a read after my end-of-semester exams! Thanks for your replies. They have been great
My heart still lies with the view of the transpose that comes out of categorical quantum mechanics (somewhat akin to this one, given the emphasis on the notion of "measurement"), but this one is very nice (and nearly as intuitive) as well. Great video!
Well, the frustration is still there: the main thing is I am, at least algorithmically, discouraged to experiment with new things. But I just put down the frustration a little bit, and return to what I'm always doing.
*NATURAL EXPLANATION OF COVECTORS AND TRANSPOSITION FOLLOWS* The title caught my curiosity but, even though I tried to follow the arguments, they felt very prolonged and awkward,. I was ultimately left confused. I believe this is the second video I watch from this channel. The other one, on Galois theory, was a good outline but I found this to obfuscate a very simple concept. *Less technical paragraph after this* If I'm not mistaken, that concept would be the contravariant functor that maps vector spaces to their dual spaces and linear maps to their dual maps. If we denote by A the linear map V -> W, then the dual map A*: W* -> V* arises by taking covector a in W* (the dual of W) to covector b=W*a in V* with the definition b(v)=a(Av) for vector v in V. Put simply, you tell the image in V* of a covector a in W* to do on each vector v in V what a does on the image Av in W. This is a very natural construction and by representing covectors as column vectors then writing out the equation for each component of b, it can be quickly derived that A*=A^T. But this is also the most confusing thing I found in this video: that both vectors and covectors are represented as column vectors. But apples are apples and bananas are bananas. If we just represent covectors as row vectors then covector (a b) acting on vector (x y)^T is just the usual matrix multiplication (a b)(x y)^T=ax+by. Then transformations are represented from right multiplication, that is, bA*(which is actually =bA as shown later). This is clear because if A is an m x n matrix that maps n dim vectors, the m dimensional covectors should be transformed to n dimensional, and the only way to do that with an m x n matrix is by multiplying from the right. Now look how simply transposition appears (using the notation from above): bA=(A^Tb^T)^T. That simply says if we have row vector representation of covectors and matrix multiplication from the right, if we insist on writing covectors as column vectors (like in this video) then matrix multiplication from the left of the transpose gives us the mapped covector in column vector form, and transposing that back to row vector gives the covector in its natural representation. So it is derivable from the definition of matrix multiplication and transposition.
As I showed (albeit briefly) in the video, it is a very deliberate choice to represent covectors as column vectors. The reason is we should not ever be considering covectors as row vectors, even if it **seems** simpler to understand. Even you have said that you can only see that transpose connection by "representing covectors as column vectors". Most people are more comfortable with applying transformations on the left, and it makes more sense to do so: after all, we apply functions on the left: x -> f(x). It makes much less sense to apply transformation on the right, and you have to resort to how matrix multiplication works technically (using dimensions and stuff), rather than the **intuition** given by matrix multiplication, which is a function! Plus, representing as rows means somehow you have to use (AB)^T = B^TA^T. This is actually intuitive if you think of the bathroom scale analogy in the video, and treating A and B as linear transformations. However, you are using it as (Ab)^T = b^T A^T, where b is a column vector, and unless you are thinking of b as a linear transformation, the intuition breaks down, and you are treating (AB)^T = B^TA^T as something purely algebraic, rather than having an intuition for it. Put simply, I don't think there is any intuition for v --> vA, for v a row vector, other than "oh it works, look at the dimensions, they match!", which isn't really an intuition. On the other hand, thinking of covectors as column vectors b, then b --> A^T b, and it would be "yes! This is the linear transformation I am familiar with!". I think the reason why it feels confusing is simply that most people are only thinking of matrices as a bunch of numbers with a specified rule for multiplication, rather than actually thinking it as a transformation, or having an actual intuition for it. Sure, that's fair enough actually, and you can think of it just purely algebraically, which again is fair enough in some applications, but the point of the video is to give the intuition of what transpose / covectors are actually doing when you have the intuition that matrices are linear transformations, and only using that intuition and **never** computations.
@@mathemaniac You are right, it seems like pure algebraic manipulation. Let me try to define precisely what column and row vectors, as well as matrices, represent. Then transposition will be defined as a very simple operation and everything will make intuitive sense, should I succeed. So bear with me. Let m x n matrices be representations of a linear transformation such that composition is matrix multiplication. Note that I do not specify what it transforms to. If you think about it, a covector is a linear transformation from a vector space V to the vector space R (or any other field). But the dual of the dual of a vector space is that vector space (finite dimensions, ofc). So a vector is also a linear transformation from a dual space (of covectors) to (the covector space) R. So it makes sense to treat both vectors and covectors as matrices (i.e. transformations). Let us arbitrarily decide vectors are the n x 1 matrices, and covectors the 1 x m matrices. This just decides how we interpret each direction in an expression of matrices. Then ABv is the composition f(g(v)) for a vector, but aAB is the composition g*(f*(a)), where f(v)=Av and the "dual" (will define in a bit) f*(a)=aA. In terms of matrix multiplication, an m x n A is a linear transformation of n dim vectors to m dim vectors if we multiply from the left. But it is clear that A gives rise to a transformation of m dim covectors to n dim covectors when we multiply by A from the right. Let's call this the dual of A. But we already have a term called the dual of a linear transformation as I defined in the original comment, given by the functor from the category of vector spaces to dual spaces. So am I abusing the term dual by using it in two different senses? No, because they are the same! It's really easy to show, almost too easy: Let v in V, w in W, b in V* and a in W*. We construct the dual of the map represented by A by letting for any v b(v)=a(Av). In the matrix notation where vectors and covectors are also matrices, we want a matrix B such that (aB)(v)=a(Av). From associativity of matrix multiplication, it's really easy to see that B=A! That is, (aA)v=a(Av). We did not have to do anything! The dual of left multiplication is right multiplication. It is just a matter of interpretation of the matrix expressions. But interpreting column vectors as vectors is arbitrary. We could have called them covectors and the row vectors just vectors. It only changes how we interpret matrices and the direction of matrix expressions. And when we want to flip what they mean, we can devise an operation for this. And let's call this magical operation 'transposition.' It works perfectly because the two directions are perfectly symmetric. And this is because which one we call a vector and which one a covector is also perfectly arbitrary in finite dimensions. What do you think? Does this give intuitive meaning to the purely algebraic manipulations now?
Who would have umagined I would be binge watching math videos on late weekend nights, they are addictive! Understanding is addiction - a good one though.
And actually, pretty annoyed at myself for not having subbed before, ive watched (and remember watching!) a few of your videos before. Thanks for the content
I always wondered why linear algebra cannot be illustrated using real life example with numbers that represent real objects such as how a tradsperson would make measurements and then transpose these measurements for other construction. For instance, a 2×2 matrix as two restaurant selling two different meals, and when it's transpose or its inverse is required, what are those changes in respect to the restaurants and the meals. Hope I have made myself reasonably clear. When too many math jargons are used, it may become lots of " hand wavings" and novice like me will get lost. Thank you.
It feels so satisfying after understanding the idea of transpose.. Keep up the amazing work. Also, are you planning on explaining Adjoint in the similar way? It would be a great help!!
The concept presentation is novel and wonderful. Deep insight. I did not get the opportunity to learn these concepts in my student and research scholar days. A great service to workers in basic sciences. We pay our sincere gratitude to 3b1b.
People start doing matrices and totally forget that in the basics. matrices is just a bunch of linear equations But with coefficients only.... and realizing that, [A] and [A]^t is either one function out of a list of functions of some sort, or a list of all linear components that correspond with its basis vector. if you were to read only the row of [A] and [A]^t.
Very good video! First one I see from you, but you earned my subscription. I’ll have to rewatch a couple of times because I’m a little rusty on the abstract side of Linear Algebra and I’m sure I’m missing some important subtleties. I loved the effort you put into the “big picture” chapter! If I may, I’d like to make a couple suggestions: - Master “the pause”, by allowing a few seconds when you deliver punchlines, to allow us to digest what was just said. Your rhythm overall is great, but I felt you just kept talking non-stop at those moments. (I’m sure that’s also a consequence of my rusty L.A.) - In your intro, please mention the “prerequisites” and the target audience. These concepts are deceptively simple at first glance, but it became clear I was missing something as the video progressed, due to my somewhat forgotten command of precise definitions. Of course, the lack of knowledge is entirely my fault, but I didn’t realize I needed it before starting watching the video (after all, I thought I knew perfectly well what transposes were, lol). Keep up the good work!
Could you do a video on visualising the transformations performed by (A transpose times A) and (A times A transpose), and perhaps extend that to the general projection matrix?
I’ve been studying Lin Alg for a little while now but I never quite saw transpose that way.. this was incredibly refreshing.. and for a moment I thought you were going to bring Eigenvalues and Eigenvectors into the fold, but you didn’t (even though we both know they were hiding there in plain sight now, didn’t they? 😉)
It's also interesting to consider cases, when matrices linearly map vectors to bivectors and back. Such as when dealing with transformation of a force into a torque (in 2d, or in 4d, since amount of bivector components are different there from amount of vector components). I've seen transpose of such matrix being used to figure out effective mass from the inertia tensor and position to which force is applied (1/effectiveMass = transpose(toTorque) × inertia × toTorque), where inertia is just a symmetric matrix (aka orthogonal resizer) that turns torque bivector into a resized angular velocity bivector. ToTorque represents geometric algebra wedge of a force-vector by offset-vector. And inverse effective mass just shows how much of tangential velocity unit force imparts onto the object (ignoring linear velocity). Anyway, sorry for the unclear monologue; I'm just too tired after a long day and I wanted to share a little piece of cool info I recently learned.
Completely lost sorry, no idea what you're on about. Sometimes you'd say the most simple thing, like explain matrix vector multiplication or a linear combination but I've absolutely no idea what this "measuring device" was supposed to be and you're coming back to it like it's established or an actual mathematical object. With the transpose you're going from column space to row space obviously. Maybe that's what you meant. And obviously the relationship between the vectors is preserved, it's just a reflection. I suspect you've just over-thought this and fallen down a rabbit hole of waffle. Who knows, maybe it all makes sense and it's me but I'm stopping here.
I agree. This video is overly complicated than what it is supposed to be. Measuring device is also very bad analogy since he is introducing extra input parameter into the system which did not exist when he was defining its concept.
All these concepts would be easier to learn and memorize on college if professors have invested time and energy into visualizing things for students the way you did here.
Well Profs don't need to do those everyday. If a view like you, 3b1b, math sorcerer etc. cover most stuff you can just link the videos and get all the intuition you need. ^^ much better than the good old sentence: "that's easy too see" after you copied half a dozen blackboards in 90min 5x a day. I studied while videos like this were emerging and they are awesome to gain intuition and boost confidence in all the gaps a lecture leaves. Just recap the video and you are usually good to go to keep solving problems.
I loved watching this video but I need something that will lead into it for me as I never studied this topic before. I studied Physics 50 years ago and am currently taking a QM class so need to understand matrices more deeply particularly Hermitian operators and what adjoint, transpose etc reaally mean. This seemed clear as I watched it but I haven't really grasped the meaning.
If you consider the space of square integrable functions (countably infinite, Hilbert space, for example, vectors of Fourier coefficients), the dual vectors include non-funtion distributions like the Dirac delta function. This is a dense, uncountable basis (one for each real number).
I believe this is the corollary of Hahn-Banach Theorem. Corollary is as follows; For a given m > 0 and a vector x_0 in the normed space, there exists a functional(covector) such that ||f|| = m and f(x_0) = m ||x_0|.
Hello, I am a bit confused with your introduction, it looks like you are looking for a substitute to composition, which you do not mention at all. However i really like the column vector representation for linear forms as their matrix expressions involve the transpose of these columns for a dot product: let µ be our measurement, of vector X, t is the transposition, then µ(Y) = t(X) . Y and µ(alpha(Y)) = t(X) . A . Y = t(t(A) . X) . Y therefore t(A) . X is the vector of our "same measurement" in the starting space and the rank of A is not a limitation.
Thanks for a video with great visuals. To add to it, I strongly recommend the Tensor Calculus series from the YT channel MathTheBeautiful by Prof Pavel Grinfeld. I've been studying these topics for many years and, to me, there is currently a plethora of notations and names that shuffle conventions from vector calculus, linear algebra, etc. - often making the topic very confusing. He employs a clear and unifying notation that it makes it VERY EASY to understand covectors, metrics, inverses, transposes... I cannot recommend it enough.
The point of the transpose is to move the measuring device back to the original space, so that you can do the measurements in the original space. So you ask, what is the operation so that the measurements remain the same, but the basis vectors have not changed? That would be the inverse. As I said later on in the video, it only looks like inverse, but you have to treat those gridlines as covectors.
i have difficulty conceptually separating vectors and covectors when, in nearly all vector algebra i've used, there's been zero meaningful distinction between them in practice for one: with matrix notation, whether covectors are the "row vectors" or the "column vectors" is entirely arbitrary, and likewise with "bras" and "kets" in bra-ket notation - covectors with no corresponding vector didn't seem to come up in QM, for me, but maybe i missed them - but since the algebra doesn't seem to suggest an inherent distinction on its own, it feels like weird human artifice to impose it for another: when dealing with special relativity, you can bake the metric directly into your basis elements if you don't restrict yourself to standard linear algebra, removing the metric tensor from consideration entirely, and allowing every vector and its corresponding "covector" to be fundamentally equal - this was extra useful to me, because many applications of the metric tensor are very dependent on situational "covariance" and "contravariance", and these concepts were _never_ explained in a useful manner or clearly demonstrated within the algebra itself, so it just looked highly arbitrary at worst and fudged by-eye at best baking the metric directly into the basis also allowed me to discard the cross-product in favour of an operation that makes sense and works in more than 3 dimensions, but that's a different matter (also, i have a somewhat better grasp of covariance and contravariance now, because using explicit bases instead of implicit ones seems almost mandatory in genuinely demonstrating the covariant and contravariant parts of a "basis transformation", which it turns out, when done correctly, is less a transformation than it is a fun notational trick that transforms nothing except the way things are written down)
expanding slightly on the second point, about the metric tensor, now that i remember it slightly better: when taking a vector-gradient of the electromagnetic 4-potential, versus taking the divergence of the resulting electromagnetic tensor, the metric tensor would switch between "covariant" and "contravariant" forms (by inverting the tensor in some capacity, i think), and most everywhere treated this like it should be absolutely obvious when and why it is doing that - not to mention how unintuitive the missing minus-signs were for one of the forms when i construct an equivalent gradient operator using geometric algebra, with the metric baked into the basis elements (their "quadratic form"), i need only understand that the basis vectors in each term are actually _multiplicatively inverse*_ (not obvious in most explanations of the gradient): we don't have x̂ d/dx, but rather 1/x̂ d/dx, for example, and the necessary signs emerge naturally from the metric thereafter and neither do i need to do anything to that operator before also applying it to the resulting electromagnetic bivector (for its divergence): the elements handle the metric _inherently_ it's much cleaner, and its easier for me to understand the math because it feels like the math better understands itself *there're some details to this i have yet to work out, because i've been very depressed and possibly the only person on the planet who cares; if i am wrong in some capacity here, the easiest way for me to understand it might be to understand how (or whether!) the gradient operator can meaningfully exist for elementary null-vectors (that is, null-vectors that aren't just a sum of elements with opposing non-null metrics) - given a null-metric shouldn't have an invertible metric tensor either, i've been assuming it cannot
Very nice video about not so easy to grasp concepts: The content of this video and more on coordinate transform, dual spaces, transpose maps, kinematics with a rigorous full-concept-notation (rather than proof complete) for mechanics and robotics can be found in appendix D in DOI: 10.3929/ethz-a-010662262
Very good explanation, but at 1:20 you say that the measuring device takes a vector and takes it into a number. Then in subsequent slides, the device is treated as something that takes a vector to a convector. Any chance of explaining that?
for me the difficulty has always been with "covector" aka "dual" aka "1-form", and I think its because I wasn't exposed to them until college, while vectors, dot-products, and cross-products where presented in high school, where "dot product" doesn't differentiate at all between vectors and covectors. And then once exposed the question becomes why choose any specific mapping between vectors and covectors, once you decide you need to do this you always end up with the simplistic choice of the identity mapping, and then wonder what all the fuss was about, abstraction for the sake of abstraction without any benefit is what it seems to be.
I love your videos, generally. However, the density of the use of the words 'this,' 'that,' 'those,' 'here,' 'it,' and other ambiguous references makes your examples more taxing. But I understand the idea-a beautiful concept.
You say in infinite dimensional vector spaces we have many more vectors than co-vectors. Is this true in general? I was under the impression that it is essentially arbitrary which space we consider the dual. And in infinite dimensional (complex) vector spaces we encounter another quirk. We find operators that have no Eigenvectors in either the space or the dual. They either have left or right eigenvectors. An example from quantum mechanics are the lowering operator, which only has right eigenvectors, and its adjoint, the raising operator, which only has left eigenvectors. This is a concept that I am having trouble understanding, as the characteristic polynomial should just give us the Eigenvalues and not care about left or right?
Many more *covectors* than vectors (the other way around). See the description for the explanation. There are more covectors than vectors precisely when the vector space is infinite-dimensional. How do you have the characteristic polynomial in infinite dimensions?
@@narfwhals7843 That's not all the duals in the infinite dimensional case. If we go back to the wavefunctions in 1 dimension (to make things a bit more concrete), then the function psi --> psi (0) is a perfectly good dual [it is linear, and it sends a wavefunction to a number], but you can't write it in terms of
The ladder operators are defective matrices in finite dimensional spaces (thus missing eigenvectors), but not in infinite spaces because it takes infinite operations to kill arbitrarily excited states, one level at a time. See the comment above on Hilbert spaces about th different sizes of spaces.
Bro my professor in 4th year of Engineering told us that even adjoint and adjugate of a matrix aren't the same thing. I mean he told the difference but i didn't understand. Make a video on that also
It decomposes the determinant into a contraction, giving A B = |A| I. The determinant can be visualized as the matrix acting on p-forms (cross or triple products). See Flanders for a beautiful introduction.
![The moment we stopped understanding AI [AlexNet]](/img/n.gif)
The 100k Q and A form is here: forms.gle/dHnWwszzfHUqFKny7
Please consider sharing, liking, commenting on the video - this is probably one of my favorite videos on my channel.
By the way, if you are wondering why (A^T)^T = A, this will require the double duals, or co-covectors and the "canonical isomorphism". Essentially, the co-covectors are machines that measures covectors, and they can be thought of as vectors themselves. The explanation is not that visual, and so I am probably not making videos on the double duals.
Standard linear algebra course should start from a very nice amateur 😂definition of vector as a co-covector - (1,0) tensor. Besides jokes if you could give some visual intuition for a general tensor definition - it would be cool !)
3b1b linear algebra series saved my life during college, but oh boy do i wish this have seen this video sooner
the 3b1b series was great but definitely left me wanting more
It was amazing but crucially left out transposition, SVD and QR decomposition
3b1b should be watching while learning each topic from your textbook or other video. Their main focus is granting you more intution and understanding. Not fully learning a concept.
@@TheFlemishNut definitely. Don’t rely only on his videos but they are great. They did help me a lot in understanding
This makes it a little more complicated then it actually is. Here it is simply
- Every vector space V has a dual space V*
- Elements of the dual space are called dual vectors or co-vectors. When you think of dual space, just think of this as the space of row vectors, while the original space is the space of column vectors.
- Every linear map from V -> W induces a linear map between the dual spaces W* -> V* called the transpose.
- Every finite dimensional vector space is isomorphic to its dual, so we can really view the transpose as going from W -> V
One more point:
Row vectors eat column vectors and return a scalar. In R^2, you can visualize row vectors as equally spaced stacks of planes with one through the origin, and the application of the row vector to the column vector is the number of planes that the column vector pierces. This is the intuition the video is trying to get at.
Thanks for clarifying! I had a big "sigh" while the chosen way to present the topic was the "physics-inspired" one, when the algebraic one would have been much easier like you wrote. Of course visualization is important, but sometimes it just clobbers things up IMHO.
I think there’s something worth to be mentioned about the second to last point. In general vector spaces, the isomorphism between a vector space and its dual is not natural. One would need to specify a basis for the vector space. Only when the vector space is endowed with an inner product is the isomorphism natural.
I would definitely call that a "simple but doesn't explain anything". It doesn't convey any meaning, you don't even say how these things exist, but more importantly why we would care about doing such things. That's probably how classic textbook math presents it, but it doesn't explain much, that's why visual explainatory math videos by the likes of him or 3B1B are amazing, they actually explain the intuition behind these ideas.
I agree. This was quite useless.
@@JthElement Eh, it really depends on the person. I actually found this comment easier to understand than the video, but different people prefer different methods, so I'm glad both of them exist. The problem to me is that I have to keep translating the geometric explanation to the algebraic one while watching the video. The problem is that this concept is just too complex for a geometric explanation to really explain it well, (at least for me, you may feel differently). And just so it's clear, I appreciated the video, but it made more sense after I read this comment. For example, the explanation of why A^T * A = I for orthogonal matrices was really insightful.
In Haskell matrix transpose is called `zip`, because of how it unzips a list of M N-tuples into a N-tuple of N lists of scalars. Very useful in general programming, even if you never think about it being linear algebra.
This video is EXACTLY what I've been looking for! Please don't stop making these!
This was actually very refreshing and a great way to visualise what a transpose really is, even if there are limitations.
What limitations?
I've always had a hard time visualizing matrices as vector transformations, but you do a very good job with that here. You also do a terrific job of explaining the transpose as an inverse vector transformation rather than as a new matrix. Great video!
I commend you for a simple and overlooked process and revealing the hidden intricacies as to why it happens to look like ‘just switching rows and columns’. There’s alway a reason we do something in math and when we are first introduced to new concepts it are often give the how before we are given the why.
Keep up the good work and please continue on this trend of explaining the hidden reasons why we some procedures that are surely taken for granted
In my opinion, the explanations are a bit ambiguous, at least to me. I am not a native English speaker, I don't know how other people feel, but I think what confuses me the most is how you chose very long, compound sentences, which made it hard for me to follow. I also feel like I was not explained the terms like "co-vector", "measurement",... and sometimes I have no idea what is being talking about. I really appreciate the hard work you put into this video though.
Okay i just watched the video a few more times. I think i should have focused more on the visualization. It is a lot easier to understand with the animation.
There is a difference between transpose and adjoint that should not be swept under the rug. It all boils down to the inner product. If the inner product is just multiply and sum, then yes, transpose is adjoint. If it's more complicated though, the adjoint follows suit.
To be more specific about the "more complicated", for example, the dot product in the complex vector space is not just a multiply and sum, since you have to take conjugates. But in this case, the adjoint is the conjugate transpose, so not too different.
This is why I said something like you have to restrict the space of covectors the "transpose" acts on. That refers to covectors of the form , where you can directly make a 1-1 correspondence with the vectors, even though in infinite dimensional spaces, there are covectors not of this specific form.
In fact, the whole last chapter is actually about that inner product! (actually, more generally, a symmetric bilinear form) I just did not call it that way, and said something about the "pairing" between vectors and covectors, which is only doable with some notion of inner product anyway (or generally, a symmetric bilinear form).
@@mathemaniac Well I was referring to covectors of this form specifically, just with a different inner product. I didn't even think about the crazier stuff in infinite dimensions! I love these deep things lurking in corners of seemingly mundane stuff.
@@btnt5209 That's not coincidental, you can derive the adjoint from the inner product and it will follow its form. For instance, if = x^T Hy, then A* = H^-1 A^T H.
Agreed. Of all you want is the transpose, stay with duals (as row vectors) and not their adjoints: w=vA Then just transpose the whole equation. But better: keep the rows and just multiply backwards.
Could you please make a video about tensors? I don't like the usual introduction to it, and I feel that you could give a more intuitive and tangible approach.
Thank you for your work, it is amazing. The 13:30 got me real good 😄
I will have to when I do general relativity some time in the future, but really tensor should not be described as "transforms like a tensor".
Eigenchris's explanations are the best I've found. I'm excited for Mathemaniac's take, though
The main issue with learning about tensors is that the word means different things in different fields, so when you look for information on tensors you receive conflicting preliminaries and definitions.
In physical science, tensors are considered a "generalization" of vectors in the sense that they have more indices, but their mathematical properties are identical to (mathematical) vectors. For mathematicians, contravariant tensors are best considered as the dual of multilinear forms: for a vector space V and field F,
- A multilinear k-form or (covariant) k-tensor is a function f: Vᵏ -> F that varies linearly with respect to each of its components
- A contravariant k-tensor is a member of Vᵏ, often considered as the input to a multilinear k-form (or covariant tensor).
In mathematics, a tensor is *almost always* a covariant tensor, while in physical science, it is *always* a contravariant tensor.
Check out the Tensor Calculus series by Prof Pavel Grinfeld on his YT channel MathTheBeautiful. I deal with tensors since 2011 and seeing his lectures was totally eye-opening. I've never ever seen such a clear and didactic but also deep and rigorous approach. In all I do there's a clear distinction between before and after I came across his channel. Only now I know what Einstein meant by "horse of true mathematics" in a letter he wrote to Levi-Civita (as the tale tells, at least). I cannot recommend Prof Grinfeld's channel enough.
Why don’t you make a video on tensors?
Your intention to teach deeper meaning of transpose is awesome, but lot of viewers are more confused now.
to me linearity and its duality between vector spaces is the most mesmerizing thing in all of math- especially because it is the motivation behind so many more complex entities like differential equations and laplace transforms which rely on those principles for their unshakeable power and accuracy for everything from interpolation to analysis
Oh boy you have no idea how much I’ve needed a video about this exact series of topics. Combining this with differential forms + knot theory and we about to *transcend.*
This was just a bit difficult to grasp at first, but then I got the feel for it. Linear Algebra never stops asking more and more questions, and no wonder why I like it so much. A video on metrics would really help, considering the fact that I have to tame Special Relativity next semester.
I have been searching for an explanation about transposé for à decade, thank you so much 😁
Covectors are, put it simply, coordinates on a vector space: this is, in my opinion, the simplest way to think about them. In matrix notation, if vectors are given by column matrices then covectors are given by row matrices, so that transposition stablishes the duality between vectors and covectors v |-> v^T. One can readily deduce the meaning of the transpose matrix from this: (Av)^T = v^T A^T
18:00 the fact that when refering to 'rotational' matrixes the transpose is the inverse implies that QtQ = I (if Q is orthogonal) because orthogonal matrixes can be viewed as rotations. Really great video, greetings from Spain
Keep up the great work Mathemaniac as an early undergrad your content is at a perfect level and it’s so interesting different interpretations of linear algebra from those I’ve already seen.
Try and look up a category theory approach if you want to learn a new perspective. There you could for example learn that what he this video is giving intuition for is the way the dual functor maps lineare maps to maps in the dual space. In general it is a very abstract viewpoint and can be incredibly beuatiful if you understand it.
@@roybean9983 I intend to eventually
Nice geometric interpretation of the transpose. Two comments: 1) you can get the components of the one form directly from the intercepts of the c=1 hyperplane. This can be used for a straightforward demonstration that the adjoint matrix is the transpose. 2) you mentioned linearity of the forms as functions, but they are also linear objects in the sense that they add like vectors. Both properties were used.
The time when i am looking for all resources available on internet for covectors , i am learning Special and General relativity kind of in transitioning between them , this video helped me🥰
These past two semesters at uni, we covered the book "Linear algebra done right", by Sheldon Axler (it is free, btw. def recommend it!). As suggested in the preface to the teacher at the beginning of the book, we skipped over the section detailing quotients and duality due to the limited time span. This concept of covectors and transposition gives off the vibes of adjoint, which I had trouble grasping. The mathematics is really interesting, especially in the context of inner products. But it is very difficult to digest. This is a good video and will undoubtedly benefit others in the future. I love algebra and hope to see more of it!
Edit: I picked up my text, and I am thinking of the matrix conjugate transpose, which is also related to the adjoint. For a linear mapping T, with its adjoint T*, there exists a matrix M which is the matrix representation of T (to and from some bases). E.g, if we perform a change of basis from B to C, then the matrix representation of T is written M(T; B, C). Then, the matrix conjugate transpose is M(T*; C, B). This can be found on page 208 of Axler's third edition book.
The reason why you think of adjoint as conjugate transpose is only because in complex spaces, the usual inner product is = u(dagger)v. In real spaces, the adjoint is exactly the transpose!
@@mathemaniac Thanks for that! I will certainly give the section in textbook a read when I have the time. It is really interesting stuff!
We used all of axler except quotients in our second sem first year course. I thought the dual spaces chapter was the most fascinating and strongly recommend it. I have summarised it in the my reply to you elsewhere though. It was interesting watching this video as the scales analogy screamed dual map to me but the later video screamed adjoint map. From my other comment the matrix for the dual map with respect to the canonical dual basis is always the transpose even under C whilst the adjoint map’s matrix as you said is conjugate transpose
@@jacb2997 Yeah, same with us! Some of the later parts of the book are skipped for brevity, but we go through to SVD decomposition. I believe my uni introduces quotients in our standalone algebra course, but I am not too sure. I will certainly be giving the chapter a read after my end-of-semester exams! Thanks for your replies. They have been great
My heart still lies with the view of the transpose that comes out of categorical quantum mechanics (somewhat akin to this one, given the emphasis on the notion of "measurement"), but this one is very nice (and nearly as intuitive) as well. Great video!
I’m glad that you didn’t get frustrated by the low view of videos! Keep going man I love your math videos
Low views ? He's getting at least 100 k per video that's incredible for a math channel
@@vinvic1578 He once posted something in the "community" go check it out
Well, the frustration is still there: the main thing is I am, at least algorithmically, discouraged to experiment with new things. But I just put down the frustration a little bit, and return to what I'm always doing.
@@zhuolovesmath7483 ohh I see thanks for the information, apologies i didn't know
@@mathemaniac Always support you!
1:05 to 1:52 best explanation of duality in linear algebra.❤
Really clarified why transposing the matrix of a linear map has to do with the matrix dual linear map!! Nice video well done
Msterpiece!
Thank you for this huge work with the topic and animations!
*NATURAL EXPLANATION OF COVECTORS AND TRANSPOSITION FOLLOWS*
The title caught my curiosity but, even though I tried to follow the arguments, they felt very prolonged and awkward,. I was ultimately left confused. I believe this is the second video I watch from this channel. The other one, on Galois theory, was a good outline but I found this to obfuscate a very simple concept.
*Less technical paragraph after this*
If I'm not mistaken, that concept would be the contravariant functor that maps vector spaces to their dual spaces and linear maps to their dual maps. If we denote by A the linear map V -> W, then the dual map A*: W* -> V* arises by taking covector a in W* (the dual of W) to covector b=W*a in V* with the definition
b(v)=a(Av)
for vector v in V. Put simply, you tell the image in V* of a covector a in W* to do on each vector v in V what a does on the image Av in W.
This is a very natural construction and by representing covectors as column vectors then writing out the equation for each component of b, it can be quickly derived that A*=A^T. But this is also the most confusing thing I found in this video: that both vectors and covectors are represented as column vectors. But apples are apples and bananas are bananas.
If we just represent covectors as row vectors then covector (a b) acting on vector (x y)^T is just the usual matrix multiplication (a b)(x y)^T=ax+by. Then transformations are represented from right multiplication, that is, bA*(which is actually =bA as shown later). This is clear because if A is an m x n matrix that maps n dim vectors, the m dimensional covectors should be transformed to n dimensional, and the only way to do that with an m x n matrix is by multiplying from the right.
Now look how simply transposition appears (using the notation from above):
bA=(A^Tb^T)^T.
That simply says if we have row vector representation of covectors and matrix multiplication from the right, if we insist on writing covectors as column vectors (like in this video) then matrix multiplication from the left of the transpose gives us the mapped covector in column vector form, and transposing that back to row vector gives the covector in its natural representation. So it is derivable from the definition of matrix multiplication and transposition.
As I showed (albeit briefly) in the video, it is a very deliberate choice to represent covectors as column vectors. The reason is we should not ever be considering covectors as row vectors, even if it **seems** simpler to understand.
Even you have said that you can only see that transpose connection by "representing covectors as column vectors". Most people are more comfortable with applying transformations on the left, and it makes more sense to do so: after all, we apply functions on the left: x -> f(x). It makes much less sense to apply transformation on the right, and you have to resort to how matrix multiplication works technically (using dimensions and stuff), rather than the **intuition** given by matrix multiplication, which is a function!
Plus, representing as rows means somehow you have to use (AB)^T = B^TA^T. This is actually intuitive if you think of the bathroom scale analogy in the video, and treating A and B as linear transformations. However, you are using it as (Ab)^T = b^T A^T, where b is a column vector, and unless you are thinking of b as a linear transformation, the intuition breaks down, and you are treating (AB)^T = B^TA^T as something purely algebraic, rather than having an intuition for it.
Put simply, I don't think there is any intuition for v --> vA, for v a row vector, other than "oh it works, look at the dimensions, they match!", which isn't really an intuition. On the other hand, thinking of covectors as column vectors b, then b --> A^T b, and it would be "yes! This is the linear transformation I am familiar with!".
I think the reason why it feels confusing is simply that most people are only thinking of matrices as a bunch of numbers with a specified rule for multiplication, rather than actually thinking it as a transformation, or having an actual intuition for it. Sure, that's fair enough actually, and you can think of it just purely algebraically, which again is fair enough in some applications, but the point of the video is to give the intuition of what transpose / covectors are actually doing when you have the intuition that matrices are linear transformations, and only using that intuition and **never** computations.
@@mathemaniac You are right, it seems like pure algebraic manipulation. Let me try to define precisely what column and row vectors, as well as matrices, represent. Then transposition will be defined as a very simple operation and everything will make intuitive sense, should I succeed. So bear with me.
Let m x n matrices be representations of a linear transformation such that composition is matrix multiplication. Note that I do not specify what it transforms to. If you think about it, a covector is a linear transformation from a vector space V to the vector space R (or any other field). But the dual of the dual of a vector space is that vector space (finite dimensions, ofc). So a vector is also a linear transformation from a dual space (of covectors) to (the covector space) R. So it makes sense to treat both vectors and covectors as matrices (i.e. transformations). Let us arbitrarily decide vectors are the n x 1 matrices, and covectors the 1 x m matrices. This just decides how we interpret each direction in an expression of matrices.
Then ABv is the composition f(g(v)) for a vector, but aAB is the composition g*(f*(a)), where f(v)=Av and the "dual" (will define in a bit) f*(a)=aA. In terms of matrix multiplication, an m x n A is a linear transformation of n dim vectors to m dim vectors if we multiply from the left. But it is clear that A gives rise to a transformation of m dim covectors to n dim covectors when we multiply by A from the right. Let's call this the dual of A.
But we already have a term called the dual of a linear transformation as I defined in the original comment, given by the functor from the category of vector spaces to dual spaces.
So am I abusing the term dual by using it in two different senses? No, because they are the same! It's really easy to show, almost too easy:
Let v in V, w in W, b in V* and a in W*. We construct the dual of the map represented by A by letting for any v
b(v)=a(Av).
In the matrix notation where vectors and covectors are also matrices, we want a matrix B such that
(aB)(v)=a(Av).
From associativity of matrix multiplication, it's really easy to see that B=A! That is,
(aA)v=a(Av).
We did not have to do anything! The dual of left multiplication is right multiplication. It is just a matter of interpretation of the matrix expressions.
But interpreting column vectors as vectors is arbitrary. We could have called them covectors and the row vectors just vectors. It only changes how we interpret matrices and the direction of matrix expressions. And when we want to flip what they mean, we can devise an operation for this. And let's call this magical operation 'transposition.' It works perfectly because the two directions are perfectly symmetric. And this is because which one we call a vector and which one a covector is also perfectly arbitrary in finite dimensions.
What do you think? Does this give intuitive meaning to the purely algebraic manipulations now?
interesting
I'd never considered that a transpose might be more than a mere "matrix reflection". Thanks for another enlightening video.
You just made the best youtube explanation of machine learning svm without even mentioning svm in the title xd
Who would have umagined I would be binge watching math videos on late weekend nights, they are addictive! Understanding is addiction - a good one though.
Im only 4 minutes in and ive done this kind of math for years and so far this has hit like 3 or 4 of those "ohhh" moments. Subbed.
And actually, pretty annoyed at myself for not having subbed before, ive watched (and remember watching!) a few of your videos before. Thanks for the content
I always wondered why linear algebra cannot be illustrated using real life example with numbers that represent real objects such as how a tradsperson would make measurements and then transpose these measurements for other construction. For instance, a 2×2 matrix as two restaurant selling two different meals, and when it's transpose or its inverse is required, what are those changes in respect to the restaurants and the meals. Hope I have made myself reasonably clear. When too many math jargons are used, it may become lots of " hand wavings" and novice like me will get lost. Thank you.
I’ve always learnt transpose as rotation along the diagonal. Even in high school.
This is amazing! I didn't expect this to help in understanding rigid monoidal categories! AMAZING!!!
It feels so satisfying after understanding the idea of transpose.. Keep up the amazing work.
Also, are you planning on explaining Adjoint in the similar way? It would be a great help!!
I will have to rewatch this video a few times.
the borrowed scale metaphor is just brilliant
This just left me with more confusion. It replaces a simpler meaning with a complex one but does not make it deeper.
Same dude😅. And it seems like his explanation is longer than it needs to be
Brilliant video
Deep insights in Linear Algebra ( Reminds of Essence of Linear Algebra series on 3B1B)
The concept presentation is novel and wonderful. Deep insight. I did not get the opportunity to learn these concepts in my student and research scholar days. A great service to workers in basic sciences. We pay our sincere gratitude to 3b1b.
Great! Please more about matrix operations and the idea behind it! :)
People start doing matrices and totally forget that in the basics. matrices is just a bunch of linear equations But with coefficients only.... and realizing that, [A] and [A]^t is either one function out of a list of functions of some sort, or a list of all linear components that correspond with its basis vector. if you were to read only the row of [A] and [A]^t.
Tristan Needham is a legend!
Very good video! First one I see from you, but you earned my subscription. I’ll have to rewatch a couple of times because I’m a little rusty on the abstract side of Linear Algebra and I’m sure I’m missing some important subtleties. I loved the effort you put into the “big picture” chapter!
If I may, I’d like to make a couple suggestions:
- Master “the pause”, by allowing a few seconds when you deliver punchlines, to allow us to digest what was just said. Your rhythm overall is great, but I felt you just kept talking non-stop at those moments. (I’m sure that’s also a consequence of my rusty L.A.)
- In your intro, please mention the “prerequisites” and the target audience. These concepts are deceptively simple at first glance, but it became clear I was missing something as the video progressed, due to my somewhat forgotten command of precise definitions. Of course, the lack of knowledge is entirely my fault, but I didn’t realize I needed it before starting watching the video (after all, I thought I knew perfectly well what transposes were, lol).
Keep up the good work!
Could you do a video on visualising the transformations performed by (A transpose times A) and (A times A transpose), and perhaps extend that to the general projection matrix?
it's interesting how I'm currently studying 4 topics and this video managed to get into all of them! Badass
Extraordinary explanation
I never understood tensors quite well, but in this video I smelled the tensors cooking up!, still don’t know where though!.
Awesome vedio.....Waiting for your special and general relativity vedios....
that presentation is really good. how could you do that? I mean is there a special program you use? thanks
Really great video! One technical recommendation though: think about adding some background music - it'll make it flow more smoothly :)
this is amazing. please make more math concepts. thankyou!!
I’ve been studying Lin Alg for a little while now but I never quite saw transpose that way.. this was incredibly refreshing.. and for a moment I thought you were going to bring Eigenvalues and Eigenvectors into the fold, but you didn’t (even though we both know they were hiding there in plain sight now, didn’t they? 😉)
It's also interesting to consider cases, when matrices linearly map vectors to bivectors and back. Such as when dealing with transformation of a force into a torque (in 2d, or in 4d, since amount of bivector components are different there from amount of vector components).
I've seen transpose of such matrix being used to figure out effective mass from the inertia tensor and position to which force is applied (1/effectiveMass = transpose(toTorque) × inertia × toTorque), where inertia is just a symmetric matrix (aka orthogonal resizer) that turns torque bivector into a resized angular velocity bivector. ToTorque represents geometric algebra wedge of a force-vector by offset-vector. And inverse effective mass just shows how much of tangential velocity unit force imparts onto the object (ignoring linear velocity).
Anyway, sorry for the unclear monologue; I'm just too tired after a long day and I wanted to share a little piece of cool info I recently learned.
Takes some effort to understand but really insightful. Thanks 😊:)
exactly what I needed
Completely lost sorry, no idea what you're on about. Sometimes you'd say the most simple thing, like explain matrix vector multiplication or a linear combination but I've absolutely no idea what this "measuring device" was supposed to be and you're coming back to it like it's established or an actual mathematical object.
With the transpose you're going from column space to row space obviously. Maybe that's what you meant. And obviously the relationship between the vectors is preserved, it's just a reflection.
I suspect you've just over-thought this and fallen down a rabbit hole of waffle. Who knows, maybe it all makes sense and it's me but I'm stopping here.
I agree. This video is overly complicated than what it is supposed to be. Measuring device is also very bad analogy since he is introducing extra input parameter into the system which did not exist when he was defining its concept.
i don't really like the neighbor's measuring device analogy. i feel like there should have been something more intuitive.
Yeah, that made no effing sense at all
All these concepts would be easier to learn and memorize on college if professors have invested time and energy into visualizing things for students the way you did here.
The thing is professors have *many* other things to do! Even if they only teach, it is impossible to do this sort of visualisation every day!
Well Profs don't need to do those everyday. If a view like you, 3b1b, math sorcerer etc. cover most stuff you can just link the videos and get all the intuition you need.
^^ much better than the good old sentence: "that's easy too see" after you copied half a dozen blackboards in 90min 5x a day.
I studied while videos like this were emerging and they are awesome to gain intuition and boost confidence in all the gaps a lecture leaves. Just recap the video and you are usually good to go to keep solving problems.
That's the job of the students to figure out.
It's university, not school.
I loved watching this video but I need something that will lead into it for me as I never studied this topic before. I studied Physics 50 years ago and am currently taking a QM class so need to understand matrices more deeply particularly Hermitian operators and what adjoint, transpose etc reaally mean. This seemed clear as I watched it but I haven't really grasped the meaning.
eigenchis has a great series on this sorta stuff
Off-diagonal matrix coefficients can be also kinda thought of as bivector components btw
Do you have a brief explanation as to why there are more covectors than vectors in infinite dimensional spaces?
If you consider the space of square integrable functions (countably infinite, Hilbert space, for example, vectors of Fourier coefficients), the dual vectors include non-funtion distributions like the Dirac delta function. This is a dense, uncountable basis (one for each real number).
I believe this is the corollary of Hahn-Banach Theorem. Corollary is as follows;
For a given m > 0 and a vector x_0 in the normed space, there exists a functional(covector) such that ||f|| = m and f(x_0) = m ||x_0|.
Just wonderful! Thanks
1/gap = dy/dx is the slope
Can you do more videos on Matrices and Vectors? This video was awesome!
Hello,
I am a bit confused with your introduction, it looks like you are looking for a substitute to composition, which you do not mention at all.
However i really like the column vector representation for linear forms as their matrix expressions involve the transpose of these columns for a dot product:
let µ be our measurement, of vector X, t is the transposition,
then µ(Y) = t(X) . Y
and µ(alpha(Y)) = t(X) . A . Y = t(t(A) . X) . Y
therefore t(A) . X is the vector of our "same measurement" in the starting space and the rank of A is not a limitation.
Thanks for a video with great visuals. To add to it, I strongly recommend the Tensor Calculus series from the YT channel MathTheBeautiful by Prof Pavel Grinfeld. I've been studying these topics for many years and, to me, there is currently a plethora of notations and names that shuffle conventions from vector calculus, linear algebra, etc. - often making the topic very confusing. He employs a clear and unifying notation that it makes it VERY EASY to understand covectors, metrics, inverses, transposes... I cannot recommend it enough.
glad i could understand this stuff. 3b1b really helps! amazing stuff!
12:18 why do the basis vectors go back to their starting points? That's should only happen if you multiply by A^-1 right? Not A^T
The point of the transpose is to move the measuring device back to the original space, so that you can do the measurements in the original space. So you ask, what is the operation so that the measurements remain the same, but the basis vectors have not changed? That would be the inverse.
As I said later on in the video, it only looks like inverse, but you have to treat those gridlines as covectors.
"t" is not supposed to go BEFORE the matrix we want to transpose ?
You know what?
8ᵀ = ∞
If 8 is a matrix of pixels, instead of literal number
I have enjoyed it and learn new things today. Thank you ❤️
i have difficulty conceptually separating vectors and covectors when, in nearly all vector algebra i've used, there's been zero meaningful distinction between them in practice
for one: with matrix notation, whether covectors are the "row vectors" or the "column vectors" is entirely arbitrary, and likewise with "bras" and "kets" in bra-ket notation - covectors with no corresponding vector didn't seem to come up in QM, for me, but maybe i missed them - but since the algebra doesn't seem to suggest an inherent distinction on its own, it feels like weird human artifice to impose it
for another: when dealing with special relativity, you can bake the metric directly into your basis elements if you don't restrict yourself to standard linear algebra, removing the metric tensor from consideration entirely, and allowing every vector and its corresponding "covector" to be fundamentally equal - this was extra useful to me, because many applications of the metric tensor are very dependent on situational "covariance" and "contravariance", and these concepts were _never_ explained in a useful manner or clearly demonstrated within the algebra itself, so it just looked highly arbitrary at worst and fudged by-eye at best
baking the metric directly into the basis also allowed me to discard the cross-product in favour of an operation that makes sense and works in more than 3 dimensions, but that's a different matter
(also, i have a somewhat better grasp of covariance and contravariance now, because using explicit bases instead of implicit ones seems almost mandatory in genuinely demonstrating the covariant and contravariant parts of a "basis transformation", which it turns out, when done correctly, is less a transformation than it is a fun notational trick that transforms nothing except the way things are written down)
expanding slightly on the second point, about the metric tensor, now that i remember it slightly better: when taking a vector-gradient of the electromagnetic 4-potential, versus taking the divergence of the resulting electromagnetic tensor, the metric tensor would switch between "covariant" and "contravariant" forms (by inverting the tensor in some capacity, i think), and most everywhere treated this like it should be absolutely obvious when and why it is doing that - not to mention how unintuitive the missing minus-signs were for one of the forms
when i construct an equivalent gradient operator using geometric algebra, with the metric baked into the basis elements (their "quadratic form"), i need only understand that the basis vectors in each term are actually _multiplicatively inverse*_ (not obvious in most explanations of the gradient): we don't have x̂ d/dx, but rather 1/x̂ d/dx, for example, and the necessary signs emerge naturally from the metric thereafter
and neither do i need to do anything to that operator before also applying it to the resulting electromagnetic bivector (for its divergence): the elements handle the metric _inherently_
it's much cleaner, and its easier for me to understand the math because it feels like the math better understands itself
*there're some details to this i have yet to work out, because i've been very depressed and possibly the only person on the planet who cares; if i am wrong in some capacity here, the easiest way for me to understand it might be to understand how (or whether!) the gradient operator can meaningfully exist for elementary null-vectors (that is, null-vectors that aren't just a sum of elements with opposing non-null metrics) - given a null-metric shouldn't have an invertible metric tensor either, i've been assuming it cannot
Very nice video about not so easy to grasp concepts:
The content of this video and more on coordinate transform, dual spaces, transpose maps, kinematics with a rigorous full-concept-notation (rather than proof complete)
for mechanics and robotics can be found in appendix D in
DOI: 10.3929/ethz-a-010662262
Very good explanation, but at 1:20 you say that the measuring device takes a vector and takes it into a number. Then in subsequent slides, the device is treated as something that takes a vector to a convector. Any chance of explaining that?
Wow. I understood this topic before, but the ideas never really felt intuitive. This video is amazing.
perfect timing
Beautiful explanation 👌 I would love that 10years ago 😆 still fun to watch
for me the difficulty has always been with "covector" aka "dual" aka "1-form", and I think its because I wasn't exposed to them until college, while vectors, dot-products, and cross-products where presented in high school, where "dot product" doesn't differentiate at all between vectors and covectors. And then once exposed the question becomes why choose any specific mapping between vectors and covectors, once you decide you need to do this you always end up with the simplistic choice of the identity mapping, and then wonder what all the fuss was about, abstraction for the sake of abstraction without any benefit is what it seems to be.
Great video, well done
The exact thing I've been looking for, the last 4 years through 😢
I love your videos, generally. However, the density of the use of the words 'this,' 'that,' 'those,' 'here,' 'it,' and other ambiguous references makes your examples more taxing. But I understand the idea-a beautiful concept.
You say in infinite dimensional vector spaces we have many more vectors than co-vectors. Is this true in general? I was under the impression that it is essentially arbitrary which space we consider the dual.
And in infinite dimensional (complex) vector spaces we encounter another quirk. We find operators that have no Eigenvectors in either the space or the dual. They either have left or right eigenvectors. An example from quantum mechanics are the lowering operator, which only has right eigenvectors, and its adjoint, the raising operator, which only has left eigenvectors.
This is a concept that I am having trouble understanding, as the characteristic polynomial should just give us the Eigenvalues and not care about left or right?
Many more *covectors* than vectors (the other way around). See the description for the explanation. There are more covectors than vectors precisely when the vector space is infinite-dimensional.
How do you have the characteristic polynomial in infinite dimensions?
@@mathemaniac Again in QM we have the state space with elements |x> and the dual space with the
@@narfwhals7843 That's not all the duals in the infinite dimensional case. If we go back to the wavefunctions in 1 dimension (to make things a bit more concrete), then the function psi --> psi (0) is a perfectly good dual [it is linear, and it sends a wavefunction to a number], but you can't write it in terms of
The ladder operators are defective matrices in finite dimensional spaces (thus missing eigenvectors), but not in infinite spaces because it takes infinite operations to kill arbitrarily excited states, one level at a time. See the comment above on Hilbert spaces about th different sizes of spaces.
Great video. Thank you
Yes, all of this comes directly from the transpose, so what do you mean?
Thanks, King :)
Bro my professor in 4th year of Engineering told us that even adjoint and adjugate of a matrix aren't the same thing. I mean he told the difference but i didn't understand. Make a video on that also
so convinced in first two minutes.....and also figured out what the key feature of the video is going to be like. Thank you so much!
Can you please make a video on the proof of Laplace Expansion formula
...you also need to rotate the characters 90 degrees, so they stand up straight!
16:15 hmmm I think this is why robot kinematics uses Jacobean transpose as an approximation to inverse
18:57 is more to the point I think. In robotics you are generally dealing with rotations, in which case the transpose really _is_ the inverse.
THANK YOU
Great video!
Can you share video code?
Goodness what a ridiculously complicated explanation! Am I being thick?
Is there a visual way of understanding the matrix of cofactors as well?
It decomposes the determinant into a contraction, giving A B = |A| I. The determinant can be visualized as the matrix acting on p-forms (cross or triple products). See Flanders for a beautiful introduction.
what has the length of a covector got to do with density?
_Aren't the best transposes when they bring shivers to the diagonals of your matrix?_