Thanks a lot! I feel that for this particular topic there was no clear explanation for us who didn't understand the design decisions beforehand. Ok, just to check that I got it all right, I understand that we can have: - Send + Sync: applies to "most" types, no restrictions. - !Send + Sync: cross-thread transfer of ownership is not fine, but transfer of a &-reference is fine. Other threads can access the value but are not allowed to drop it, nor &mut-modify it. Prototypical example: lock guards (like MutexGuard), "per-thread" allocation via thread_local/TLS (?). - !Send + !Sync: neither owned nor &-reference transfer are sound. This is because there are methods accessible via &-references that break some invariant if called from a different thread. Prototypical example: Rc, it is !Send because of its non-atomic reference counter (can cause a double-free (UB!) or leak (not so serious)), !Sync because clone() also manipulates said counter. Arc is Send + Sync by using atomics (performance tradeoff). Raw pointers are purposefully !Send + !Sync to "contaminate" any enclosing types, just as an extra security measure. - Send + !Sync: weirdest case, this applies when we want to have all references in the same thread as the owning one, but as the same time we want to allow cross-thread ownership transfer. This is the case of interior mutability wrapper types. Prototypical example: {,Ref,Unsafe}Cell.
31:50 It's not solely about multithreading that makes mutation from behind a shared reference dangerous. Imagine the data has a Vec, and someone has a direct reference into the Vec (gained by Vec::get()), and you push a new item and it re-allocates the Vec, making that reference point into deallocated memory. I guess that's why Cell doesn't implement Borrow / doesn't allow you to get a shared reference, only to replace the value entirely?
11:10 T does not have to be Clone in Rc 11:45 If &mut is omitted, the code would still work, as dereferencing a mutable raw pointer (self.inner: *mut Inner) gives mutable access to Inner 13:10 &unsafe { &*self.inner }.value: &* dereferences the raw pointer and casts the Inner to a shared reference, & casts the value to a shared reference 25:30 MutexGuard is Sync + !Send, Rc is !Send (clone Rc and send to another thread, reference count is not atomic) + !Sync (send &Rc to another thread and call clone, requires all access happens on one thread) 28:40 Cell is Send + !Sync, can't get reference to Cell in another thread, therefore safe to mutate in current thread as no other reference is mutating it. T must also implement Send + !Sync. 31:00 Application of Cell in graph traversal (can't take exclusive references, could walk same node), Cell allows mutation through a shared reference 36:20 If &mut T is Send, then T must be Send (std::mem::replace) 45:00 T is Sync because all the Arc instances reference T, T is Send because the last Arc must drop the inner type 46:00 &T is Send if T is Sync 47:30? Sender is !Sync, multiple shared references to Sender but only one in each thread 54:30 dyn syntax allows only one trait, exceptions are auto traits (Send, Sync) 59:50 thread::spawn requires type is 'static & and Send, not Sync as it doesn't take references 1:00:40 thread::scope does not need 'static & arguments, current thread can't return until scoped thread joined
You mention that !Send is sometimes manually (un)implemented for types that rely on thread-locals. Does that mean that auto implementation of Send may cause unsoundness/UB? Or is it "solved" by thread-locals being inherently unsafe to use?
There shouldn't be any unsoundness in safe code. Manually unimplementing Send or Sync is useful because then your unsafe code can rely on it and actually be safe, whereas with the automatic implementations that unsafe code you wrote would actually be unsafe. Basically, it gives you an additional invariant you can rely on in unsafe code.
Isn't your RC implementation UB? Is it fine to dereference a *mut pointer inside a struct through &self? You said it's technically safe because there is no multiple write access to that field and this is correct without Send and Sync but isn't going from & to &mut always UB? Even for struct fields?
Audio and video seem to be a little out of sync. I watched it with ~ -250ms for audio in VLC. The live version was fine thou when I watched it briefly, but I might be wrong. Aside from that awesome video as always.
Oh, interesting. I actually found that the video file OBS produced this time around itself was out of sync between audio and video, and fixed that before uploading. Could be that I fixed it too far the other way, or not far enough somehow. Looking at the video again, it looks at least pretty close to me, which is more than I can say about the original video file :p
23:15 worse yet, two threads concurrently increment, but both increment it to the same value - then, a few drops later and somebody has a pointer - as far as the compiler is concerned - into the depths of hell.
I never understood why "duplicating" the Rc is implemented with "Clone". When I clone something, I expect a deep copy, and Rc breaks that. In other words, if I clone it, then mutate the clone, the original is mutated as well! What a mess, honestly.
Clone never made any promise regarding "deep" copies. It just guaranteed that you receive a value of the same type behind the reference. The semantic of the trait expects implementations to return a value that is logically the same, but nothing besides that. If you clone the sender side of a channel you now have two senders to the same channel, they are logically the same. Any reference is Clone, since they are also Copy. If you clone one you get two references, two different variables, but pointing to the same thing. If you clone a Vec you are cloning something you own, so you get two things that you own. You can't own the same thing twice, so Vec clones the items. You are mixing the semantics of ownership with the Clone trait. Clone give the same type behind a reference, that type can be anything, owned or not, even another reference. If the thing behind the reference/(smart)pointer is clone you can still clone it directly by doing (*foo).clone()
@@FryuniGamer Yes, "clone" doesn't mean "deep clone" in Rust, but it does in other in some other languages. Just another thing to keep in mind, the list just goes on.
I looked at the channel yesterday and was sad to see no new videos for 3 months. Just now I wake up to a new video notification. Today is a good day!
It was the same for me but first I found out about a new video through Reddit which doubled the effect of a happy surprise
Thanks a lot! I feel that for this particular topic there was no clear explanation for us who didn't understand the design decisions beforehand.
Ok, just to check that I got it all right, I understand that we can have:
- Send + Sync: applies to "most" types, no restrictions.
- !Send + Sync: cross-thread transfer of ownership is not fine, but transfer of a &-reference is fine.
Other threads can access the value but are not allowed to drop it, nor &mut-modify it.
Prototypical example: lock guards (like MutexGuard), "per-thread" allocation via thread_local/TLS (?).
- !Send + !Sync: neither owned nor &-reference transfer are sound. This is because there are methods accessible via &-references that break some invariant if called from a different thread.
Prototypical example: Rc, it is !Send because of its non-atomic reference counter (can cause a double-free (UB!) or leak (not so serious)), !Sync because clone() also manipulates said counter. Arc is Send + Sync by using atomics (performance tradeoff).
Raw pointers are purposefully !Send + !Sync to "contaminate" any enclosing types, just as an extra security measure.
- Send + !Sync: weirdest case, this applies when we want to have all references in the same thread as the owning one, but as the same time we want to allow cross-thread ownership transfer. This is the case of interior mutability wrapper types.
Prototypical example: {,Ref,Unsafe}Cell.
Yes, that's a good summary!
Thank you for your time and effort. Very much appreciated!
please keep doing crust of rust!! thank you!
Really enjoyed the video. Feel like I understand send and sync better now
You are my hero Jon! Rust for life.
Great content, many thanks for all time and effort
31:50 It's not solely about multithreading that makes mutation from behind a shared reference dangerous. Imagine the data has a Vec, and someone has a direct reference into the Vec (gained by Vec::get()), and you push a new item and it re-allocates the Vec, making that reference point into deallocated memory.
I guess that's why Cell doesn't implement Borrow / doesn't allow you to get a shared reference, only to replace the value entirely?
Lovely explanation, thank you!
11:10 T does not have to be Clone in Rc
11:45 If &mut is omitted, the code would still work, as dereferencing a mutable raw pointer (self.inner: *mut Inner) gives mutable access to Inner
13:10 &unsafe { &*self.inner }.value: &* dereferences the raw pointer and casts the Inner to a shared reference, & casts the value to a shared reference
25:30 MutexGuard is Sync + !Send, Rc is !Send (clone Rc and send to another thread, reference count is not atomic) + !Sync (send &Rc to another thread and call clone, requires all access happens on one thread)
28:40 Cell is Send + !Sync, can't get reference to Cell in another thread, therefore safe to mutate in current thread as no other reference is mutating it. T must also implement Send + !Sync.
31:00 Application of Cell in graph traversal (can't take exclusive references, could walk same node), Cell allows mutation through a shared reference
36:20 If &mut T is Send, then T must be Send (std::mem::replace)
45:00 T is Sync because all the Arc instances reference T, T is Send because the last Arc must drop the inner type
46:00 &T is Send if T is Sync
47:30? Sender is !Sync, multiple shared references to Sender but only one in each thread
54:30 dyn syntax allows only one trait, exceptions are auto traits (Send, Sync)
59:50 thread::spawn requires type is 'static & and Send, not Sync as it doesn't take references
1:00:40 thread::scope does not need 'static & arguments, current thread can't return until scoped thread joined
how can I get notifs for livestreams? these videos are literally the highlight of my UA-cam watch history
It seems impl
Great video. Thank you very much!
Can you make video about std::any
You mention that !Send is sometimes manually (un)implemented for types that rely on thread-locals. Does that mean that auto implementation of Send may cause unsoundness/UB? Or is it "solved" by thread-locals being inherently unsafe to use?
There shouldn't be any unsoundness in safe code. Manually unimplementing Send or Sync is useful because then your unsafe code can rely on it and actually be safe, whereas with the automatic implementations that unsafe code you wrote would actually be unsafe.
Basically, it gives you an additional invariant you can rely on in unsafe code.
Great video, really interesting to listen to you, keep up the good work!
non related question: what is your terminal font? looks really good and readable for coding
Thank you, I want know what's your editor font.
Isn't your RC implementation UB? Is it fine to dereference a *mut pointer inside a struct through &self? You said it's technically safe because there is no multiple write access to that field and this is correct without Send and Sync but isn't going from & to &mut always UB? Even for struct fields?
Yep, I think I mention that too, and how it'd really need UnsafeCell.
Thanks!
Audio and video seem to be a little out of sync. I watched it with ~ -250ms for audio in VLC. The live version was fine thou when I watched it briefly, but I might be wrong.
Aside from that awesome video as always.
Oh, interesting. I actually found that the video file OBS produced this time around itself was out of sync between audio and video, and fixed that before uploading. Could be that I fixed it too far the other way, or not far enough somehow. Looking at the video again, it looks at least pretty close to me, which is more than I can say about the original video file :p
@@jonhoo Perfectly fine the way it currently is. It is very minor. From the past I know that I'm more irritated by it then normal.
impl !Sync for OBS {}
@@jonhoo Try `use clap;` :)
Coming to Spain this summer? 😃
23:15 worse yet, two threads concurrently increment, but both increment it to the same value - then, a few drops later and somebody has a pointer - as far as the compiler is concerned - into the depths of hell.
great vid! could you please share your vim plugins?
59:01: `static`s also require at least Sync
Yessss
Yeeees!!!!
I never understood why "duplicating" the Rc is implemented with "Clone". When I clone something, I expect a deep copy, and Rc breaks that. In other words, if I clone it, then mutate the clone, the original is mutated as well! What a mess, honestly.
Clone never made any promise regarding "deep" copies. It just guaranteed that you receive a value of the same type behind the reference. The semantic of the trait expects implementations to return a value that is logically the same, but nothing besides that.
If you clone the sender side of a channel you now have two senders to the same channel, they are logically the same.
Any reference is Clone, since they are also Copy. If you clone one you get two references, two different variables, but pointing to the same thing.
If you clone a Vec you are cloning something you own, so you get two things that you own. You can't own the same thing twice, so Vec clones the items.
You are mixing the semantics of ownership with the Clone trait. Clone give the same type behind a reference, that type can be anything, owned or not, even another reference.
If the thing behind the reference/(smart)pointer is clone you can still clone it directly by doing (*foo).clone()
@@FryuniGamer Yes, "clone" doesn't mean "deep clone" in Rust, but it does in other in some other languages. Just another thing to keep in mind, the list just goes on.
Hey did my comment get deleted? I asked an interesting question