Crust of Rust: Declarative Macros
Вставка
- Опубліковано 28 лип 2024
- In this second Crust of Rust video, we cover declarative macros, macro_rules!, by re-implementing the vec! macro from the standard library. As part of that, we cover not only how to write these, but some of the gotchas and tricks you'll run into, and some common use-cases.
You can find the final code at gist.github.com/jonhoo/ec5788..., and the Little Book of Rust Macros at danielkeep.github.io/tlborm/b....
0:00:00 Introduction
0:01:35 The vec macro
0:04:08 The Little Book of Rust Macros
0:05:17 Macro syntax and hygiene
0:16:42 The empty vector
0:19:26 Non-empty vectors
0:25:50 Macros v2
0:26:34 Macro delimiters
0:27:54 Declarative vs procedural macros
0:30:15 Repeated macro arguments
0:39:49 Trailing commas
0:44:10 Why are macros useful?
0:47:29 Vector by repetition
0:51:02 Macro rules readability
0:52:00 Invalid macro inputs
0:54:52 Test that something doesn't compile
0:56:50 Tidying up the patterns
0:59:05 Reallocations for repetition constructor
1:04:08 Macro argument trait bounds
1:06:40 "use" hygiene in macros
1:08:28 The standard library pattern
1:10:20 The need for counting
1:11:38 Eager macro errors
1:13:00 Counting in macros
1:24:48 Other ways of counting
1:27:27 Ensuring count is computed at compile time
1:28:32 Hiding internal macro patterns
1:31:13 Other collection literals
1:33:00 Comparing against the std implementation
You can watch the live version with comments at • Crust of Rust: Declara... - Наука та технологія
For anyone watching this more recently: there's a nightly feature called macro_metavar_expr that gives you special expressions to count the number of repetitions of a metavariable, among other things. So you won't have to write that @COUNT helper macro thing.
when you show cargo expand, it's such an "Ooooh" moment inducing thing. Really useful! Would love to see the series continue
Can you do an episode of Crust of Rust with the topic of error handling. You could explain how to manage different error/result types from multiple crates and how to bundle them up. If I'm not mistaken there are some crates like Failure or Fehler which help with error handling.
Especially things like:
- Dealing with a peekable iterator over a Result whose Err() does not implement Clone - so you can't use ? because you have &Result but can't move the Error either.
- Dealing with Results whose Err() does not implement Try - so you cannot use ? (is the only option to match and convert the error?)
- Dealing with Results whose Err() does not implement std::Error::Error
- Working with all of the above with anyhow and thiserror
@@BertPdeboy Can confirm. If you look at my first several rust projects, none of them handle errors the same way. I think I'm in a better place now, but uh.. how does one know without peer review?
Want the same, especially after 'Failure' deprecation: github.com/rust-lang-nursery/failure/pull/347
Actually would be super helpful, recently got into a corner when I was trying to handle errors from two different types of objects ? main function or that forced me into using .unwrap() for both cases. This is obviously not ideal, because segfaults are not fun for the CPU and will crash the program. The documentation on this, is actually not that helpful. doc.rust-lang.org/stable/rust-by-example/error/option_unwrap.html as really none of that means anything to me.
Johnny Knight I’m actually doing my best to avoid using dependencies. I’d like my code to be portable enough that I can use it on a embedded device.
I can't believe how good these videos are
1:10:17 - it has to be in that order because it needs to hit the non-recursive case first. Technically [ ] satisfies both conditions so you want it to satisfy the the one that does work rather than the one that recurses back to itself.
Thank you, Jon, for a very useful stream.
Suggestion for another Crust of Rust: how to use all these Rc, Cell, RefCell, Weak, etc. After reading all the documentation I've managed to compile my code, but it ended up panicking all the time. So a good explanation about shared ownership could be useful for many, I think.
Awesome video. And can I just say that responding to comments on chat is very helpful. I did not watch the live stream but your viewers asked all of the questions I thought of during the session.
What I learned from this video is that rust really needs to provide a way to get the length of the macro arguments
I missed the stream but thanks for this! Please keep doing the series as long as you feel up to it; it's been helpful so far.
I just love how your non-Windows machine is called "defenestration". This is so deep
These videos are so freaking helpful, thank you Jon.
Amazing content man, I always learn so much from it. Thank you
Thanks for the awesome content. Such a pleasure to follow along.
I never liked vim till watching your video series. Awesome and well done. :)
Very good video. Thank you. If you have the time, please do more.
And the "awesomeness" continues !! Thank you so much for making these videos ! In addition to the super-human intelligence (MIT and that too PDOS !), you have an exceptional ability to demonstrate things by building upon concepts in a very approachable way. Thank you!
That's awsome and helpful! Just hope Rust will later provide handy way to count
best rust content hands down
Wow, just want to learn that, thanks!
The Little Book of Macros is a gem. I've been trying to do this `bitflags! {` for a while now!
Your stuff is amazing :D
1:32:10 I like the "left as an exercise for the reader" :)
Macros in lisp are way easier to implement, because the language is represented as a recursive list.
However the parentheses get old really fast, and I don't usually write enough macros to make up for it. I prefer the rust way of creating macros, because the language can be way more expressive than lisp in general.
your video is great sir
Excellent again! Thanks!
Looking forward to the next one...
Hope you aren't jeopardising your PhD doing these?
Very cool video. Make me know the macro of Rust a lot more.
Relatively new Rustacean here (only a few crates to my name). Love this channel.
Relatively new Computer Scientist here ( only around 43 papers, first in author to my name). Love this channel.
The demonstration with const was really useful.
By the way, what are your vim settings? (and terminal emulator if you are running it directly in the terminal?) - I have a few problems with rust-analyzer and auto-completion plus showing the types in gvim.
He has a separate video on his setup called "Desktop and editor setup for Rust development". His vim configs are available on his github. I think he is using Neovim.
@@lauh8406 thanks, I'll check that out!
Great video. :) Sorry for being incredibly picky again ;-), but:
At 1:25:38 you misread the docs (hence the confusion about it not being const) - you **can** use it in Rust 1.2 - it just warns you that in that version it is not a const expr (although still might be evaluated by the compiler in an optimized build), in much more modern Rust, as you showed, it is const.
It seems to me the book is quite out-dated since last commits to it were 4 years ago.
And it warns you about it, as the trick expanding to `0 + 1 + 1 + 1 + 1…`, although much heavier for the compiler, did produce a constant expr already in 1.2 (as it was just simple arithmetic on constant integers). So in Rust 1.2, if you absolutely needed const expr to be produced, then the `+ 1` trick might have been preferred.
Today it seems it doesn’t make sense to use it anymore, and the slice length of an array of units is always the ‘correct’ way.
All good - always good to have things clarified! I think that's the same thing we concluded at the end of the stream, but may be misremembering.
thank you for the nice video - just a remark where you might want to re-evaluate the expression multiple times - if the expression produces different values every time evaluated. An example for this would be a random number generator, or measuring something from a device. Arguably this could also be done by providing an iterator (I guess - but I'm only a C++ guy), but at least traditionally random number generators are simple functions that produce different numbers when evaluated multiple times.
Ah, but even then it's not clear that you want the expression to be evaluated multiple times. Maybe you're using the random number as an index in multiple correlated data structures, and the _same_ random number therefore has to be used for each. But you're right that there _are_ cases where you want to execute something multiple times too. Generally, there isn't a good way to indicate whether your macro evaluates multiple times or just one currently in macro syntax, except by documenting it well.
@@jonhoo sure, the default should be not to evaluate multiple times - and only create special versions for exactly the use-cases (with good names so that nobody calls them by accident)
Its very minor, but I really feel like you should use absolute line numbers while streaming.
I used to have relative line numbers, but I realised I hardly ever looked at them, they only made it harder to communicate with others when screen sharing.
macros do rule indeed!
Proc macros were easy for me (albeit it can take a lot of code). It's declarative macros I am trying to figure out
how does he do the thingy in vim where he highlights the code and then pastes it below without moving the curser, he really needs the thing that shows his keyboard keys
Could anyone explain what happens at 1:20:49? Why do we need to explicitly mention the type? Are there different implementations for len for each type? And if that is the case why? I thought len just returns the number of elements in an array/slice/Vec?
I know you did a viedo about proc macros but can you do another one with this explaining methodology?
I don't think that procedural macros are suited for a video this short sadly. There are far too many things to cover :)
1:04:20 just like c++ templates - generate the code and pray it implements the methods you invoke
Have you tried playing the Gloomhaven video game?
Can you do another Crust of Rust about proc_macros? :)
Procedural macros are a much thornier beast, and probably wouldn't fit into this format. I've done a longer stream on procedural macros though, so maybe you want to give that a watch? ua-cam.com/video/geovSK3wMB8/v-deo.html
You should put a link to your twitter in your description
00:50:10 - so I guess rust macros are better than their c/c++ counterparts, but they still hold some edge cases we need to be aware of.
Can someone help me?
Does std::iter::repeat().take() implement TrustedLen which provides a size for "extend" to get the right mem allocation ahead of time?
Im new to Rust and I read the source code to build this intuition, so a confirmation could be of great help!
I believe it does. From memory Take implements it if the Iterator it wraps implements it, which I believe it does. If I remember right, the memory allocation will take advantage of the Iterator size hint regardless, which take definitely gives.
anyone that is familiar with C macros will perfectly understand the @SUBST at around 1:19:00
nice
I would prefer rarer interruptions for questions to the main line of the video.
hey, is it possible to have a macro that only receive item that implemented some trait? to be more specific im trying to implement my own HashMap and a macro that receives an RandomState in order to use it in HashMap::with_hasher. thanks in advance!
No, you can't add trait bounds to macros. However, most of the time there isn't really a need to. If someone uses a type that doesn't work in the generated code, the compiler will simply give them an error in the generated code instead :)
@@jonhoo thank you! I thought of that too after seeing it in you video :)
But now I have a problem because I have 2 macros that receive one element (one for count one for RandomeState) and which ever I declare first the compiler indicates an err on the other because they both could fit the pattern.
Use dark reader for firefox :D
As soon as he showed the counting with empty I was like... wait, we can just substitute with whatever this way? So like... with a 1? like I could turn each element into an expanding (1+) pattern and terminate with a 0?...Paused and tested. Quite pleased. Assuming this is what we do next... nope... I guess it makes sense that there is a depth limit for ast. Wouldn't have thought a mere 500 levels would be it though. Guess there isn't really a good reason to have more than 512 levels deep ast with normal code. Could add a pipeline in the tokenizer stream to convert literal a + literal b pattern and to a simplified literal sum, which would be streamable. But then you loose the mapping capabilities an ast provides for those tokens. Makes sense that they can't do that in the standard toolchain, ast is the sacred source all IDE's, formatters, linters, checkers, and transformers depend on.
macro_rules! lifetime_rules_too!
Compared to the rest of the language, declarative macros feel kinda awkward, especially the counting thing.
I'm new to Rust ,how to do something like this in rust e.g. if I have a byte/char buffer :
char ptr[] = { 0xAA, 0xAA,0xBB, 0xBB, 0xDD, 0xDD, 0xEE, 0xEE };
int a = *(int*)&ptr;
could someone write this C (casting) code in Rust? so that a == 0xbbbbaaaa? thanks
That style of type punning is illegal even in C (strict aliasing rule) and you should memcpy from char array to int.
In Rust you have several methods associated with integer types (from_ne_bytes/from_le_bytes/from_be_bytes) that let you do that e.g.
let buf: [u8; 8] = [0xaa, 0xaa, 0xbb, 0xbb, 0xdd, 0xdd, 0xee, 0xee];
let value = i32::from_ne_bytes([buf[0], buf[1], buf[2], buf[3]]);
👍👍
48:38
could not concentrate over the fact that he has his browser nav on the bottom of the screen. I would not let this guy near my children
That was an ugly hack but informative session
I feel like the macros state is really bad, docs, macros creation, pseudo regex are all bad... I think rust should work on that
Not sure this is performant, however you could just have an expression that evaluates to 1 and add it for the counting?
```
macro_rules! avec {
($($el: expr),*) => {{
let len = 0usize $(+ {avec!(@SUBST; $el); 1})*;
#[allow(unused_mut)]
let mut v = Vec::with_capacity(len);
$(v.push($el);)*
v
}};
(@SUBST; $_el: expr) => { () };
}
```
So this would reduce to `0 + {(); 1 } + {(); 1 } + {(); 1 } + {(); 1 }`, or `0 + 1 + 1 + 1 + 1 + 1 + 1`. What do you think?
Oh someone else already thought of that.. 😅
Two years late but for whoever was curious about the inspiration for macro by example, here you go... legacy.cs.indiana.edu/ftp/techreports/TR206.pdf