Prime, iterator cannot be an interface because Go doesn't (and may never) support generic methods in interfaces, so they can't be used to make iterators. (Unless you want to duplicate every kind of iterator for every element type...) There are huge discussions about it, but nobody has come up with a workable idea yet. After looking into it IMO it's not possible.
The iterator type itself could be generic, but none of the implementations could be. For example, you cannot not define a generic Map or Sum iterator, you would have to re-implement it for every combination of types. Rangefunc iterators allow the use of generic iterator combinators.
This goes to show how adding an otherwise simple functional feature to a "simple" (i.e. reductive) programming language makes is far more complicated and messy than it could be.
The reason why they went with this is that it makes it really easy to rewrite common go code where you pass a function that pushes into a channel that you range over. The only change here is you pass a function that pushes into a yield callback instead of straight into a channel. Your example has two levels of func nesting only because it is a generic reverse function as opposed to a less clever lambda or function thats used in a single place and that does not try to do something generic. By contrast, rewriting a function that pushes into a channel to be a struct with a Next() method would be way, way harder and most people would not do it just to avoid channel overhead. The advantage of not pushing into a channel is that no locks or synchronization are required so tight loops become an order of magnitude faster, and defers will run in the right order so cleanup is easy. That's it.
Its not the only reason, there was a big argument around control flow/cleanup. In the method they used, its clear in the construction of the range where the resources are deallocated.
So the effort of rewrite of existing code worth the struggle with all new code yet to be written. Old code is still working, they should have focused more on the future and DX. Funks are already very verbose. Now try to implement a pull iterator, or lazy iterators, or zip over n iterators, it's just a mess.
Agreed. I made a conversion from an iterator interface to a Seq from the spec. My first iteration has quite a few problems. Here's my attempt: go.dev/play/p/uwM46wWKTuH
10:42 you need to think in higher order functions The code is essentially wrapping up the stuff in the for loop and stuffing it into a function to pass into your Map function - the only quirk is map returns a function to allow deferred execution. You could just skip out on range and the deferred function, and just put the for loop body into a function (change break to return true) and call map with the slice and body func
I'm learning Go and I find it very confusing at times. But I only coded in JS, Python, Java and C-hashtag. Sometimes I find Python confusing. I guess I'm just easily confused.
I'm so confused by the people saying that using an Iterator interface is Rust-like or Java-like as if interfaces aren't one of the most core ideas in Go
@@benbowers3613only because it’s not principled like, for example, Odin syntax. ident : type = value Where type can be implied ident := value And a constant is ident : type : value Or ident :: value
Prime should have read the discussion before commenting, his preferred solution is like the obvious idea and there are good reasons it doesnt work like that
My feeling weeks before of the release was that i found it too complicated. But I've turned around. Behind the messy syntax for the Seq type, it is really simple: just call yield to produce a value and return to signal the end. The one footgun is forgetting to check for breaks in yield's result. Besides that, what most will always use is the standard lib and that is just quality of living to be able to range over slice.Reverse() or something like that. On the design decision, golang is not only simple, but efficient for being gc and compiles as fast as a lang can. Parsing a for range Seq is just passing the body of the loop as the yield func and converting continue to return true and breack to return false, making the new syntax trully backwards compatible, not adding any new complexity ( funcs taking funcs returning funcs already existed) and being extremely fast to parse and compile
I honestly feel that it _is_ beautiful though. It's beautiful how they fit these sorts of things into the language without adding new magic keywords. There's something intensely satisfying about that, knowing that things aren't being hidden from you.
I'm surprised that (what I assume are) so many JS guys are puking over this range syntax. Generators are an inversion of control flow within a single thread. Therefore, channels/goroutines are not a solution. Go has had rich support for closures and anonymous functions...JS-style.. since day 1. I'd argue it's as core an idea as structural typing, though the two ideas really are opposite each other. If you need behavior defined later -- a control flow issue -- in Go, you reach for closures and functions. Personally, I've felt the tension between these worlds any time I try to hide ugly control flow details from code that calls my code. If anything, the range syntax chosen is the best Go can do with "as little magic as possible". I imagine the choice was always "don't add iterators" or "add iterators in the way that Go supports and make a lot of coder-tubers throw up in their mouths". Time will tell if the helpers are worth all the vomit everywhere. Honestly, if this is a language change that enables a lot of nice helpers in slices and maps sections of the std lib, and your average dev is forced to deal with a triply func-y iterator syntax, it could be the best compromise between facilitating data structures in the language, making those hooks available to advanced devs (Or those devs willing to become advanced to use it), and dissuading the devs who think it's all magic from casting too arcane an incantation. I also like how, any time we start talking about LISP's metacircular evaluator, or its modern implementation of first-class functions and closures, we end up talking less about science and sequence and more about magic and spells. Hey Prime, want to do something cool for a stream? React to the first lecture of the MIT SICP class!
I'm fine with the syntax, I'm just not sure what the point of the range support is, when it looks like it's barely better than just calling the iterator func with an inline func. The value of iterators as the interface with next is that the pull interface is more useful to drive manually, and therefore can be more easily composed in several cases (though not the ones that actually get used most of the time, like filter and map) It seems like they have an answer for that with that std Pull method, but I don't know how that works, in general you can't use a push as a pull (a blocking channel and a buffer?) Otherwise this pretty much looks fine, I'm assuming the fuss is mostly actually about putting a very functional design in a very procedural language.
@@SimonBuchanNz the advantage is in defining a standard way to iterate over a new data structure so that the iteration doesn't have to be rewritten every time. Particularly useful when creating packages, can expose the iterator without the user having to do anything more than range over it.
@@voskresenie- but it's functionally just callbacks, like JS had on its Arrays for forever: there was nothing stopping you from doing it lazily, and plenty of people did, though with all the issues you have to deal with with a push model like handling back-pressure (they're still around as interfaces like Streams and Observables, since a push iterator is also implicitly async in an event loop world). In other words, sure, adding some standard functionality is fine, my question (if i remember correctly) was why did it need to be blessed with syntax support for iteration? That's going to make things confusing when every other language uses a pull model for loops.
@@SimonBuchanNz They didn't need to, but it's always been rather weird that 'for ... := range ...' was only ever available for arrays/slices and maps. it makes sense to allow other data types to take advantage of that syntax. In order to do that with a pull paradigm, there needs to be state, which must live inside the iterator (in order to conform to existing range syntax), and that's far grosser than using push iterators with this syntax. The way they are implemented, they are easy to use as the consumer, and while they are a bit complicated to write, the logic behind them is very straightforward (you're just writing a loop where the 'yield' function takes the place of the internal part of the loop); it's only the syntax that might cause confusion. Go has never been particularly concerned with what other languages do and what may or may not cause confusion due to that. That's a good thing, imo - how many bad language design decisions are we stuck with in major languages for no reason other than that that's the way it's always been done? That doesn't mean being different is inherently a good thing, but it's also not inherently a bad thing, either. I am far from a go expert and I had 2 versions (inductive, recursive) each of iterators for depth first and breadth first traversal written within 15 minutes after the first time encountering the new feature. It's not that complicated to write, and it's incredibly natural to use in loops, which is how 99% of people will be interacting with the feature.
Tbh I wish they would stop trying to fix the for := range pattern. Once you learn it, it feels great already. They're only going to make it worse from there. It's inherently an annoying thing to deal with in programming, but the current implementation gives you pretty much everything you need to handle it flexibly.
I don't like Go, due to the simplicity. Not because its simple (that is a goal every language should strife), but because the simplicity is in cost of usability. I don't know who said this, but a language should simple as possible, but not simpler. People like it, so its good.
The amount of people in chat thinking that the issue with gopls not supporting the new feature yet is an issue with Neovim and thinking that switching to VSC**e would solve it is too damn high
I think the actual point is the new iter package and the iter extensions for map and slices. The examples is just the under the hood code that you would rarely use.
Everytime I see that I keep feeling like they could've made it so much simpler. But at the same time I know I'm not in their shoes so I have no idea if I would've made a different decision if I had full context. There are things that are there I imagine for technical reasons but that really feel like unnecessary boileplate for a language that's meant to be simple. If we want to avoid specific syntax, we could just use yield in a way much closer to a generator. You'd know when it's exhausted, you'd also wouldn't have code running after the break. func Map(s []int) yield func(int, int)) { return yield func(int, int) { for i := len(s)-1; i >= 0; i-- { yield(i, s[i] - 1) } return } } (that one is fine, but before I spent 5 minutes redoing exactly what they did like that one Prime story 👍) EDIT: a bit late on this but yeah the reserved word thing got me by surprise. I genuinely thought it was
That would be more the role of a linter since you would broke the 1.x promise and split the golang codebase in half. But I understand the appeal of not using external tools
@@suede__thats a work around I use but it gets annoying after a while and isnt as widely adopted as a convention as something like checking for errors is. Also, I’ve had some problems in the past using a library with nil props because the maintainers didnt assure non nils.
@6:37 some dude asked go test it in VS **** and I did. It gives the same error. If I am not mistaken gopls is also managed by Google so I don't know why would you not support your own language? You added a fancy feature to this language, at least support it with the LSP! WTF Google just keeps taking L's. I guess they're too focused on losing the AI race, they forgot they had a language to maintain. I love Go btw.
HostLayout is for struct padding in general, in future Go team wants to reorder struct in a best way to hit cache lines (and reduce the struct size, because now they pad them just following the fields order). However, one might want to keep the order of the struct fields for compatibility. At least this is how I understood this feature.
Every now and then, they invent a new feature-lacking language and call it "easier" or "safer". Then, they start adding features. They do the same discussions other communities had over and over again, long ago. Finally, they add half baked, crappy syntax, defeating the raison d'être of said language. Rinse, repeat. Meanwhile, complex languages continue to get easier.
Amen. That's the reason I didn't take a job doing Go. My interview question was to make a function that takes two arrays and returns their union. Basic question but I realized Go cannot do this natively, meanwhile in C# a.Union(b);
@@natescode that's a feature. too many languages hide inefficient operations behind a single method call. I've interviewed a couple hundred candidates, it's shocking how many think operations like that are O(1).
I don't think anyone behind go called it easier. That's something I've heard a lot of, but it's not meant to be easy, it's meant to be simple, and it is. They've done a great job of keeping it simple while adding new functionality that was missing in earlier versions. Like C, the language is simple, the code is not necessarily simple. However, it's also not really that difficult. Took me about 20 seconds of staring at that iterator code to understand how it worked. Yes, there are a lot of things there, but it's all just a combination of existing syntax - they didn't add anything new apart from the ability to pass that syntax as an arg to 'range'. If you know go, you don't have to google what it means even the first time seeing it. You don't have to remember any magic. If you understand conceptually what is happening, you can write an iterator like that from scratch without reference in a minute tops.
management analogy for the yield function is the concept of “just-in-time” inventory management, instead of holding large stocks of inventory, a company orders and receives goods only as they are needed for production or sales. This minimizes storage costs and reduces waste, similar to how the yield function works ???
I really like how Jai does iterators. for_expansion :: (list: *LinkedList, body: Code, flags: For_Flags) #expand { iter := list; i := 0; while iter != null { `it := iter.data; `it_index := i; #insert body; iter = iter.next; i += 1; } } LinkedList :: struct { data: int; next: *LinkedList; } You just make a macro named for_expansion with those specific arguments, and essentially just insert the for loop body inside it. Now the next time you use that type of value in a for loop it will use your iterator. The first argument is the thing you are iterating. The seccond argument is the body of the for loop. The third argument is any flags passed to the for loop, like if you want a pointer to the value, or if you want to loop backwards. The macro provides two variables. "it", which is the current value, and "it_index" which is the index of the value.
To be honest, Rust is becoming my go to language because of some things which I don't like in go. Rust does feel a bit taunting in start but will keep becoming simpler as you go with it for longer.
yep. the way iters are implemented alongside inline declarations makes it easy to write a recursive algorithm that's usable as an iterator without building out the slice. it's beautiful, tbh. just started prepping for interviews again and writing tree traversal/search like this is superb. (eta: to anyone reading this, if you want to recurse, you need to declare the function as a variable first, then assign it separately afterwards. Otherwise the function def will be evaluated before the var is declared and you'll get compiler errors trying to reference it to recurse.)
I'm struggling to understand when you would want to use this (write your own iterator). Now that I think about it, making an iterator is something I remember doing back in school but ended up never doing in my career as a programmer. I'm sure there's a good reason to do it and I've just never worked on projects where it made sense to do it. Are for loops and range expressions not enough? Don't they iterate over things, like slices? How else would you iterate over things except doing one operation at a time on each piece of data in them? In this example, the iterating is customized because it's going backwards. Okay, that makes sense, but if someone asked me to implement that, I would just make a function that reverses a slice. I'd call that function and then iterate over the result. Would love to hear some more examples.
Seeing the Map example Prime created I got a feeling that the design was motivated by some other usecase. Does the inner, returned function need to be nested like that? Maybe they were thinking of a usecase where it makes sense to define in the wider scope and reference it by multiple generators. That would explain why it is written as clearly pure function. If it is shared it definitely shouldn’t have an internal state. This would make sense if it did something significantly more complex, and where generator was not a linguistic choice of implementation but a well justified feature. The very idea of a map function feels bit against what I know against golang to begin with. For any such simple usage I think the syntax intentionally discourages using the feature. If you have the option of looping the original input to map instead of a generator, you probably should do that instead. Even if the interface and syntax was more friendly, the code should reflect the underlying complexity. Calling a function in a loop is very explicit and easy to understand. When the bulk of the calculation happens in the declaration of the loop structure its kind of hidden. If your yield function can be defined in a lambda, you should just make it a function and call it in a loop. If there is an actual usecase for generating a unique instances of some sequence whenever a thing is accessed, you probably don’t want to inline the definition like that. And the function should probably be pure, aside from caching and such. There this syntax probably looks a lot less clunky as there genuinely would be a need to do more stuff.
I was staunchly against generics in Go because I knew something like iterators would come along. If you give them an inch… Also, how did iterators get through but arena memory allocation didn’t?
Iterators with yield() smacks of maybe laying groundwork for coroutines, i.e., use the range syntax to range over a coroutine generator. Go adds features very slowly, though, so is just a guess that maybe they were forward thinking to another future feature.
That is where Go resembles C - simple structure typing. But Swift enum combined with pattern matching is very possibly the best, nicest feature of that language. It would be excellent to see similar added to Go.
@@dranon0o You don't get it, yes WE can choose what features to use, but everyone else can decide to use those features. And guess what, then WE are the maintainers who need to read the code of OTHERS to understand what's going on and we will see all this new crap all over the code base. Do you actually work in the Programming Area? Most of the time you will read and maintain a code base, so you have to deal with everything the language offers.
I know, unpopular opinion, but the range-over-func is sensibly designed. Some people are rubbed the wrong way because they think the feature shouldn’t have been introduced in the first place, irrespective of its design, as Go is a simple language and should stay so. This is, in my opinion, a fetish. If the language doesn’t provide you with abstractions to absorb complexity from your application code, then the complexity will be in your application code. As long as compilation time and runtime performance stay great, give me them generics and them ranges.
I haven't done too much with Go and I do agree that there are too many "funcs", but here's a nifty thing you can do with these semantics that would be really hard and not as performant with an Iterator interface: type Vec3 struct { x, y, z float32 } func IterVec(v Vec3) (func(func(int, float32) bool) { return func(yield func(i int, v float32) bool) { if !yield(0, v.x) { return } if !yield(1, v.y) { return } if !yield(2, v.z) { return } } }
22:00 Don't use `go mod tidy` to install new packages to your module, use `go get`... `go mod tidy` is similar to `poetry sync` and/or `poetry update` - it's meant to tidy up your deps as the name suggests
Iterator syntax isn't pretty, but 95% of Go devs were never going to write their own iterator anyway. There are more comments saying "so much for Go being a simple language" than there are devs who will ever actually have to deal with this.
Plus this is pretty simple of you spend 10 minutes figuring out what it's doing. This whole controversy is just people seeing unfamiliar code and having a knee jerk reaction. Prime should know better than this
@@adriancruz2822 lazy operations in inherently non-lazy languages look ugly, since you have to bring the state with you. Check how elegantly Haskell handles such things.
I was trying to learn about using Iterators before watching this, I ran into the same confusion around LSP not supporting my range .. I am very glad to see it happened in the video but less sad that someone in chat saved you 10minutes of scratching your head before the - let me fucking run it and see the go error
i use go generics for one thing, a ternary operator function func Ternary[T any] (condition bool, ifval, elseval T) T { if condition { return ifval} else return elseval } i have this in all my go projects
ngl it's sad that you have to write your own basic control structures (as unexpressive functions no less) because the language tries to be "simple" to a fault.
@@Arcwise honestly, it doesn't bother me at all. What I love about the language is that simplicity. You do a lot of clever yet simple stuff with it. Honestly I've learned more while working with Go than any other languages (system programming languages excludes of course)
That doesn't work very well because you have to evaluate _both_ ifval and elseval, whereas with a real ternary operation, only the value in the chosen branch is actually evaluated. I suppose this works for simple things where evaluating both is not costly, but it's something you have to keep in mind if you ever do anything more complex with it.
@@maleldil1 if you're doing something complex then maybe just use a simple if statement. People abuse this operator but can we all agree it's only meant to be used this way 😑
That's exactly what this is. Generators are an implementation of coroutines, "calling back into" the calling function. It just uses closures and first class functions -- because Go has supported that since day 1 -- rather than some of the more magical control flow features of Python (try/catch and yield being keywords in the latter, and concepts in the former).
I don’t understand why you find this syntax awful. It’s behave like a Javascript generator that your consumer can said in your yield to stop execution. You basically just create a function that returns your iterator function. Your iterator function take as argument a function to send your data and you receive if the caller of the iterator left. It’s just that, I don’t see why this generator syntax make you angry.
I love iterators but reading the new syntax for them is enlightening me to stop using them and instead use for loops for simplicity's sake. Hope it builds into something more suitable with time.
You never used yield in any language then? all behave similarly: csharp: public static IEnumerable foo(int number) { for (int i = 0; i < number; i++) yield return i; } js: function* foo(number) { for(let i = 0; i
bro is just a function that return a function that take in input a function array of function over generic function... Just like in derivatives maker, as long as we go on and there is no return, nobody will realize is just a functional factoid
Jesus people, it's literally just a function that returns a higher order function. You all are acting like the did something super complex here. You pass the range a higher order function, and it uses it to pass values to your loop. It's honestly not that confusing if you spend 10 minutes doing it.
Also 90% of people are only going to use this with iterator functions from libraries anyway. It's a nice convenience that barely adds Any complexity to the language.
Hm Feels like they are trying to go for a python style for k,v in Map(foo).items(): print("{} : {}".format(k,v)) But instead of .items(), they use the range keyword
@@ivanjermakov Or LINQ, or Rust iterators, or Kotlin stream extensions, or literally just anything of that sort, they're all the same imo. Except for LINQ's ability to transform into squeal due to compile-time funkiness but that's a separate use-case entirely.
Hm. I don't really see the use case for this, just do the Map (or whatever) beforehand and range over the result of that. The compiler should optimize that away in any case and it's much more readable.
@@squishy-tomato relax man, I know what the term means, it doesn't change the fact it's wrong. E.g. select should be race. Like if you select who will come first...
One of the dumbest things in go is that you can't really easily work with function definitions from other packages which is part of why the yield is so ugly instead of being a named signature.
In Rust terms, think of these Go iterator like something of a try_for_each API. It's not THAT different after all. (Just that their try_for_each has a callback with bool return type instead of Result, and they probably have a completely different API contract story on re-doing and/or continuing iteration on the same iterator.)
One year ago, I refused to believe my coworker who said that there are no iterators and functions on them (map, filter, etc.) in Go's std. When they compare themselves to C they really mean it huh. And the way they integrated this feature into the language is anything but elegant.
Channels and goroutines are for cross-thread communication. Generators are for communication/cooperation **within a single thread**. Using channels to implement generators/coroutines is like using a mallet to swat a gnat. You can do it , but it will make more work for you elsewhere. Sometimes you need a more deft touch.
it's not an "instead of" - a for loop (in modern languages) iterates over something, normally a collection like a map or slice (or whatever your language calls them), but in general could be anything, such as lines typed in from the user, the position of each zip file entry, every prime number... "iterator" is the name of the "something" you need to be to use a for loop on it. The value is being able to tidy away behavior that happens at the *edges* of a loop, making it easier to use (and harder to write!) There's lots of smaller details, but they won't matter much unless you're going to write these.
All arrays are iterable but not all iterables are arrays. With iterable, you can iterate over lazy, infinite, or unfinished data, such as user input, IO streams and generators.
Prime, iterator cannot be an interface because Go doesn't (and may never) support generic methods in interfaces, so they can't be used to make iterators. (Unless you want to duplicate every kind of iterator for every element type...) There are huge discussions about it, but nobody has come up with a workable idea yet. After looking into it IMO it's not possible.
An interface itself can be generic though.
type Iterator[T any] interface {
Next() (T, bool)
Stop()
}
The iterator type itself could be generic, but none of the implementations could be. For example, you cannot not define a generic Map or Sum iterator, you would have to re-implement it for every combination of types.
Rangefunc iterators allow the use of generic iterator combinators.
This goes to show how adding an otherwise simple functional feature to a "simple" (i.e. reductive) programming language makes is far more complicated and messy than it could be.
3:48 It Gets Funkier but every time it gets funkier, it gets funkier
never been a more perfect soundtrack to a funktion.
vulfpeck mentioned
Laughing in Haskell 😂
I’m func playing a func, disguised another func.
The reason why they went with this is that it makes it really easy to rewrite common go code where you pass a function that pushes into a channel that you range over. The only change here is you pass a function that pushes into a yield callback instead of straight into a channel. Your example has two levels of func nesting only because it is a generic reverse function as opposed to a less clever lambda or function thats used in a single place and that does not try to do something generic.
By contrast, rewriting a function that pushes into a channel to be a struct with a Next() method would be way, way harder and most people would not do it just to avoid channel overhead. The advantage of not pushing into a channel is that no locks or synchronization are required so tight loops become an order of magnitude faster, and defers will run in the right order so cleanup is easy. That's it.
Its not the only reason, there was a big argument around control flow/cleanup. In the method they used, its clear in the construction of the range where the resources are deallocated.
So the effort of rewrite of existing code worth the struggle with all new code yet to be written. Old code is still working, they should have focused more on the future and DX. Funks are already very verbose. Now try to implement a pull iterator, or lazy iterators, or zip over n iterators, it's just a mess.
Agreed. I made a conversion from an iterator interface to a Seq from the spec. My first iteration has quite a few problems. Here's my attempt: go.dev/play/p/uwM46wWKTuH
the reasons are there in the github discussion page
10:42 you need to think in higher order functions
The code is essentially wrapping up the stuff in the for loop and stuffing it into a function to pass into your Map function - the only quirk is map returns a function to allow deferred execution.
You could just skip out on range and the deferred function, and just put the for loop body into a function (change break to return true) and call map with the slice and body func
"go syntax is simple and easy to read"
The new range syntax:
This change doesn't add any new syntax btw
The range syntax didn't changed
Drunk go yield walks in "hold my beer"
I'm learning Go and I find it very confusing at times. But I only coded in JS, Python, Java and C-hashtag. Sometimes I find Python confusing. I guess I'm just easily confused.
@@matiasbpg I meant iterator syntax. Brain fart.
I'm so confused by the people saying that using an Iterator interface is Rust-like or Java-like as if interfaces aren't one of the most core ideas in Go
Do you want to see Prime take 21d10 psychic damage? Check it out: 5:48
🤣
>Prime shows Go
>Chat's eye are bleeding from the syntax
It was healing to my self-taught programmer soul
Mostly fine but the implicit assignment operator feels so cursed ( := )
@@benbowers3613only because it’s not principled like, for example, Odin syntax.
ident : type = value
Where type can be implied
ident := value
And a constant is
ident : type : value
Or
ident :: value
@@jack-d2e6i Ooh I like that actually
Prime should have read the discussion before commenting, his preferred solution is like the obvious idea and there are good reasons it doesnt work like that
My feeling weeks before of the release was that i found it too complicated. But I've turned around. Behind the messy syntax for the Seq type, it is really simple: just call yield to produce a value and return to signal the end. The one footgun is forgetting to check for breaks in yield's result. Besides that, what most will always use is the standard lib and that is just quality of living to be able to range over slice.Reverse() or something like that.
On the design decision, golang is not only simple, but efficient for being gc and compiles as fast as a lang can. Parsing a for range Seq is just passing the body of the loop as the yield func and converting continue to return true and breack to return false, making the new syntax trully backwards compatible, not adding any new complexity ( funcs taking funcs returning funcs already existed) and being extremely fast to parse and compile
It's just like if err not nil, golang will never be beautiful, but when your get used to it's qwirks, it is just convenient
I honestly feel that it _is_ beautiful though. It's beautiful how they fit these sorts of things into the language without adding new magic keywords. There's something intensely satisfying about that, knowing that things aren't being hidden from you.
I'm surprised that (what I assume are) so many JS guys are puking over this range syntax.
Generators are an inversion of control flow within a single thread. Therefore, channels/goroutines are not a solution.
Go has had rich support for closures and anonymous functions...JS-style.. since day 1. I'd argue it's as core an idea as structural typing, though the two ideas really are opposite each other.
If you need behavior defined later -- a control flow issue -- in Go, you reach for closures and functions. Personally, I've felt the tension between these worlds any time I try to hide ugly control flow details from code that calls my code.
If anything, the range syntax chosen is the best Go can do with "as little magic as possible". I imagine the choice was always "don't add iterators" or "add iterators in the way that Go supports and make a lot of coder-tubers throw up in their mouths". Time will tell if the helpers are worth all the vomit everywhere. Honestly, if this is a language change that enables a lot of nice helpers in slices and maps sections of the std lib, and your average dev is forced to deal with a triply func-y iterator syntax, it could be the best compromise between facilitating data structures in the language, making those hooks available to advanced devs (Or those devs willing to become advanced to use it), and dissuading the devs who think it's all magic from casting too arcane an incantation.
I also like how, any time we start talking about LISP's metacircular evaluator, or its modern implementation of first-class functions and closures, we end up talking less about science and sequence and more about magic and spells.
Hey Prime, want to do something cool for a stream? React to the first lecture of the MIT SICP class!
I'm fine with the syntax, I'm just not sure what the point of the range support is, when it looks like it's barely better than just calling the iterator func with an inline func.
The value of iterators as the interface with next is that the pull interface is more useful to drive manually, and therefore can be more easily composed in several cases (though not the ones that actually get used most of the time, like filter and map)
It seems like they have an answer for that with that std Pull method, but I don't know how that works, in general you can't use a push as a pull (a blocking channel and a buffer?)
Otherwise this pretty much looks fine, I'm assuming the fuss is mostly actually about putting a very functional design in a very procedural language.
@@SimonBuchanNz the advantage is in defining a standard way to iterate over a new data structure so that the iteration doesn't have to be rewritten every time. Particularly useful when creating packages, can expose the iterator without the user having to do anything more than range over it.
@@voskresenie- but it's functionally just callbacks, like JS had on its Arrays for forever: there was nothing stopping you from doing it lazily, and plenty of people did, though with all the issues you have to deal with with a push model like handling back-pressure (they're still around as interfaces like Streams and Observables, since a push iterator is also implicitly async in an event loop world).
In other words, sure, adding some standard functionality is fine, my question (if i remember correctly) was why did it need to be blessed with syntax support for iteration? That's going to make things confusing when every other language uses a pull model for loops.
@@SimonBuchanNz They didn't need to, but it's always been rather weird that 'for ... := range ...' was only ever available for arrays/slices and maps. it makes sense to allow other data types to take advantage of that syntax. In order to do that with a pull paradigm, there needs to be state, which must live inside the iterator (in order to conform to existing range syntax), and that's far grosser than using push iterators with this syntax. The way they are implemented, they are easy to use as the consumer, and while they are a bit complicated to write, the logic behind them is very straightforward (you're just writing a loop where the 'yield' function takes the place of the internal part of the loop); it's only the syntax that might cause confusion.
Go has never been particularly concerned with what other languages do and what may or may not cause confusion due to that. That's a good thing, imo - how many bad language design decisions are we stuck with in major languages for no reason other than that that's the way it's always been done? That doesn't mean being different is inherently a good thing, but it's also not inherently a bad thing, either. I am far from a go expert and I had 2 versions (inductive, recursive) each of iterators for depth first and breadth first traversal written within 15 minutes after the first time encountering the new feature. It's not that complicated to write, and it's incredibly natural to use in loops, which is how 99% of people will be interacting with the feature.
Map.keys is nice. In 1.24 we get type aliases which means iterator stuff won’t be so ugly. Kinda weird they didn’t just wait to do them together
Seems weird for a statically typed languages to not have type aliases in the first place
Map.keys was about time. One boilerplate function less I have to write.
@@catfan5618Why subject your self to this tho lmao
3:48 I'm a dude playing a dude disguised as another dude
Tbh I wish they would stop trying to fix the for := range pattern. Once you learn it, it feels great already. They're only going to make it worse from there. It's inherently an annoying thing to deal with in programming, but the current implementation gives you pretty much everything you need to handle it flexibly.
3:48 I’m the func playin a func disguised as another func
I don't like Go, due to the simplicity. Not because its simple (that is a goal every language should strife), but because the simplicity is in cost of usability. I don't know who said this, but a language should simple as possible, but not simpler. People like it, so its good.
The amount of people in chat thinking that the issue with gopls not supporting the new feature yet is an issue with Neovim and thinking that switching to VSC**e would solve it is too damn high
"go mod tidy" should be "go mod sync"
That’s exactly what they did for work. go work sync
I think the actual point is the new iter package and the iter extensions for map and slices. The examples is just the under the hood code that you would rarely use.
How did they make go more complex than rust
I’m only 1/3 of the way in, but it would be interesting to see you go through the proposal discussion.
That's more like "Set, Ready, 123 GO"! "
Everytime I see that I keep feeling like they could've made it so much simpler. But at the same time I know I'm not in their shoes so I have no idea if I would've made a different decision if I had full context.
There are things that are there I imagine for technical reasons but that really feel like unnecessary boileplate for a language that's meant to be simple. If we want to avoid specific syntax, we could just use yield in a way much closer to a generator. You'd know when it's exhausted, you'd also wouldn't have code running after the break.
func Map(s []int) yield func(int, int)) {
return yield func(int, int) {
for i := len(s)-1; i >= 0; i-- {
yield(i, s[i] - 1)
}
return
}
}
(that one is fine, but before I spent 5 minutes redoing exactly what they did like that one Prime story 👍)
EDIT: a bit late on this but yeah the reserved word thing got me by surprise. I genuinely thought it was
Still waiting for either a compiler flag or syntax to make struct properties required
That would be more the role of a linter since you would broke the 1.x promise and split the golang codebase in half. But I understand the appeal of not using external tools
100%
Thats why I was thinking a compiler flag or optional syntax to set a field as required
What about using a constructor and returning an interface?
@@suede__thats a work around I use but it gets annoying after a while and isnt as widely adopted as a convention as something like checking for errors is.
Also, I’ve had some problems in the past using a library with nil props because the maintainers didnt assure non nils.
set fields to lowercase and use a builder
It's done like that so the compiler can inline the compiled code. It's how the go compiler gets most of the speed improvements.
7:50 JS has generators like that
@6:37 some dude asked go test it in VS **** and I did. It gives the same error.
If I am not mistaken gopls is also managed by Google so I don't know why would you not support your own language? You added a fancy feature to this language, at least support it with the LSP! WTF
Google just keeps taking L's. I guess they're too focused on losing the AI race, they forgot they had a language to maintain.
I love Go btw.
HostLayout is for struct padding in general, in future Go team wants to reorder struct in a best way to hit cache lines (and reduce the struct size, because now they pad them just following the fields order). However, one might want to keep the order of the struct fields for compatibility. At least this is how I understood this feature.
Every now and then, they invent a new feature-lacking language and call it "easier" or "safer".
Then, they start adding features. They do the same discussions other communities had over and over again, long ago.
Finally, they add half baked, crappy syntax, defeating the raison d'être of said language.
Rinse, repeat.
Meanwhile, complex languages continue to get easier.
Amen.
That's the reason I didn't take a job doing Go.
My interview question was to make a function that takes two arrays and returns their union. Basic question but I realized Go cannot do this natively, meanwhile in C# a.Union(b);
@@natescode that's a feature. too many languages hide inefficient operations behind a single method call. I've interviewed a couple hundred candidates, it's shocking how many think operations like that are O(1).
I don't think anyone behind go called it easier. That's something I've heard a lot of, but it's not meant to be easy, it's meant to be simple, and it is. They've done a great job of keeping it simple while adding new functionality that was missing in earlier versions. Like C, the language is simple, the code is not necessarily simple. However, it's also not really that difficult. Took me about 20 seconds of staring at that iterator code to understand how it worked. Yes, there are a lot of things there, but it's all just a combination of existing syntax - they didn't add anything new apart from the ability to pass that syntax as an arg to 'range'. If you know go, you don't have to google what it means even the first time seeing it. You don't have to remember any magic. If you understand conceptually what is happening, you can write an iterator like that from scratch without reference in a minute tops.
management analogy for the yield function is the concept of “just-in-time” inventory management, instead of holding large stocks of inventory, a company orders and receives goods only as they are needed for production or sales. This minimizes storage costs and reduces waste, similar to how the yield function works ???
Once I realised yield is not a keyword it is actually very easy to understand for me
Ow, we want the func
Give up the func
Ow, we need the func
We gotta have that func
"OF COURSE I KNOW WHO I AM. I'M A FUNCTION THAT TAKES A FUNCTION THAT RETURNS ANOTHER FUNCTION!"
When a lisp developer implements iterator😂
I really like how Jai does iterators.
for_expansion :: (list: *LinkedList, body: Code, flags: For_Flags) #expand {
iter := list;
i := 0;
while iter != null {
`it := iter.data;
`it_index := i;
#insert body;
iter = iter.next;
i += 1;
}
}
LinkedList :: struct {
data: int;
next: *LinkedList;
}
You just make a macro named for_expansion with those specific arguments, and essentially just insert the for loop body inside it. Now the next time you use that type of value in a for loop it will use your iterator.
The first argument is the thing you are iterating.
The seccond argument is the body of the for loop.
The third argument is any flags passed to the for loop, like if you want a pointer to the value, or if you want to loop backwards.
The macro provides two variables. "it", which is the current value, and "it_index" which is the index of the value.
To be honest, Rust is becoming my go to language because of some things which I don't like in go. Rust does feel a bit taunting in start but will keep becoming simpler as you go with it for longer.
lmfao i’d take java over these hieroglyphics any day of the week and twice on sunday
You declare functions inside of functions like variables. Foo := func(args ...any) bool {...}
yep. the way iters are implemented alongside inline declarations makes it easy to write a recursive algorithm that's usable as an iterator without building out the slice. it's beautiful, tbh. just started prepping for interviews again and writing tree traversal/search like this is superb.
(eta: to anyone reading this, if you want to recurse, you need to declare the function as a variable first, then assign it separately afterwards. Otherwise the function def will be evaluated before the var is declared and you'll get compiler errors trying to reference it to recurse.)
I've been learning Go after years of Java/Kotlin/Python, and this range/iterator stuff was rather erm... jarring
The more features go will get the more Messier it will become
I'm struggling to understand when you would want to use this (write your own iterator). Now that I think about it, making an iterator is something I remember doing back in school but ended up never doing in my career as a programmer. I'm sure there's a good reason to do it and I've just never worked on projects where it made sense to do it. Are for loops and range expressions not enough? Don't they iterate over things, like slices? How else would you iterate over things except doing one operation at a time on each piece of data in them? In this example, the iterating is customized because it's going backwards. Okay, that makes sense, but if someone asked me to implement that, I would just make a function that reverses a slice. I'd call that function and then iterate over the result. Would love to hear some more examples.
Seeing the Map example Prime created I got a feeling that the design was motivated by some other usecase. Does the inner, returned function need to be nested like that? Maybe they were thinking of a usecase where it makes sense to define in the wider scope and reference it by multiple generators.
That would explain why it is written as clearly pure function. If it is shared it definitely shouldn’t have an internal state. This would make sense if it did something significantly more complex, and where generator was not a linguistic choice of implementation but a well justified feature.
The very idea of a map function feels bit against what I know against golang to begin with. For any such simple usage I think the syntax intentionally discourages using the feature.
If you have the option of looping the original input to map instead of a generator, you probably should do that instead. Even if the interface and syntax was more friendly, the code should reflect the underlying complexity.
Calling a function in a loop is very explicit and easy to understand. When the bulk of the calculation happens in the declaration of the loop structure its kind of hidden. If your yield function can be defined in a lambda, you should just make it a function and call it in a loop.
If there is an actual usecase for generating a unique instances of some sequence whenever a thing is accessed, you probably don’t want to inline the definition like that. And the function should probably be pure, aside from caching and such. There this syntax probably looks a lot less clunky as there genuinely would be a need to do more stuff.
Go is the GoAT
More like a sheep 🐑
I was staunchly against generics in Go because I knew something like iterators would come along.
If you give them an inch…
Also, how did iterators get through but arena memory allocation didn’t?
Iterators with yield() smacks of maybe laying groundwork for coroutines, i.e., use the range syntax to range over a coroutine generator. Go adds features very slowly, though, so is just a guess that maybe they were forward thinking to another future feature.
Arena is hard to integrate into a GC langauage
Go now looks more complicated... without all the benefits of a "complicated" language
THIS, Go is all about simplicity, but all the new stuff doesn't look simple anymore. The language takes a really false direction.
complicated languages at least have nice syntax to make complexity readable
what are you both talking about
you're not forced to use those features at all
Go is still Go, it just got extra features
That is where Go resembles C - simple structure typing. But Swift enum combined with pattern matching is very possibly the best, nicest feature of that language. It would be excellent to see similar added to Go.
@@dranon0o You don't get it, yes WE can choose what features to use, but everyone else can decide to use those features. And guess what, then WE are the maintainers who need to read the code of OTHERS to understand what's going on and we will see all this new crap all over the code base. Do you actually work in the Programming Area? Most of the time you will read and maintain a code base, so you have to deal with everything the language offers.
Would not have any issues if you just used Common Lisp 😉
You'd then have the issue of parens and Emacs
Well, in LISP you make your problems yourself
Lisp: you create your own spaghetti syntax
I know, unpopular opinion, but the range-over-func is sensibly designed.
Some people are rubbed the wrong way because they think the feature shouldn’t have been introduced in the first place, irrespective of its design, as Go is a simple language and should stay so.
This is, in my opinion, a fetish. If the language doesn’t provide you with abstractions to absorb complexity from your application code, then the complexity will be in your application code.
As long as compilation time and runtime performance stay great, give me them generics and them ranges.
I haven't done too much with Go and I do agree that there are too many "funcs", but here's a nifty thing you can do with these semantics that would be really hard and not as performant with an Iterator interface:
type Vec3 struct {
x, y, z float32
}
func IterVec(v Vec3) (func(func(int, float32) bool) {
return func(yield func(i int, v float32) bool) {
if !yield(0, v.x) { return }
if !yield(1, v.y) { return }
if !yield(2, v.z) { return }
}
}
22:00 Don't use `go mod tidy` to install new packages to your module, use `go get`... `go mod tidy` is similar to `poetry sync` and/or `poetry update` - it's meant to tidy up your deps as the name suggests
Iterator syntax isn't pretty, but 95% of Go devs were never going to write their own iterator anyway.
There are more comments saying "so much for Go being a simple language" than there are devs who will ever actually have to deal with this.
Plus this is pretty simple of you spend 10 minutes figuring out what it's doing. This whole controversy is just people seeing unfamiliar code and having a knee jerk reaction. Prime should know better than this
Could someone explain to me why Prime says that go supports structural typing ? I couldn't find anything about this on the internet.
Look how Odin does its iterators
I've looked at the iterators in the core library and personally i find them to be incomprehensible
@@adriancruz2822 lazy operations in inherently non-lazy languages look ugly, since you have to bring the state with you. Check how elegantly Haskell handles such things.
I was trying to learn about using Iterators before watching this, I ran into the same confusion around LSP not supporting my range .. I am very glad to see it happened in the video but less sad that someone in chat saved you 10minutes of scratching your head before the - let me fucking run it and see the go error
i use go generics for one thing, a ternary operator function
func Ternary[T any] (condition bool, ifval, elseval T) T {
if condition { return ifval}
else return elseval
}
i have this in all my go projects
ngl it's sad that you have to write your own basic control structures (as unexpressive functions no less) because the language tries to be "simple" to a fault.
@@Arcwise honestly, it doesn't bother me at all. What I love about the language is that simplicity. You do a lot of clever yet simple stuff with it. Honestly I've learned more while working with Go than any other languages (system programming languages excludes of course)
That doesn't work very well because you have to evaluate _both_ ifval and elseval, whereas with a real ternary operation, only the value in the chosen branch is actually evaluated. I suppose this works for simple things where evaluating both is not costly, but it's something you have to keep in mind if you ever do anything more complex with it.
@@maleldil1 if you're doing something complex then maybe just use a simple if statement. People abuse this operator but can we all agree it's only meant to be used this way 😑
A yield() would open the door to coroutine implementations
That's exactly what this is. Generators are an implementation of coroutines, "calling back into" the calling function.
It just uses closures and first class functions -- because Go has supported that since day 1 -- rather than some of the more magical control flow features of Python (try/catch and yield being keywords in the latter, and concepts in the former).
I don’t understand why you find this syntax awful. It’s behave like a Javascript generator that your consumer can said in your yield to stop execution.
You basically just create a function that returns your iterator function.
Your iterator function take as argument a function to send your data and you receive if the caller of the iterator left.
It’s just that, I don’t see why this generator syntax make you angry.
I love iterators but reading the new syntax for them is enlightening me to stop using them and instead use for loops for simplicity's sake. Hope it builds into something more suitable with time.
You never used yield in any language then? all behave similarly:
csharp:
public static IEnumerable foo(int number) {
for (int i = 0; i < number; i++)
yield return i;
}
js:
function* foo(number) {
for(let i = 0; i
Pablo Francisco was a great stand-up comedian
it would have been much easier if they made it imperative code instead of going all functional for it.
bro is just a function that return a function that take in input a function array of function over generic function...
Just like in derivatives maker, as long as we go on and there is no return, nobody will realize is just a functional factoid
Jesus people, it's literally just a function that returns a higher order function. You all are acting like the did something super complex here. You pass the range a higher order function, and it uses it to pass values to your loop. It's honestly not that confusing if you spend 10 minutes doing it.
Also 90% of people are only going to use this with iterator functions from libraries anyway. It's a nice convenience that barely adds Any complexity to the language.
6:00 truly a lsp moment
Create a simple Map function within a callback is way easier than do this yield stuff
I love Go but my god this syntax is horrific
This is not even the final form...
Hm
Feels like they are trying to go for a python style
for k,v in Map(foo).items():
print("{} : {}".format(k,v))
But instead of .items(), they use the range keyword
I would prefer Java’s verbosity over this convoluted Go syntax
Java's Stream API is the most elegant and nice to use way to operate on iterables (after Haskell).
@@ivanjermakovyou ever written Ruby?
@@Patrickdaawsome yeah, they're all very similar in JS, Java, Groovy, Ruby. My point was that you wouldn't expect this from Java :D
@@ivanjermakov Or LINQ, or Rust iterators, or Kotlin stream extensions, or literally just anything of that sort, they're all the same imo. Except for LINQ's ability to transform into squeal due to compile-time funkiness but that's a separate use-case entirely.
'I prefer hieroglyphics over this convoluted alphabet!'
looks almost as complicated as C++ 20 coroutines
Prime drifts further towards how PHP is implemented
Go doesn't miss much but when it misses it's by a mile.
Hm. I don't really see the use case for this, just do the Map (or whatever) beforehand and range over the result of that.
The compiler should optimize that away in any case and it's much more readable.
I mean, ok, yeah, you could range over an RNG but even that would be simpler with a simple loop.
So if you don't implement break then what happens if user tries to break? It just keeps on looping!?
I believe that's why you check the return of the yield function. Break is the same as returning false.
Not a problem if you don’t use lsps like a crutch
i very hope go has error handling like your_function.If_Error() that will automatically catch the error when an error happens
What neovim theme is Primeagen using?
it should not be called "yield", it should be called "do_iter_than_maybe_break" (or something shorter with similar sense)
@@squishy-tomato relax man, I know what the term means, it doesn't change the fact it's wrong. E.g. select should be race. Like if you select who will come first...
what means "lsp is not supported"?
People have to give Haskell a serious Go, I swear… 😅
And people are saying C++ is bad....
Sequence 2 Electric Boogaloo...
1, 2, 3 Go!
I love Go but this is awful. I will not be using this.
you probably will be at some point, just as the consumer of an iterator from a package you import.
arenas died for this
I’m so sad I wanted them to add arenas 😢
One of the dumbest things in go is that you can't really easily work with function definitions from other packages which is part of why the yield is so ugly instead of being a named signature.
I am kinda amazed how well rust managed their iterators back 10y ago. This yield keyword/variable bool syntax doesn't make sense to me.
There is no yield keyword in Go
Yield isn't actually a reserved keyword.
In Rust terms, think of these Go iterator like something of a try_for_each API. It's not THAT different after all.
(Just that their try_for_each has a callback with bool return type instead of Result, and they probably have a completely different API contract story on re-doing and/or continuing iteration on the same iterator.)
1. 2 3 here we go...
11:02
Yield is not a reserved word?
it's the name of the function hence: if !yield(args) {}
Go gets more complex and by getting more complex people will start to abuse the language like with JavaScript
Kaboose!
One year ago, I refused to believe my coworker who said that there are no iterators and functions on them (map, filter, etc.) in Go's std. When they compare themselves to C they really mean it huh.
And the way they integrated this feature into the language is anything but elegant.
3 2 1.GO!
look I had to do this 😅
at this moment we func up
Who needs iterators in go? Use a channel.
Channels have a lot of overhead
Channels and goroutines are for cross-thread communication. Generators are for communication/cooperation **within a single thread**.
Using channels to implement generators/coroutines is like using a mallet to swat a gnat. You can do it , but it will make more work for you elsewhere. Sometimes you need a more deft touch.
Looks very pythonic
That syntax is horrible.
It feels worse than most legacy spagetti code i've seen.
I can't even see a benefit or a problem it actually solves
go mod tidy = go mod sync
I almost forgot go existed. It's been a while
I was thinking about learning Go before watching this. This made me grateful I work with Java instead 😂
Or just any other language with perhaps the exception of C where you have a gun to your head while programming
I am a beginner dev, why don't we use a simple for loop instead of a iterator ??
it's not an "instead of" - a for loop (in modern languages) iterates over something, normally a collection like a map or slice (or whatever your language calls them), but in general could be anything, such as lines typed in from the user, the position of each zip file entry, every prime number...
"iterator" is the name of the "something" you need to be to use a for loop on it. The value is being able to tidy away behavior that happens at the *edges* of a loop, making it easier to use (and harder to write!)
There's lots of smaller details, but they won't matter much unless you're going to write these.
All arrays are iterable but not all iterables are arrays. With iterable, you can iterate over lazy, infinite, or unfinished data, such as user input, IO streams and generators.
@@SimonBuchanNz understood, thank you .!!