Thank you prime for being an absolute legend and helping me so much in keeping discipline in my journey to career swap from "catch-all IT Guy"/glorified runner of sfc /scannow to software engineer in my 30s. It's hard but I'm learning fast!
True, however they are using Kotlin + Spring, so it's almost guaranteed to be on the JVM. And even with Kotlin + Ktor I've only seen it run on the JVM. Do you know of anyone using for example Kotlin Native for microservices?
11:10 re: the billion dollar mistake. Haskell and Ocaml came out in the 90s and had option types pretty early. Python didnt, nor ruby, but they were dynamic langs.
@@kreuner11 Run-of-the-mill error checks and fallbacks. You can also get try-catch functionality with really simple, very popular libraries. Rolling your own try-catch is like 10 lines of code
@@deistormmods I feel like you just ignored the second half of my comment, which was about getting try-catches going in Go, which is ALWAYS an option even if not optimal. Regardless, Go's error handling is very robust even if we personally don't like it. It's just compiler-enforced guard clauses.
@@JanJozefo I think the "Go error handling bad" meme is more to do with the fact that functions return a result tuple that contains an error + a set of values. Allows weird edge cases where the function returns both an error and a value ... and you can ignore the error. People that are used to error unions that must be one of the other, find this a bit annoying and open to abuse. At least Go errors contain payload context, which is nice The whole "if err != nil" thing though, I don't see that as an issue at all. It might look OTT verbose sometimes, but at least it's explicit. There is nothing worse that inmplicit magic when you are trying to understand someone else's code. That's very hard to write Go code that is hard for someone else to understand.
Java introduced Optional types on Java 8, but null is so intrinsic to the design of the language that it is hard to avoid it. Unlike Kotlin that was designed from the ground up to explicitly state if a variable can be null or not, in that case you are bound to use the Optional type
Also, on Java, quality tools like to bitch if you state that a method is going to receive an Optional as a param. They don't want you to do that for some reason. You have to receive the var and envelop it in an optional inside the method, and have to do that in every method. Quite silly if you ask me.
not "liking null" is a memory management skill issue, that should go away the moment you hear the words "pointer", "stack", or "heap" a pointer on the Stack is the size of int, and the thing you are pointing to could be Far, Far larger then L3, or even RAM could hold, like what is the Size of a given Map(T): "(number of elements + a magic amount of elements) * structure size" so even the option wrapper might not be able to hold the entire Map. and testing if(MyDataType) or the more explicit if(MyDataType != Null/nil/nullptr) is just sanity. the other problem with java and Null is the wrangling it can do for the sake of "truthiness", has "null reference exception 0x0000" or "out of bounds exception" destroyed far to many programmers, that doesn't mean null is bad, it means "and next time we're going to check for null right?" (nods head while looking at ground)
“I started pushing Go to everyone because I don’t understand the other languages and now I am forcing inexperienced teams into a new language for mission critical stuff”. Awesome. The OP didn’t know Kotlin well enough to know about its null safety features but was sure enough to dismiss it and force Go on everyone. This sounds like a very inexperienced dev.
Exactly maybe for production revenue generating apps, just keep using the stack the team is expert in unless the current tech stack is giving some issues. As they say " if it works don't touch it". Ofcourse using some other language would make sense for say some experimental project.
The Dart language retro-fitted null-safety into a language that didn't previously have it. It took a migration period for all the various libraries to port onto the new major version. In the meantime, you could add hints as comments to get some preliminary lints during the transition. Once a project was ready to commit, they had a tool to convert all the comments to the finalized syntax.
Ahh yes. Go. I liked their error-safe style I actually started returning errors in my JS codebases instead of throeing them. Literally never had an uncought error since then.
I also feel the "we where down for an hour and lost $xxxk in revenue!" Claims forget that if something's broken people will comeback and try later. Outages happen and most clients will understand having an occasional hour long outage
Lack of null safety is probably the reason why I didn't really start using Go. My reaction was basically "why create sophisticated type system, and leave out the most valuable feature?" Uh, also implementing full-fledged exception handling, but insisting that you don't have them. I mean, I very much appreciate explicit error handling, but I don't appreciate lying about it, I guess.
Go has _a_ type system, but not a sophisticated one. the lack of null-safety probably stems directly from this trade-off (sac ADTs to gain simplicity). other languages also make simplicity a core value but make different trade-offs in doing so... everyone's favorite newboi Gleam seems to be one of these, and i'd bet that one can learn a lot from contrasting the two
@@radfordmcawesome7947 Go has static duck-typing, basically. I'd say it's sophisticated, even though it's not complex. OTOH, basic null tracking is quite straightforward. You just paint stuff nullable or non-nullable, that's it. What am I missing?
@@McFrax I would consider Algebraic Data Types to be "sophisticated" as it lets you compose types ad infinitum. The language would also have to treat the types as first-class citizens to qualify in my book. I wasn't referring to null-handling specifically. I have to admit that I haven't looked deeply at Go since before it had generics, so I may be misinformed. The most Go I do these days is mess around with other people's k8s controllers when something goes down at work.
In addition to sql pkg nulltypes, I’m surprised no one mentioned having a recovery handler to stop panics from crashing the program. Most Go http libs do this for you, and there are ways to do it with grpc middleware or to roll your own. The downside is that it might take longer to realize there’s an issue without a thorough monitoring setup
I am glad you made that statement about options being so important. Definitely emphasized how important the use is for professional programming for me. I will carry that with me into the future.
4:11 - No, the Go designers knew all about optional types (which are actually old AF, especially for people with even a bit of theoretical knowledge) They explicitly made the choice to not include them.
I never heard of the term "shadow traffic" before today but instantly recognized it as something like what I'd call "parity testing" when explained. To check that the new service has functional/output parity with the old service. Idk if that's an actual term either, I just started thinking about it in those terms one day at work. Unfortunately for me, this is very hard to do at my job because we have so many processes intertwined that operate on the same data (and lots of it) that it's hard to keep track to compare effectively at different points in the timeline. We've always done it in a pretty lackadasical manner when updating services which I want to change, but am having trouble getting buy-in because for now at least, it really slows me down. Anyone have any resources on practical effective strategies for this sort of thing?
I've come across this before with different terminology. Whenever we've had two similar systems where we want to transition from System A to System B, we'd do it in a couple of steps. Step 1 would be what we called ghosting, where System A is live and System B is run in the background while its output is tested. Then System B is live while System A is kept running to roll back to in case System B fails. Then System A is taken offline. Key takeaway is that we called it 'ghosting' not shadow testing. But I think the idea is the same.
We always called it mirrored traffic. Same thing. It's a great approach. Once you build your tools it's possible you can use the collected traffic to simulate testing during development and testing.
I just discovered that the time it takes to grind fresh coffee beans with my hand cranked grinder, set up the coffee maker, make the coffee, pour it in a cup, and bring the cup to my lips is the same amount of time it takes to watch this video. 13 minutes and 24 seconds.
oh nice. i assume you are brewing it manually as well. do you make it with french press or do you have fancy brewing equipment? i never really timed it but i think my similar routine takes around 7-8 minutes total, 4 min brewing time for filter coffee. seeing anyone using a hand cranked grinder makes me really happy.
@@ardnys35 no I gave up on all that and just use drip, lol. The biggest difference is using a burr grinder, I think. I even had an espresso machine at one point, but it wasn't worth it. At some point my french press broke, so I broke out my old Mr. Coffee. Hotter water would be better, but the drip method is almost as good. I'll pick up a new french press someday, if I see one on sale. I got the hand cranked grinder because the good electric burr grinders were $$$, and I didn't want to buy one only to decide I liked a different one better. Plus, when I'm making coffee, I'm trying to wake up anyway, so adding a little elbow grease into the ritual is pointing me in the right direction.
Shadow Traffic is simple in principle, but shadowing a service that mutates data in some live datastore can be tricky. You can’t let New Service update the same downstream Prod systems as Old Service, but you can’t verify that NS works without running the code; do you mirror all of Prod to run your NS on? That works but can be challenging or at least expensive at scale. Do you mock the downstream datastores with in-memory diffs? Or if all the calls to downstream systems are identical to the calls made by OS so you can cache those and use them in NS - except this affects performance metrics for NS so interpreting the results gets harder. Basically, if is a readonly service with no side effects, then shadow traffic is super simple, or if you can spin up a full mirror of Prod (this is awesome by the way). Sadly not all situations are so lucky.
I think the problem is that to add an option type to Go in the standard adds a big question about every single function in the standard that returns raw pointers. For example, errors. All errors are interfaces (pointers) which need to be nil checked. If there is a standard option type, should error checking be done in the old way, or should functions return an optional error? Or perhaps a result type that could be either a value or an error. Go has built itself so far in one direction that it might just be too far gone at this point, for better or for worse
Go doubled down on the C nil train. You have to be more explicit in rust when you write something like an unwrap instead of more careful handling, but using unwrap is roughly the equivalent of not checking nil in go.
It can be simulated with generics for your own library uses, but in order for it to become an option in stdlib and generally pushed as a standard, we'd need a breaking version change, and that's something the Go team isn't really into until there's enough reason to do it. Backwards compatibility is a huge selling point and Go isn't likely to break that anytime soon
"100 Go Mistakes and How to Avoid Them" is a good book indeed. But it's mostly not about developers' mistakes, it's about language shortcomings and how to avoid them. // Go devs: Give us iteration protocol, to use with custom collections in loops! Go authors: Let's change how the 'for' loop captures variables, and make it in a minor release, that's _really important_! ))) Yeah...
loop variable fix was a good and awaited change Iterators would make the language more complex and Go strives to be a simple language (I do fear that at some point someone will still add them just like they did with generics though) Not everything needs to be a kitchen sink like C++ or Rust I really love how stable and backward-compatible Go is
@@bionic_batman Other languages, like JS (w/ var) or Python, have the same behavior w/ for, so almost none care about it. And, there some dudes suggested just adding new syntax construction, like 'foreach', to cover this fix and backward compatibility. It was just neglected. The Go way... Golang possibly would be a simple language. But it is just pretending to be, instead, it is rather "ad-hoc" language. Almost any piece of language is built in a non-obvious way, different from almost any existing language. And worst of all, new fixes just add new ad-hocs, and devs instead of having some solid instruments, that should be learned once and can rely on in any conditions - like generics or iteration protocol - have to learn all nuances of those ad-hocs and their combinations. The main focus of language developers is clearly in the wrong place. The sum-type (enum)? No. Some Option type? No. Better syntax for handling errors? No. Support custom collections in loops? No. The MIN and MAX built-ins! YES! You lack it! You need them so badly!!! ))))
@@AlexanderBorshak Sometimes less is more. Just accept that Go has different philosophy which really shines when you have a large team of devs working on the same codebase. If you prefer languages with kitchen sink mentality that have all sorts of bells and whistles which iterators and sum-types are then use Rust, nobody forces you to use Go But making all languages into the same strange monstrosities that have tons of different paradigms mashed together is a really bad idea. Some people just value backward compatibility, simplicity and stability over some badly over-engineered syntactic sugar mess
@@bionic_batman The irony is, that even with 'for' despite just one syntax construction there are many semantics - one for regular loop (C-style), one for while (for w/ condition), one for 'foreach' (w/ range), and one for infinite loop (w/o conditions). So it's just trying to pretend simple, while you still have to remember all nuances of each semantic, so having different syntax possibly even would be better since the mental load would be lower. But OK, I accept "different philosophy which really shines". Let it shine!
kotlin simulates the “reduction” functionality of an optional type with its rigid nullable types system and its type inference system paired with things like the elvis operator
We had exactly this a week ago. A nil dereference made a Go microservice crash and restart infinitely. There was a cascading failure and the whole prod went down.
Going from a raised type to its underlying value isn't a lift operation, but you were right calling it a catamorphism. It's duel an anamorphism, is going from a type to a raised type (e.g. functor, monad etc). A lift operation is related but its focused on transforming a value in a given context, this is most commonly called a `map` Assuming you have an array datastructure that satisfies functor laws, a reduction algorithm of [T] -> T is a catamorphism where as something like T -> [T] would be an anamorphism. Additionally if you compose these together (Cata after Ana) its called a hylomorphism (e.g. Cata :: F A -> A, Ana :: A -> F A, Hylo :: Cata | Ana). So if you had functions: safeDivide :: int -> int -> Option # catamorphism optionMap:: (int -> int) -> Option -> Option # lift operation x = safeDivide 2 0 # results in empty Option a catamorphism from int -> Option (assuming curried) y = optionMap ((+) 1) x # lifts increment operation into Option context to be applied against the inner value (if not null in this case)
Watching the Primeagen makes me understand how much more there is to learn. Thank god. Could you imagine how boring life would be if you already knew everything and ran out of life material to feed your brain?
Rust may not have null pointers, but it has string[index] which panics if your index is not on a char boundary. Something you can easily fuck up and end up with a panic during runtime. Every language has at least a few pitfalls you just gotta stumble into and experience first.
@y00t00b3r yes, if you take a slice of a multi byte character not on a char boundary, there will be a run time panic. Note that directly indexing into a string is not allowed, only slices can be taken. Taking a slice of string is not recommended. You can get around this by using .get and handling the case when it’s not on a char boundary, or more preferably being explicit with what you want with .chars to get an iterative of chars or .bytes to get an iterative of bytes, neither of which can fail during runtime
This same problem also happened to me and I also took down production. TL;DR: Every goroutine in go needs to run with a recover. Sometimes the package that you're using already does that for you, but if it don't, you have to do by yourself. Correcting the chat: a panic inside a goroutine, indeed break the entire application and not only the goroutine where it happened. The problem is that usually (at least for my job) we are building http services and if even if you use the standard http library from go, it already comes with a recover mechanism. Each request becomes a goroutine and for every single one of them, the http package makes it safe from panicking. The problem is that if your code is already running inside that "request specific goroutine" and from there you start another goroutine, you have to explicitly use the recover keyword for that new goroutine. If you don't and your code panics, the entire program crashes.
I don't know why I'm entertained by this channel. Half the time I don't know what he's talking about. I guess I feed the occasional nuggets of cool insights he puts out.
I'm a big fan of TypeScript, but it's important to acknowledge that it can sometimes give a false sense of security. For instance, when you declare an array of numbers and use the array's sort method, you might expect the elements to be sorted numerically by default, considering it's an array of numbers. However, the default behavior of arrays in JavaScript is to sort the elements by converting them to strings and ordering them lexicographically, not numerically. This highlights the importance of using a linter tool like ESLint to help catch and prevent these subtle, potentially confusing behaviors.
One thing I dislike about std::optional is the implicit padding from that lone bool. I want an optional that can be grouped in memory as bool[8] followed by the payload for those 8 variables.
@@JuusoAlasuutari That's fair. There is definitely a cost to using std::optional or std::expected. For my frequently called functions, I still choose another strategy.
@@JuusoAlasuutari Seems like something that would require polymorphism, which would break if you returned optional_set by value. Not certain how you would achieve that without some additional flag byte. ¯\_(ツ)_/¯
8:40 Option or Optional is a functional programming thing, even Java has it since Java 8 (the version that included functional programming paradigm in Java) and it's often used inside Spring library or with Stream functions, in Java has 2 functions, isPresent and get, the first one checks whatever has a value, and the second one gets it (with all the risks related to it), intellij tells you as warning that get without if(isPresent) is a bad practice
There is ifPresentOrElse which allow me to handle the error on point and there are functions to assign a default value if the optional is empty. There are nice functions for a lot of use cases and I have been getting a lot less NullPointerExceptions. but these are a bit weird when you just see them in these languages. i learned it first with Rust and then transferred the ideas to java. i tried to explain Options and how nice they are to many people but they don't really get it
This is not a programming problem. That company gave an inexperienced team the responsibility of putting a critical app in production, using a language they weren't comfortable on. They launched the app without reviewing the code, and without enough QA. Why couldn't they fallback to the old service? All the question that comest to mind are related to their processes, not their programming skills.
The option type thing just sounds like the `return value, err` pattern in Go. If you're not checking `if err != nil` then you're going to have problems anyway.
So glad people are finding out about Monads now. Would've been better if they caught on in like 1974, but hey, everyone learns at their own speed I guess
Write and use your own Optional type and use it until it's a built-in. It'll work wonderfully, and with a little bit of work you can also get it to work when en/decoding. I also wrote (RW)Mutex wrappers similar to Rust, which have been wonderful.
Panic returns to the calling function recursively until its handled if not it reaches main and kills the app. Doesn't automatically kill the app as soon as you panic
Every time that i see some “problem” related to go, usually is more related to the skills of the programmer. So, they probably don’t know about pointer/error checking, neither recovering. I admit, they do have balls to ship something to production without knowing how it’s working (and without test, probably)
@@svetlinzarev3453Well, rust is safer but that doesnt equal superior. You know, choosing the right tool for the right job. While I dont like go's nil pointers (altough there is tooling to catch these errors in the ide) this story has so many red flags that we cannot blame the whole thing on the language. Ps: I love both rust and go
@@svetlinzarev3453 I can make Rust code that has expects everywhere, it will compile just fine but will crash in production No amount of safeguards will prevent you from shooting yourself in the foot if you are persistent enough
This is why I like Salesforce development - you really can't take down the server via any normal means. And writing code to delete the database or even a large table is not easy to do, so that probably wouldn't happen by mistake either.
Using try / catch helps to: - skip a couple stack frames to handle errors (sometimes) - find where errors come from (not) - accidentally "handle" errors that should never occur (skill issue) (still a bug)
I don't understand how it can be claimed that an option type would fix this situation. If they fail to check the returned value they will still have an issue. In Java... they call rv.get() and they had "none" then it throws. And if they didn't bother to check the return value for null why would we assume they would have written the exception handling properly? The dereference isn't the problem. It is failure to do proper error checking.
Hi Prime pretty naive question here: where I can learn about Architecture Patterns and those things like: canary builds | shadow traffic and those type of thing aside from goggling those stuffs when I hear them. can you suggest book, course or anything like that. would be very thankfully happy for your answer
Programing sub reddits might be a better place to ask this question. There are a bunch of great books that cover these topics from O'Reilly and Manning.
To have an entire app crash in production because of a nullpointerexception means: 1. you used a value where you should have checked for null without checking. 2. whenever that happens your entire app crashes. 3. Either nobody checked your code or they didn't see it either. 4. It wasn't tested sufficiently. Point 1 and 3 I understand. People make mistakes and a nullcheck is easily forgotten, especially if you're new to it. But where was the testing in all this? Where are the guardrails to get your app back on track when something goes wrong? Are you telling me that every exception or problem just kills the entire app? This doesn't seem like a GO issue. This seems like either a process issue or a skill issue.
5:40 "This whole `try...catch` business" - you are missing the point about `try....catch` vs `error` in Go. Exceptions BUBBLE, you don't have to try-catch-rethrow on every call-stack level. Also, all `async/await` errors, even though calls are asynchronous, are properly handled with regular try/catch block that wraps them. So text author is right. One try/catch at the top level would keep service alive. Would it help customers, that's another question that depends on the service implementation.
@@samwilde8311 That is Markdown annotation for code. So that it is clear that I don't ask reader to "try" something, but that I refer to a `try` reserved keyword in a program. :)
Shadow traffic is great, but won’t help if your flag is turned off. This is why feature flags can also be a huge problem if not used correctly. You put a flag into the codebase, in an “off” state, merge code to main, deploy, and then don’t switch the flag on until weeks later. Meanwhile further commits and releases have happened, which version do you roll back to? Turning the feature flag “off” again may solve the problem in the short term, but how do you know for sure which commit caused it? Sounds like a combination of things that didn’t go right here.
How would you deal with side effects in shadow traffic scenario? Do you go all-functional and do managed side-effects? Doesn’t seem feasible with the whole “rewrite of an old imperatively written service”
It doesn't work in all situations. You can clone the DB so the shadow gets its own data store, if your calling out to other external services that have side effects it's probably not a good fit for shadow traffic.
Exactly. You need to have really thought through your "is shadowing" env flags or whatever is best to mock up or turn off or otherwise handle all of those things the best way you can engineer. The more complex the system, lots of microservices lots of third party apis, bockchains, emails, sms, S3, kafka, nats, stream processing, billing and payments, lord knows what, your shadowing can really get you into trouble if you have not engineered your shadowing for all of that ( looks like our shadow double billed everyone and deployed 50k redundant smart contracts ). Crud app that cruds to a DB? Shadowing is a simple and great choice.
Dunno about option/optional. Not providing a value is not in every context the same as providing an empty value (normalized empty value). When do you know that the empty value is intended or no value was provided? Providing an unsuitable empty value can lead to wrong inferences when doing statistics on the data. E.g. you don't provide a price for a good but set it per default to zero. When someone else naively calculates average and median prices. Boom.
I gave up on Java exceptions ages ago. I ended up writing a whole try-catch around the entire app and restarting it when it blew up. It technically never crashed. It just restarted instead of crashing. Everything in Java is an exception including the stupidest stuff you'd never think in a million years would be an exception. The best solution is to catch and ignore all exceptions and just restart the Java app. Problem solved.
an "uncaught" panic in a goroutine brings down the whole app. Frameworks like Fiber etc provide "recover" middleware to catch them on the request goroutines themselves which is nice but if you were to say create another goroutine within it and make an uncaught panic there everything will die. ie. go func() { panic("oops") }() inside a handler. Anything having a nil zero value is a pain, I'd rather opt-in to it if it's a nillable type. My biggest gripe with maps though is they are not thread-safe for writing OR READING, had quiet a few data cache corruption issues (not crashes, just plain wrong data coming out) from people forgetting that. Don't put maps in a RWLock either, only one reader at a time ever or bad things will eventually happen.
@@teodormaxim5033 I believe it's due to how it does the key lookup internally. It doesn't like multiple happening on the map at once. I've seen maps end up in a state where multiple keys end up having the same value in memory, the original values lost to the wind after only reads/lookups, not great for a thumbnail cache let's say (last time I saw it was a map of string to []byte for thumbnails a front-end was requesting like 20 at a time for a webpage when it loaded). It's a shame that Go only has one builtin map type and it's very not thread-safe. It is very efficient though as a tradeoff afaik. The tldr, which the docs do mention but I don't think quite stress hard enough is always access a map via a mutex for concurrent access, regardless of what you're doing with it. Once you've got the value out of the map (even if it's a map of pointers / slices etc) you can unlock it and do whatever you want with the value without having a lock and be thread-safe, the issue seems to be the mapping of the keys to the values rather than the values themselves moving around or changing underneath you.
There's also an issue with slices I think and how some web frameworks work (thinking Fiber), so pulling a slice out of a map and throwing it straight into a response without cloning I think can end badly due to some pointer cascading. There's quite a few annoying caveats with slices now and then depending on how they were made but if in doubt, clone.
Note, Kotlin is null aware, it doesn't mean you can't run into a null pointer provider from the client json request, it will throw NullPointerException at run time even if declared as non-nullable. Code example @Serializable data class Project(val name: String) fun main() { val dataSentByClient = mapOf("name" to null) val gson = Gson() val serializedJson = gson.toJson(dataSentByClient) val parsedObject = gson.fromJson(serializedJson, Project::class.java) println(parsedObject.name.uppercase()) }
While this is a fair point, it's also pertinent to point out that the source of that null value is a Java library: null-checks on platform types (e.g. values coming from Java code) are relaxed so that you aren't forced to elvis every single value you get from Java. The unexpected NullPointerException is a consequence of the limitations of the Java type system and the convenience that Kotlin provides for interacting with it. You'll be hard-pressed to get a null from pure Kotlin code without shenanigans. There is unfortunately no way to make the compiler enforce this, but you could in principle follow a convention of always assigning platform-typed values to variables of nullable type. IIRC at one point the Kotlin compiler forced you to !! every platform-typed value, but this turned out to be impractical so it was dropped.
6:00 100% disagree. The set of errors of, let say C, is greater than in Java just due to what the language allows: certain kind of errors cannot exist in some languages due to their design (this does not mean that "there are no errors in X language")
>Building a microservice which returns data from in-memory cache Sounds like typical overengineered bs. RDBMS already have caching mechanism like shared buffers, also you can scale RDB for read-heady traffics by just adding read replicas as much as you like.
Dealing with nils in go, is mostly the same problem as with JS, if anything is nilable, either make sure, we have a Builder or Factory populating that value, or check for nil in code, or in the database. The enforcement has to come from somewhere. Generally while migrating old code, db changes aren't always possible. So better to handle nil explicitly. Sad that we mostly have typescript programmers these days.
null and option are really the same thing. The only difference is when the language doesn't make you check for null explicitly before trying to read the value. They generally check it implicitly, and then throw an exception rather than letting you access garbage, it just needs to be enforced explicit. You can do this with either null or option, but need language support.
So, obviously, they are NOT the same thing, since with Option you MUST check if a value exists before using it, i.e. Option gives you strict guarantees; while null shifts all responsibility onto you. It's opposite things, not the same.
@@AlexanderBorshakI think he’s referring to null in null-safe languages like Kotlin, where null is truly just syntax sugar for Option. Kotlin’s “!!” operator is exactly the same as “.unwrap()”, “?:” == “.unwrap_or”, and “?.” lets you call a method (or read a property) if and only if the value is not null. By default, you can’t do anything else with a value of type “T?”
I love Zig for how it chooses to handle null values and errors. Putting it into my hands so I can control what happens instead of doing the usual style of try-catch (I am glad I am not the only one who hates those, wasn't able to really verbalize why until using Zig). It all just becomes directed behavior instead of relying on how they choose for errors and such to be handle behind the scenes. It does require you to be conscious of even more engineering, but if you are using Zig, you specifically signed up for that.
I think go is a little bit worse than Kotlin in terms of footguns. For sure. Go seems to not care about null safety at all, were as Kotlin obviously at least tries to do something about it.
if I ever write a programming language I'm going to call my option type null and the none type also called null, and you access it with the following statement: if null is not null...
Why switch to Go when you already have a good modern language like kotlin and devs experienced in using it? I doubt you would gain any significant performance gains as Netty is quite fast and you can easily write non blocking code with kotlins coroutines. Startup times shouldn’t be a big concern as they are loading a bunch of data on startup as well, and if it is there are other ways of optimizing it rather than switching stack entirely. Go uses less memory but that by itself isn’t a big deal. I don’t see any language features in Go that would warrant switching either.
Imo its shortsighted to just blame it on skill issue and be done with it. In any given team very few developers will be very skillful and others will have skill issues (often because they do not care about their job, its just a job for them). Good languages will not take your entire product down just because some junior had a skill issue. Good languages nudge you towards writing readable and performant code just by default. Kotlin is great because it does it, specifically with nulls, often i saw noobs who would fix their own prs becuase the code looked "ugly", too many question marks, so they tried to fix it and actually wrote a null safe clean code that also gets chewed up into well optimised machine code by jvm... thats the whole point, you write normal code but it gets compiled into some mad C developer super optimised magic without your knowledge. Kotlin rocks, and with kotlin native/cross platform imo jetbrains is well on its way to totally domainate language and tooling business in the years to come.
I often find "skill issues" means the dev doesn't know the gotchas. All languages have them and the less they have the better. The default way of doing stuff should be safe where at all possible.
@@101Mant yes and thats why java and jvm was so dominant all these years. Languages that are capable to unlock all hardware performance + be maintainable with low barrier of entry are the most dominant. Only exception i can think of is JS but it has a good reason why it got popular.
@@Patmorgan235Us i agree with lack of testing/review process argument, skill issue imo is not avoidable, very rare companies/products can afford only hiring seniors. Most companies will have more juniors than seniors. So you want to minimise possible issues that depend on skill. Ideally you should be able to take someone fresh from bootcamp and get them to be a useful contributor. So skill issue is not an argument here (unless the person fucking up is actually a well paid senior)...
btw in Go, you can do reference | nil (and the language actually knows the difference). "panic: illegal move: Kg9: index out of range [8] with length 8 ", pythonistas!
"can" is the problem, right? Because the word should be "must" and "checked by the compiler". And since typing is not required and code is not AOT compiled, Python is a whole other level of awfulness. btw I use -arch- Kotlin
Go is fantastic, especially in the hands of former C programmers. But Go does seem particularly vulnerable to onboarding Java developers. Not suggesting they are inherently evil, but they do have a "different" way of looking at systems, and that thinking - when applied to Go, ends up In a horrible mess of pseudo-Java abstractions more often than not. The worse thing that ever happened to Go was when it got "popular", a lot of experienced Java devs jumped in, and steered it in a terribly wrong direction. Now we have Go with a broken java-style implementation of generics, which is kinda messed up. It would have been far more advantageous if the Go team ignored the pleas of the new Java devs, and instead concentrated on - optional manual memory management + better error handling options to tune it towards low level systems programming. Ah well, it's a bit late now. We are stuck with a Go that is commonly overlooked for a few reasons.
"Every function that has await can throw." EVERY function call can throw, whether or not it's called with await. Or do you mean even when analyzing the body and it has no way to throw an async call can throw?
Ahh, the devs earned their, "I brought down production" badge. Well done, well done.
Some would say it's the most important badge of all maybe only second to wiping the database without a recent snapshot/backup 😌
Thank you prime for being an absolute legend and helping me so much in keeping discipline in my journey to career swap from "catch-all IT Guy"/glorified runner of sfc /scannow to software engineer in my 30s. It's hard but I'm learning fast!
All the best bud
How did you find your first gig
Just a note, Kotlin doesn't necessarily assume it's running on the JVM, there are Native, JS, and WASM targets.
"Just a note, Kotlin doesn't necessarily assume it's running on the JVM, there are Native, JS, and WASM targets."
- 🤓
@@igoralmeida9136 kick this bot outta here...
Isn't it compiling to Java? And Java to bytecode?
Or am I missing the latest developments in both languages?
True, however they are using Kotlin + Spring, so it's almost guaranteed to be on the JVM. And even with Kotlin + Ktor I've only seen it run on the JVM. Do you know of anyone using for example Kotlin Native for microservices?
@@VojtasIIktor can compile to any of those services, you need to use a different engine instead of Netty iirc
11:10 re: the billion dollar mistake. Haskell and Ocaml came out in the 90s and had option types pretty early. Python didnt, nor ruby, but they were dynamic langs.
F# too has the option type and is a very good language. It seems that having roots in the ML family language is a very good thing!
i felt that they just 1:1 ported that service to golang and replace those old try catches with fatal/panics
Hm, what's there to do if there's no other handling?
@@kreuner11 Run-of-the-mill error checks and fallbacks. You can also get try-catch functionality with really simple, very popular libraries. Rolling your own try-catch is like 10 lines of code
@@JanJozefoGolang error handling isn't great.
@@deistormmods I feel like you just ignored the second half of my comment, which was about getting try-catches going in Go, which is ALWAYS an option even if not optimal. Regardless, Go's error handling is very robust even if we personally don't like it. It's just compiler-enforced guard clauses.
@@JanJozefo I think the "Go error handling bad" meme is more to do with the fact that functions return a result tuple that contains an error + a set of values. Allows weird edge cases where the function returns both an error and a value ... and you can ignore the error. People that are used to error unions that must be one of the other, find this a bit annoying and open to abuse.
At least Go errors contain payload context, which is nice
The whole "if err != nil" thing though, I don't see that as an issue at all. It might look OTT verbose sometimes, but at least it's explicit. There is nothing worse that inmplicit magic when you are trying to understand someone else's code. That's very hard to write Go code that is hard for someone else to understand.
Java introduced Optional types on Java 8, but null is so intrinsic to the design of the language that it is hard to avoid it. Unlike Kotlin that was designed from the ground up to explicitly state if a variable can be null or not, in that case you are bound to use the Optional type
Also, on Java, quality tools like to bitch if you state that a method is going to receive an Optional as a param. They don't want you to do that for some reason. You have to receive the var and envelop it in an optional inside the method, and have to do that in every method. Quite silly if you ask me.
@@Titere05I think it is like that because you can pass null instead of Optional as an argument
null is just incredibly useful thats why we love it so much
not "liking null" is a memory management skill issue, that should go away the moment you hear the words "pointer", "stack", or "heap"
a pointer on the Stack is the size of int, and the thing you are pointing to could be Far, Far larger then L3, or even RAM could hold,
like what is the Size of a given Map(T): "(number of elements + a magic amount of elements) * structure size" so even the option wrapper might not be able to hold the entire Map.
and testing if(MyDataType) or the more explicit if(MyDataType != Null/nil/nullptr) is just sanity.
the other problem with java and Null is the wrangling it can do for the sake of "truthiness", has "null reference exception 0x0000" or "out of bounds exception" destroyed far to many programmers, that doesn't mean null is bad, it means "and next time we're going to check for null right?" (nods head while looking at ground)
@@gardian06_85 pointers are almost always 8 bytes idk what u mean is the size of int
“I started pushing Go to everyone because I don’t understand the other languages and now I am forcing inexperienced teams into a new language for mission critical stuff”.
Awesome. The OP didn’t know Kotlin well enough to know about its null safety features but was sure enough to dismiss it and force Go on everyone. This sounds like a very inexperienced dev.
Exactly maybe for production revenue generating apps, just keep using the stack the team is expert in unless the current tech stack is giving some issues. As they say " if it works don't touch it". Ofcourse using some other language would make sense for say some experimental project.
The Dart language retro-fitted null-safety into a language that didn't previously have it. It took a migration period for all the various libraries to port onto the new major version. In the meantime, you could add hints as comments to get some preliminary lints during the transition. Once a project was ready to commit, they had a tool to convert all the comments to the finalized syntax.
As a newbie, I really appreciate when you explain some of these terms. Thank you.
Ahh yes. Go.
I liked their error-safe style I actually started returning errors in my JS codebases instead of throeing them.
Literally never had an uncought error since then.
Yep, can't catch errors if you don't throw
@@jimhrelb2135but you can uncatch errors…
A company makes 100k revenue per hour but doesn't have a test db??? Qa???
Maybe now people will learn to stop testing in prod
> maybe
Maybe they just wanna to have some fun, what's the fun part if you can't wipe your prod db easily and lose a ton of money?
@@lauraprates8764 that money was asking for it.
I also feel the "we where down for an hour and lost $xxxk in revenue!" Claims forget that if something's broken people will comeback and try later. Outages happen and most clients will understand having an occasional hour long outage
@@Patmorgan235Us valid, unless it's for some sort of live event
Lack of null safety is probably the reason why I didn't really start using Go. My reaction was basically "why create sophisticated type system, and leave out the most valuable feature?"
Uh, also implementing full-fledged exception handling, but insisting that you don't have them. I mean, I very much appreciate explicit error handling, but I don't appreciate lying about it, I guess.
Go has _a_ type system, but not a sophisticated one. the lack of null-safety probably stems directly from this trade-off (sac ADTs to gain simplicity). other languages also make simplicity a core value but make different trade-offs in doing so... everyone's favorite newboi Gleam seems to be one of these, and i'd bet that one can learn a lot from contrasting the two
@@radfordmcawesome7947 Go has static duck-typing, basically. I'd say it's sophisticated, even though it's not complex.
OTOH, basic null tracking is quite straightforward. You just paint stuff nullable or non-nullable, that's it. What am I missing?
@@McFrax I would consider Algebraic Data Types to be "sophisticated" as it lets you compose types ad infinitum. The language would also have to treat the types as first-class citizens to qualify in my book. I wasn't referring to null-handling specifically.
I have to admit that I haven't looked deeply at Go since before it had generics, so I may be misinformed. The most Go I do these days is mess around with other people's k8s controllers when something goes down at work.
In addition to sql pkg nulltypes, I’m surprised no one mentioned having a recovery handler to stop panics from crashing the program. Most Go http libs do this for you, and there are ways to do it with grpc middleware or to roll your own. The downside is that it might take longer to realize there’s an issue without a thorough monitoring setup
I am glad you made that statement about options being so important. Definitely emphasized how important the use is for professional programming for me. I will carry that with me into the future.
4:11 - No, the Go designers knew all about optional types (which are actually old AF, especially for people with even a bit of theoretical knowledge)
They explicitly made the choice to not include them.
@@nisonatic well, no lsp warnings aside it's a pretty loud stacktrace :D
I never heard of the term "shadow traffic" before today but instantly recognized it as something like what I'd call "parity testing" when explained. To check that the new service has functional/output parity with the old service.
Idk if that's an actual term either, I just started thinking about it in those terms one day at work. Unfortunately for me, this is very hard to do at my job because we have so many processes intertwined that operate on the same data (and lots of it) that it's hard to keep track to compare effectively at different points in the timeline. We've always done it in a pretty lackadasical manner when updating services which I want to change, but am having trouble getting buy-in because for now at least, it really slows me down.
Anyone have any resources on practical effective strategies for this sort of thing?
I've come across this before with different terminology. Whenever we've had two similar systems where we want to transition from System A to System B, we'd do it in a couple of steps. Step 1 would be what we called ghosting, where System A is live and System B is run in the background while its output is tested. Then System B is live while System A is kept running to roll back to in case System B fails. Then System A is taken offline.
Key takeaway is that we called it 'ghosting' not shadow testing. But I think the idea is the same.
We always called it mirrored traffic. Same thing. It's a great approach. Once you build your tools it's possible you can use the collected traffic to simulate testing during development and testing.
I just discovered that the time it takes to grind fresh coffee beans with my hand cranked grinder, set up the coffee maker, make the coffee, pour it in a cup, and bring the cup to my lips is the same amount of time it takes to watch this video. 13 minutes and 24 seconds.
oh nice. i assume you are brewing it manually as well. do you make it with french press or do you have fancy brewing equipment? i never really timed it but i think my similar routine takes around 7-8 minutes total, 4 min brewing time for filter coffee.
seeing anyone using a hand cranked grinder makes me really happy.
@@ardnys35 no I gave up on all that and just use drip, lol. The biggest difference is using a burr grinder, I think.
I even had an espresso machine at one point, but it wasn't worth it.
At some point my french press broke, so I broke out my old Mr. Coffee. Hotter water would be better, but the drip method is almost as good. I'll pick up a new french press someday, if I see one on sale.
I got the hand cranked grinder because the good electric burr grinders were $$$, and I didn't want to buy one only to decide I liked a different one better. Plus, when I'm making coffee, I'm trying to wake up anyway, so adding a little elbow grease into the ritual is pointing me in the right direction.
Shadow Traffic is simple in principle, but shadowing a service that mutates data in some live datastore can be tricky. You can’t let New Service update the same downstream Prod systems as Old Service, but you can’t verify that NS works without running the code; do you mirror all of Prod to run your NS on? That works but can be challenging or at least expensive at scale. Do you mock the downstream datastores with in-memory diffs? Or if all the calls to downstream systems are identical to the calls made by OS so you can cache those and use them in NS - except this affects performance metrics for NS so interpreting the results gets harder.
Basically, if is a readonly service with no side effects, then shadow traffic is super simple, or if you can spin up a full mirror of Prod (this is awesome by the way). Sadly not all situations are so lucky.
"The name..... is the Shadow Traffic" - Prime Traffic 2024
I think the problem is that to add an option type to Go in the standard adds a big question about every single function in the standard that returns raw pointers. For example, errors. All errors are interfaces (pointers) which need to be nil checked. If there is a standard option type, should error checking be done in the old way, or should functions return an optional error? Or perhaps a result type that could be either a value or an error.
Go has built itself so far in one direction that it might just be too far gone at this point, for better or for worse
Go doubled down on the C nil train. You have to be more explicit in rust when you write something like an unwrap instead of more careful handling, but using unwrap is roughly the equivalent of not checking nil in go.
It can be simulated with generics for your own library uses, but in order for it to become an option in stdlib and generally pushed as a standard, we'd need a breaking version change, and that's something the Go team isn't really into until there's enough reason to do it. Backwards compatibility is a huge selling point and Go isn't likely to break that anytime soon
"100 Go Mistakes and How to Avoid Them" is a good book indeed. But it's mostly not about developers' mistakes, it's about language shortcomings and how to avoid them. // Go devs: Give us iteration protocol, to use with custom collections in loops! Go authors: Let's change how the 'for' loop captures variables, and make it in a minor release, that's _really important_! ))) Yeah...
yeah, that was an interesting thread in the issue tracker... :/
loop variable fix was a good and awaited change
Iterators would make the language more complex and Go strives to be a simple language (I do fear that at some point someone will still add them just like they did with generics though)
Not everything needs to be a kitchen sink like C++ or Rust
I really love how stable and backward-compatible Go is
@@bionic_batman Other languages, like JS (w/ var) or Python, have the same behavior w/ for, so almost none care about it. And, there some dudes suggested just adding new syntax construction, like 'foreach', to cover this fix and backward compatibility. It was just neglected. The Go way... Golang possibly would be a simple language. But it is just pretending to be, instead, it is rather "ad-hoc" language. Almost any piece of language is built in a non-obvious way, different from almost any existing language. And worst of all, new fixes just add new ad-hocs, and devs instead of having some solid instruments, that should be learned once and can rely on in any conditions - like generics or iteration protocol - have to learn all nuances of those ad-hocs and their combinations. The main focus of language developers is clearly in the wrong place. The sum-type (enum)? No. Some Option type? No. Better syntax for handling errors? No. Support custom collections in loops? No. The MIN and MAX built-ins! YES! You lack it! You need them so badly!!! ))))
@@AlexanderBorshak Sometimes less is more.
Just accept that Go has different philosophy which really shines when you have a large team of devs working on the same codebase.
If you prefer languages with kitchen sink mentality that have all sorts of bells and whistles which iterators and sum-types are then use Rust, nobody forces you to use Go
But making all languages into the same strange monstrosities that have tons of different paradigms mashed together is a really bad idea.
Some people just value backward compatibility, simplicity and stability over some badly over-engineered syntactic sugar mess
@@bionic_batman The irony is, that even with 'for' despite just one syntax construction there are many semantics - one for regular loop (C-style), one for while (for w/ condition), one for 'foreach' (w/ range), and one for infinite loop (w/o conditions). So it's just trying to pretend simple, while you still have to remember all nuances of each semantic, so having different syntax possibly even would be better since the mental load would be lower. But OK, I accept "different philosophy which really shines". Let it shine!
kotlin simulates the “reduction” functionality of an optional type with its rigid nullable types system and its type inference system paired with things like the elvis operator
We had exactly this a week ago. A nil dereference made a Go microservice crash and restart infinitely. There was a cascading failure and the whole prod went down.
Going from a raised type to its underlying value isn't a lift operation, but you were right calling it a catamorphism. It's duel an anamorphism, is going from a type to a raised type (e.g. functor, monad etc). A lift operation is related but its focused on transforming a value in a given context, this is most commonly called a `map`
Assuming you have an array datastructure that satisfies functor laws, a reduction algorithm of [T] -> T is a catamorphism where as something like T -> [T] would be an anamorphism. Additionally if you compose these together (Cata after Ana) its called a hylomorphism (e.g. Cata :: F A -> A, Ana :: A -> F A, Hylo :: Cata | Ana).
So if you had functions:
safeDivide :: int -> int -> Option # catamorphism
optionMap:: (int -> int) -> Option -> Option # lift operation
x = safeDivide 2 0 # results in empty Option a catamorphism from int -> Option (assuming curried)
y = optionMap ((+) 1) x # lifts increment operation into Option context to be applied against the inner value (if not null in this case)
he be speaking the language of god
Watching the Primeagen makes me understand how much more there is to learn. Thank god. Could you imagine how boring life would be if you already knew everything and ran out of life material to feed your brain?
Rust may not have null pointers, but it has string[index] which panics if your index is not on a char boundary. Something you can easily fuck up and end up with a panic during runtime. Every language has at least a few pitfalls you just gotta stumble into and experience first.
is this in cases where UTF8 strings might include multi byte characters?
here is the JDA lord. thanks for your work haha
@@y00t00b3rI assume a string represents an array of chars in an encoding and indexing takes care of it? Not sure
@y00t00b3r yes, if you take a slice of a multi byte character not on a char boundary, there will be a run time panic. Note that directly indexing into a string is not allowed, only slices can be taken.
Taking a slice of string is not recommended. You can get around this by using .get and handling the case when it’s not on a char boundary, or more preferably being explicit with what you want with .chars to get an iterative of chars or .bytes to get an iterative of bytes, neither of which can fail during runtime
@@muddycalendar3292 unexpectedly good reply!
The null pattern adds the concept to define a default behavior
Great content, this shadow traffic concept you explained is really interesting.
This same problem also happened to me and I also took down production.
TL;DR: Every goroutine in go needs to run with a recover. Sometimes the package that you're using already does that for you, but if it don't, you have to do by yourself.
Correcting the chat: a panic inside a goroutine, indeed break the entire application and not only the goroutine where it happened. The problem is that usually (at least for my job) we are building http services and if even if you use the standard http library from go, it already comes with a recover mechanism. Each request becomes a goroutine and for every single one of them, the http package makes it safe from panicking.
The problem is that if your code is already running inside that "request specific goroutine" and from there you start another goroutine, you have to explicitly use the recover keyword for that new goroutine. If you don't and your code panics, the entire program crashes.
I don't know why I'm entertained by this channel. Half the time I don't know what he's talking about. I guess I feed the occasional nuggets of cool insights he puts out.
Same. I don't know how to code in any language yet. Having ChatGPT open and pausing to ask it to define terms I'm not familiar with helps
learnt shadow trafficking today :)
I'm a big fan of TypeScript, but it's important to acknowledge that it can sometimes give a false sense of security. For instance, when you declare an array of numbers and use the array's sort method, you might expect the elements to be sorted numerically by default, considering it's an array of numbers. However, the default behavior of arrays in JavaScript is to sort the elements by converting them to strings and ordering them lexicographically, not numerically. This highlights the importance of using a linter tool like ESLint to help catch and prevent these subtle, potentially confusing behaviors.
After learning and liking Rust's Option and Result, I began using std::optional in my C++ projects and it's vastly simplified my code.
One thing I dislike about std::optional is the implicit padding from that lone bool. I want an optional that can be grouped in memory as bool[8] followed by the payload for those 8 variables.
@@JuusoAlasuutari That's fair. There is definitely a cost to using std::optional or std::expected.
For my frequently called functions, I still choose another strategy.
@@ISKLEMMI it wouldn't be exceedingly hard to write an "optional_set" or similar actually, if anyone feels like implementing it ;)
@@JuusoAlasuutari Seems like something that would require polymorphism, which would break if you returned optional_set by value. Not certain how you would achieve that without some additional flag byte. ¯\_(ツ)_/¯
Zod is the GOAT for TypeScript validation. Provides runtime checks *and* compile-time typings with a single definition.
I think the Optional type is the most significant missing feature in Go.
Careful with default values in relational databases, as adding a new column with a default value will update every row and may cause locking problems.
The irony of yeeting someone into a bottomless pit and hitting em with the catch-phrase "don't let the ground hit you"
8:40 Option or Optional is a functional programming thing, even Java has it since Java 8 (the version that included functional programming paradigm in Java) and it's often used inside Spring library or with Stream functions, in Java has 2 functions, isPresent and get, the first one checks whatever has a value, and the second one gets it (with all the risks related to it), intellij tells you as warning that get without if(isPresent) is a bad practice
There is ifPresentOrElse which allow me to handle the error on point and there are functions to assign a default value if the optional is empty. There are nice functions for a lot of use cases and I have been getting a lot less NullPointerExceptions.
but these are a bit weird when you just see them in these languages. i learned it first with Rust and then transferred the ideas to java. i tried to explain Options and how nice they are to many people but they don't really get it
@@nisonatic If you use Optional.ofNullable(value) the optional itself cannot be null
This is not a programming problem. That company gave an inexperienced team the responsibility of putting a critical app in production, using a language they weren't comfortable on. They launched the app without reviewing the code, and without enough QA. Why couldn't they fallback to the old service? All the question that comest to mind are related to their processes, not their programming skills.
The option type thing just sounds like the `return value, err` pattern in Go. If you're not checking `if err != nil` then you're going to have problems anyway.
So glad people are finding out about Monads now. Would've been better if they caught on in like 1974, but hey, everyone learns at their own speed I guess
Write and use your own Optional type and use it until it's a built-in. It'll work wonderfully, and with a little bit of work you can also get it to work when en/decoding. I also wrote (RW)Mutex wrappers similar to Rust, which have been wonderful.
Panic returns to the calling function recursively until its handled if not it reaches main and kills the app. Doesn't automatically kill the app as soon as you panic
It's scary how many companies don't ask "should we even being doing this". Read it three times.
Every time that i see some “problem” related to go, usually is more related to the skills of the programmer. So, they probably don’t know about pointer/error checking, neither recovering. I admit, they do have balls to ship something to production without knowing how it’s working (and without test, probably)
That's why Rust is infinitely times supperior to golang. If it compiles, it's most probably ok. If it does not compile, then there is a skill issue.
@@svetlinzarev3453Well, rust is safer but that doesnt equal superior. You know, choosing the right tool for the right job. While I dont like go's nil pointers (altough there is tooling to catch these errors in the ide) this story has so many red flags that we cannot blame the whole thing on the language.
Ps: I love both rust and go
@@svetlinzarev3453 I can make Rust code that has expects everywhere, it will compile just fine but will crash in production
No amount of safeguards will prevent you from shooting yourself in the foot if you are persistent enough
This is why I like Salesforce development - you really can't take down the server via any normal means. And writing code to delete the database or even a large table is not easy to do, so that probably wouldn't happen by mistake either.
Using try / catch helps to:
- skip a couple stack frames to handle errors (sometimes)
- find where errors come from (not)
- accidentally "handle" errors that should never occur (skill issue) (still a bug)
Lift operations aren't catamorphic. Catamorphic functions are things that collapse data structures, e.g. fold.
I don't understand how it can be claimed that an option type would fix this situation. If they fail to check the returned value they will still have an issue. In Java... they call rv.get() and they had "none" then it throws. And if they didn't bother to check the return value for null why would we assume they would have written the exception handling properly? The dereference isn't the problem. It is failure to do proper error checking.
That sort of user info cache was also the first software dev task I ever did.
The cut to the HR office made me spit out my coffee
java also have that Optional thing, I recognized it from there.
I have that book. It's amazing. I recommend it highly!
Hi Prime
pretty naive question here:
where I can learn about Architecture Patterns and those things like: canary builds | shadow traffic and those type of thing aside from goggling those stuffs when I hear them.
can you suggest book, course or anything like that.
would be very thankfully happy for your answer
Prime never shows up here.
Prove me wrong.
@@y00t00b3r it's an open question to anyone not prime only (although his suggestion means lot to me).
and I didn't know that.
Programing sub reddits might be a better place to ask this question. There are a bunch of great books that cover these topics from O'Reilly and Manning.
@@H-Root Honestly, just lurking in Prime's chat on Twitch might get you a long way.
To have an entire app crash in production because of a nullpointerexception means:
1. you used a value where you should have checked for null without checking.
2. whenever that happens your entire app crashes.
3. Either nobody checked your code or they didn't see it either.
4. It wasn't tested sufficiently.
Point 1 and 3 I understand. People make mistakes and a nullcheck is easily forgotten, especially if you're new to it. But where was the testing in all this? Where are the guardrails to get your app back on track when something goes wrong? Are you telling me that every exception or problem just kills the entire app? This doesn't seem like a GO issue. This seems like either a process issue or a skill issue.
5:40 "This whole `try...catch` business" - you are missing the point about `try....catch` vs `error` in Go. Exceptions BUBBLE, you don't have to try-catch-rethrow on every call-stack level. Also, all `async/await` errors, even though calls are asynchronous, are properly handled with regular try/catch block that wraps them. So text author is right. One try/catch at the top level would keep service alive. Would it help customers, that's another question that depends on the service implementation.
To be fair one defer recover inside goroutine would also keep the service alive
What with all the accent marks?
@@samwilde8311 That is Markdown annotation for code. So that it is clear that I don't ask reader to "try" something, but that I refer to a `try` reserved keyword in a program. :)
This is a comedy of errors, but let's blame Golang instead of the mistakes made at every step.
It still never would have happened with sum types.
Experienced with Go, then rethinks the entire experience upon hitting a nil pointer 😂
seems not that much experienced right right? lol its literally the first fuk up someone do in go
Shadow traffic is great, but won’t help if your flag is turned off.
This is why feature flags can also be a huge problem if not used correctly.
You put a flag into the codebase, in an “off” state, merge code to main, deploy, and then don’t switch the flag on until weeks later.
Meanwhile further commits and releases have happened, which version do you roll back to?
Turning the feature flag “off” again may solve the problem in the short term, but how do you know for sure which commit caused it?
Sounds like a combination of things that didn’t go right here.
Sounds a lot like skillium and cope issues
How would you deal with side effects in shadow traffic scenario? Do you go all-functional and do managed side-effects? Doesn’t seem feasible with the whole “rewrite of an old imperatively written service”
It doesn't work in all situations. You can clone the DB so the shadow gets its own data store, if your calling out to other external services that have side effects it's probably not a good fit for shadow traffic.
Exactly. You need to have really thought through your "is shadowing" env flags or whatever is best to mock up or turn off or otherwise handle all of those things the best way you can engineer. The more complex the system, lots of microservices lots of third party apis, bockchains, emails, sms, S3, kafka, nats, stream processing, billing and payments, lord knows what, your shadowing can really get you into trouble if you have not engineered your shadowing for all of that ( looks like our shadow double billed everyone and deployed 50k redundant smart contracts ). Crud app that cruds to a DB? Shadowing is a simple and great choice.
Even Nim has options, and it was made in 2005
Options are so good
Dunno about option/optional. Not providing a value is not in every context the same as providing an empty value (normalized empty value). When do you know that the empty value is intended or no value was provided? Providing an unsuitable empty value can lead to wrong inferences when doing statistics on the data. E.g. you don't provide a price for a good but set it per default to zero. When someone else naively calculates average and median prices. Boom.
I gave up on Java exceptions ages ago. I ended up writing a whole try-catch around the entire app and restarting it when it blew up. It technically never crashed. It just restarted instead of crashing. Everything in Java is an exception including the stupidest stuff you'd never think in a million years would be an exception. The best solution is to catch and ignore all exceptions and just restart the Java app. Problem solved.
an "uncaught" panic in a goroutine brings down the whole app. Frameworks like Fiber etc provide "recover" middleware to catch them on the request goroutines themselves which is nice but if you were to say create another goroutine within it and make an uncaught panic there everything will die. ie. go func() { panic("oops") }() inside a handler.
Anything having a nil zero value is a pain, I'd rather opt-in to it if it's a nillable type. My biggest gripe with maps though is they are not thread-safe for writing OR READING, had quiet a few data cache corruption issues (not crashes, just plain wrong data coming out) from people forgetting that. Don't put maps in a RWLock either, only one reader at a time ever or bad things will eventually happen.
why are builtin maps not safe for concurrent reading? does reading modify internal state?
@@teodormaxim5033 I believe it's due to how it does the key lookup internally. It doesn't like multiple happening on the map at once. I've seen maps end up in a state where multiple keys end up having the same value in memory, the original values lost to the wind after only reads/lookups, not great for a thumbnail cache let's say (last time I saw it was a map of string to []byte for thumbnails a front-end was requesting like 20 at a time for a webpage when it loaded).
It's a shame that Go only has one builtin map type and it's very not thread-safe. It is very efficient though as a tradeoff afaik.
The tldr, which the docs do mention but I don't think quite stress hard enough is always access a map via a mutex for concurrent access, regardless of what you're doing with it.
Once you've got the value out of the map (even if it's a map of pointers / slices etc) you can unlock it and do whatever you want with the value without having a lock and be thread-safe, the issue seems to be the mapping of the keys to the values rather than the values themselves moving around or changing underneath you.
There's also an issue with slices I think and how some web frameworks work (thinking Fiber), so pulling a slice out of a map and throwing it straight into a response without cloning I think can end badly due to some pointer cascading.
There's quite a few annoying caveats with slices now and then depending on how they were made but if in doubt, clone.
Note, Kotlin is null aware, it doesn't mean you can't run into a null pointer provider from the client json request, it will throw NullPointerException at run time even if declared as non-nullable. Code example
@Serializable
data class Project(val name: String)
fun main() {
val dataSentByClient = mapOf("name" to null)
val gson = Gson()
val serializedJson = gson.toJson(dataSentByClient)
val parsedObject = gson.fromJson(serializedJson, Project::class.java)
println(parsedObject.name.uppercase())
}
While this is a fair point, it's also pertinent to point out that the source of that null value is a Java library: null-checks on platform types (e.g. values coming from Java code) are relaxed so that you aren't forced to elvis every single value you get from Java.
The unexpected NullPointerException is a consequence of the limitations of the Java type system and the convenience that Kotlin provides for interacting with it. You'll be hard-pressed to get a null from pure Kotlin code without shenanigans.
There is unfortunately no way to make the compiler enforce this, but you could in principle follow a convention of always assigning platform-typed values to variables of nullable type. IIRC at one point the Kotlin compiler forced you to !! every platform-typed value, but this turned out to be impractical so it was dropped.
6:00 100% disagree. The set of errors of, let say C, is greater than in Java just due to what the language allows: certain kind of errors cannot exist in some languages due to their design (this does not mean that "there are no errors in X language")
Is Rust's Option same with Optional in Java?
Yes. That’s what he’s talking about.
So like Optional etc
>Building a microservice which returns data from in-memory cache
Sounds like typical overengineered bs.
RDBMS already have caching mechanism like shared buffers, also you can scale RDB for read-heady traffics by just adding read replicas as much as you like.
Dealing with nils in go, is mostly the same problem as with JS, if anything is nilable, either make sure, we have a Builder or Factory populating that value, or check for nil in code, or in the database. The enforcement has to come from somewhere.
Generally while migrating old code, db changes aren't always possible. So better to handle nil explicitly.
Sad that we mostly have typescript programmers these days.
null and option are really the same thing. The only difference is when the language doesn't make you check for null explicitly before trying to read the value. They generally check it implicitly, and then throw an exception rather than letting you access garbage, it just needs to be enforced explicit. You can do this with either null or option, but need language support.
So, obviously, they are NOT the same thing, since with Option you MUST check if a value exists before using it, i.e. Option gives you strict guarantees; while null shifts all responsibility onto you. It's opposite things, not the same.
@@AlexanderBorshakI think he’s referring to null in null-safe languages like Kotlin, where null is truly just syntax sugar for Option. Kotlin’s “!!” operator is exactly the same as “.unwrap()”, “?:” == “.unwrap_or”, and “?.” lets you call a method (or read a property) if and only if the value is not null. By default, you can’t do anything else with a value of type “T?”
I like his emotions on the previews XD
That's what's know as hypochondria - panicking about nothing :D
I love Zig for how it chooses to handle null values and errors. Putting it into my hands so I can control what happens instead of doing the usual style of try-catch (I am glad I am not the only one who hates those, wasn't able to really verbalize why until using Zig). It all just becomes directed behavior instead of relying on how they choose for errors and such to be handle behind the scenes. It does require you to be conscious of even more engineering, but if you are using Zig, you specifically signed up for that.
you can shadow traffic if you service does "side effects"
I think go is a little bit worse than Kotlin in terms of footguns. For sure. Go seems to not care about null safety at all, were as Kotlin obviously at least tries to do something about it.
if your pointers have got nil, then it is a skill issue for not checking the error of the function that returned nil to the pointer.
if I ever write a programming language I'm going to call my option type null and the none type also called null, and you access it with the following statement: if null is not null...
Should have used Erlang. IDK what the article is about, but Erlang good.
Whaaaat? Coffee scrip used to ve pretty mainstream and they had nullables. C# had them pretty much from the start
C# had nullable value types from the start, that is a different thing. Nullability analysis for reference types came only in 2019
Lets gooo
Browsing r/golang so we don't have to and preserve our sanity, I salute your sacrifice
shadow + canary deployments. this is the way
Common Go footgun. Every pointer dereference has to be defensive every time. Also have to consider these types of inputs with integration tests.
That's a 100k training.
You have to recover goroutine panics or it'll bubble up and kill the entire app, from my experience. Has that improved?
You obviously NEED to handle your goroutines...
support must have been nuts
Why switch to Go when you already have a good modern language like kotlin and devs experienced in using it? I doubt you would gain any significant performance gains as Netty is quite fast and you can easily write non blocking code with kotlins coroutines. Startup times shouldn’t be a big concern as they are loading a bunch of data on startup as well, and if it is there are other ways of optimizing it rather than switching stack entirely. Go uses less memory but that by itself isn’t a big deal. I don’t see any language features in Go that would warrant switching either.
Imo its shortsighted to just blame it on skill issue and be done with it. In any given team very few developers will be very skillful and others will have skill issues (often because they do not care about their job, its just a job for them). Good languages will not take your entire product down just because some junior had a skill issue. Good languages nudge you towards writing readable and performant code just by default. Kotlin is great because it does it, specifically with nulls, often i saw noobs who would fix their own prs becuase the code looked "ugly", too many question marks, so they tried to fix it and actually wrote a null safe clean code that also gets chewed up into well optimised machine code by jvm... thats the whole point, you write normal code but it gets compiled into some mad C developer super optimised magic without your knowledge.
Kotlin rocks, and with kotlin native/cross platform imo jetbrains is well on its way to totally domainate language and tooling business in the years to come.
I often find "skill issues" means the dev doesn't know the gotchas.
All languages have them and the less they have the better. The default way of doing stuff should be safe where at all possible.
@@101Mant yes and thats why java and jvm was so dominant all these years.
Languages that are capable to unlock all hardware performance + be maintainable with low barrier of entry are the most dominant.
Only exception i can think of is JS but it has a good reason why it got popular.
It was a skill AND process/testing issue. They didn't test the code with null values if they did they would have found the error.
@@Patmorgan235Us i agree with lack of testing/review process argument, skill issue imo is not avoidable, very rare companies/products can afford only hiring seniors. Most companies will have more juniors than seniors. So you want to minimise possible issues that depend on skill. Ideally you should be able to take someone fresh from bootcamp and get them to be a useful contributor. So skill issue is not an argument here (unless the person fucking up is actually a well paid senior)...
alternative: swift, kotlin, dart, rust.
btw in Python, you can do str | None (Optional[str] in older versions). Checkmate, gophers!
btw in Go, you can do reference | nil (and the language actually knows the difference). "panic: illegal move: Kg9: index out of range [8] with length 8 ", pythonistas!
+1 to gophers, screw python@@0dsteel
Btw in Rust, you can rewrite everything in Rust!
@@alang.2054 you got me :(
"can" is the problem, right? Because the word should be "must" and "checked by the compiler". And since typing is not required and code is not AOT compiled, Python is a whole other level of awfulness.
btw I use -arch- Kotlin
Go is fantastic, especially in the hands of former C programmers.
But Go does seem particularly vulnerable to onboarding Java developers. Not suggesting they are inherently evil, but they do have a "different" way of looking at systems, and that thinking - when applied to Go, ends up In a horrible mess of pseudo-Java abstractions more often than not.
The worse thing that ever happened to Go was when it got "popular", a lot of experienced Java devs jumped in, and steered it in a terribly wrong direction. Now we have Go with a broken java-style implementation of generics, which is kinda messed up. It would have been far more advantageous if the Go team ignored the pleas of the new Java devs, and instead concentrated on - optional manual memory management + better error handling options to tune it towards low level systems programming.
Ah well, it's a bit late now. We are stuck with a Go that is commonly overlooked for a few reasons.
Is the option type similar to result type in Kotlin?
No, Option is the same as Kotlin's explicit nullability. Therefore, "Option" is Kotlin's "String?".
That was a bit pre-mature live production testing.
i just watched this video right after Go Panic recovery 🤣.
"Every function that has await can throw." EVERY function call can throw, whether or not it's called with await. Or do you mean even when analyzing the body and it has no way to throw an async call can throw?
I am disappointed you didn't end with "The name is.... theShadowTrafficagen"
Options are non-optional!
Why did not they use recover instead of crashing whole service?