How Should I Implement This Backend?
Вставка
- Опубліковано 27 вер 2024
- Here I need your input on my project! 😃
⚡️Checkout Miro here:
miro.com/yt/ba...
Follow me on instagram: @withmarko
Hey, I made a discord server: / discord
=== Links ===
☕️ My Notion Template:
links.withmark...
☕️ Wallpapers:
withmarko.com/...
☕️ Mac Bluetooth Toggle:
gist.github.co...
☕️ My e-Ink Tablet:
links.withmark...
☕️ My phone:
links.withmark...
☕️ My Free Window Manager:
links.withmark...
☕️ My Productivity Mouse:
links.withmark...
☕️ My Productivity Keyboard:
links.withmark...
☕️ My Headphones:
links.withmark...
☕️ My Microphone Setup
links.withmark...
links.withmark...
links.withmark...
☕️ My Camera
links.withmark...
links.withmark...
=== Business Inquiries ===
Email: business@withmarko.com
Instagram: @withmarko
=== Affiliate Disclaimer ===
I might earn a small commission at no cost to you, if you click on the product links above.
As a intern seeing a senior dev opening and talking about the perks of a personal project give so much insight about how to view things in software development.
Amazing content, you ernead a subscriber
I was about to say the same thing!
@@anonforever123 maybe there are some steps between but off course you can. Nobody is born a senior in anything my friend
My personal advice: go with the thing you know the most and let you to develop faster to get an MVP as soon(and cheap) as possible. Then if the idea works, you can improve it as the business grows. In my personal opinion, if the project is intended to become a business it is the best to think in how you could test your idea and then the engineering stuff we all loved. I am working on a personal digital business too and I started to think in the same stuff(language, platform, etc) then I realized I wanna to have a customer testable version as soon as possible and I went to node/express and heroku and a well-organized monolith (think I can develop faster) instead of my initial idea of microservices, aws and golang
Underrated comment. One of the best I've read, far.
Exactly build the basic and optimise it later
Ha I literally just wrote that actually that’s hilarious
That's right. Though on my day work we have our super big 11 years old monolith project with a lot of legacy, we still are able to move some parts of it to microservices all the time, update codebase etc. For the start - the main purpose is make testable product for you potential clients. Your project won't likely be that enormous and difficult through first couple of years, so that you would not be able to change anything in architecture. As you hire bigger and more experienced team in the scope you are willing to implement the architecture in, you will be able to scale and navigate product development in any direction you want
Hello Marko!! Just a quick life update, I’ve been following you from the start and you inspired me to get into coding. Tomorrow, I graduate from college with a Computer Science degree! I also just got MacBook (my first one! I wanted it for coding) for my birthday on August 4th! I am starting my personal coding journey now since I am finally done with school and you are a BIG reason why I got involved in this community. So for that, thank you sir! You’re the man! Keep up the awesome work 😊
Congrats!
Congratulations !! I just graduated 3 days ago and got my fist Mac!! Let's go🙌 you got this! We got this!
@@vloginnwithekom8326 yeah we do!!
I am very happy to read this message and I congratulate you
Congratulations! I also hope to graduate from college in January 2024, insha'Allah.
I was also considering between Go and Rust for my current startup and I decided to use Go. I absolutely agree about all the reasons that you mentioned and my experience has been great with Go so far. It's extremely easy to start with and it gives you ability to write production ready code on week two. Rust is great when you need some of it's low level functionality but I feel it's an overhead in 90% of all cases, especially if we're talking about backend development.
That being said, Go has a great ecosystem and an amazing standard library that allows you to focus on your product instead of fighting with the compiler itself. I know, the Rust borrow checked is supposed to help in a long run but the question is if you're ready to sacrifice 50% of your development performance for 10% of execution speed/memory.
Hey thanks for sharing your experience. Yeah, I'm leaning towards go myself. Am excited to dive deeper in one of the next episodes! Is your startup live, btw? 😃
@@withmarko i think go is good but things become messy once the project is bigger
i built a project with it before and then things become pain
so i actually switched to node / express and it was an easier time for me tbh
We are using go in our Startup as well and so far we are pretty happy. With the right structure the code is maintainable and easy to understand. We are also using the microservice architecture, because we have a large user base. And that is working for us pretty well.
I decided TS (nestjs)
Hello Mark, these chill videos are the best (watching at 3:00am)😂You never dissappoint
Hey thank you guys, this puts a smile on my face! 😊
1:46 🇲🇽
Lol ! I'm pretty close to 3:00am . These video just popped up.
3:33 x)
For deployment, you could start with docker-compose which is great for local development as well as self-hosting. Then later can use the Kompose tool to quickly start using K8s for larger prod deployments.
K8s is very complicated lol. I’d suggest he uses Cloud Run because the containers scale up and down based on traffic.
Kubernitis is a disease that is slowly strangling our industry. Just last week we had a terrible experience in production - i would not recommend this to anyone i like.
@@tobyzieglerrr Yes, it is like a nightmare. I always encourage people to avoid using this garbage.
oh my goosh.. i TRULY dont know ANYONE else here in YT with such a good chilly Vlog Vibe.. KEEP GOOING !!!
I love those personal projects. Its stressfull but also super fun to deal with all these decision on technology, to make sure you comply with gdpr, to do the system design etc. Its so much fun
Just as a couple others have said, rust is designed to be memory safe. The unwrap function isn't the intended way to get the value from the result - instead use the match statement to handle what happens that unwrap fails or not.
Also regarding gradlie, have you seen the new kotlin dsl for gradle? Worth having a look at and brings gradle more up to modern standards.
Great video as always marko!
the rain behind during the video voice recording is just 🥰
sooooo relaxing and really good video
I enjoyed your run through of each of the languages from your perspective.
How about using C#? Similar syntax and feature set compared to Java but you get nullable referencing in newer versions (to help with catching NREs at compile time).
You may be possible to implement your entire back end using PAAS or managed cloud services (such as Azure with Event Hub or AWS managed Kafka).
Hey that's a great point that C# has seen a bunch of improvements recently. I will have to check it out. As far as using PAAS, I'm considering a self-hostable alternative 😃
@@withmarkoC# also allows for the same kind of query syntax that you pointed out as nice in Rust. Switching from Java to C# is easy peasy.
And C# is faster than Go...
can u explain?
@@Parzival-ht6ey
@@TheFeljoy LINQ yeah
My only advise would be: Don’t try to make these difficult decisions all at once, go with whatever you think is the easiest and replace stuff when necessary.
I can't count how many projects I started exactly like this.
I spent hours trying to learn which language, framework, database, architecture, hosting service etc. Just to end up giving up before even finishing an mvp.
@@elan2199 you may think there could've been a better way but trust me, all options will give you headache at some point, some of the more so than others, that's true...
Yeah, I thought the same thing. It sounds over-engineered for a new project (Unless you are a youtuber wanting to make content) But if you want to seriously build something, just build.
I thought the same thing, yeah. It is like, put to much effort to bring up technologies that may be aren't aligned with the company, for example: Who will even maintain a rust application if it is not used widely in the company? I thought if it is started simpler things could be replaced if needed. Find specialized developers is hard if you thing about business.
Hey Marko! About the infra part of the product I think that the easiest way here is to go serverless. In AWS you have to the define the subnet where your lambda will be executed, and this way you can restrict to a specific availability zone to avoid GDPR issues and still be very scalable.
The DNS can be solved with Route53 regional rules that depending from where the address of your service is requested it will be solved to a different API gateway (either in Europe or outside Europe).
Regarding HTTPS you can even use ACM to sign the certificate and attach it to your API Gateway, and use cloudfront for caching the results.
I would also go here with DynamoDB for scalability and ease of management. Also the price is super low when compared to other DBs and if the product grows a lot you can even make the tables global to decrease your time to info!
I would love more Notion based setup videos. It's crazy good!
How beautiful is Norway. I hope some day I'll visit the country.
Norway! Awesome! Holland was similar but has been getting more relaxed with some stores now staying open later. Most things close at 5, some now 6. There’s a “sales-evening” once a week, Thursday in big cities and Friday in smaller cities, where shops stay open until 9. Grocery stores close anywhere between 6 and 10. And some other shops also stay open until 10. I don’t understand some of the logic here, but I guess it works.
Every language has pros and cons, maybe the best thing to do is considering all the libraries which will make your project easier. If your project is Kafka-heavy then go with Java, or Scala. Almost every language supports Kafka as a message broker, with consumers and producers, but if you want to do many stateful operations, the kafka-streams library could save your day, and it's only Java (there are some ports around, but the java library is developed by the same team around Kafka).
Considering the first two pieces of your architecture, the catch-all kafka writer and the ingress, I think that kafka-streams could be a good fit. For example, if you plan to implement rate-limiting based on messages, you could use kafka-streams time aggregations with just a couple lines of code. Moreover, kafka-streams offers you local storage which works like an embedded database on the services, and for persistent backups you could use the Kafka Connect API from a kafka topic into almost any persistence layer.
Your videos are outstanding, Marko!
Cheers from Brazil!
Hey Eduardo, nice to see a familiar face in the comments! 😃 Thank you my friend 🙌
Hey! Love your videos. As a web dev myself I am faced with issues relating to gdpr a lot. I had discussions and talks at length with lawyers and people who should know what they are doing but unfortunately because there are barely if any legal decisions/cases and the actual legal texts are very confusing in the ways they can be interpreted I would argue it's wrong to say google analytics is flat out "illegal" in the EU.
Almost every big tech company nowadays is based in the US so even using things like Microsoft Office which is perpetually online now, any browser, etc would be "illegal" in the EU because they transmit data to the US. There are a few regulations like the "standard clauses" which handle these cases.
I've been immersed in this topic a lot but unfortunately the tldr of the situation is, that it's just a big gray area with a lot of unkowns as of today.
Unless we have actual cases that have decisions made by judges, there is no real "right" way atm. just a "probably correct, but it might also be wrong"-way...
Hi Marko! Thanks for your videos, they are really nice and relaxing. About the macros in Rust, you never need to use if doesn't feel comfortable with it. It's just a feature, nice to have, for the once wanted to use. Like in C, you had macros too, not as powerful as Rust (but that's why being developed Rust), and write code for decades without using them. Regards and keep going!
Dude your videos are epic so is the city .
At my company we have similar requirements to your project. It's self hostable, it's GDPR compliant and privacy focused. What we do is, we have a docker file for each component such that we can host the entire infrastructure on kuberenetes for our SaaS solution but we also provide a "all in one" image for selfhosters if they just want to get something up and running.
Could be an idea for you as well? You could just start out with one image that encapsulates all of your components and once you need to scale you can figure out kuberenets.
Rust's .unwrap may feel clunky, but only because it is an anti pattern, .unwrap is supposed to be used only when you are 100% sure that it will not fail.
Otherwise you should be using pattern matching, which is similar with go in the sense that you check for every possible error
Really nice format of comparison
6:31 Have similar syntax with C# and LINQ with EF CORE and have good performance. Best of two worlds.
Your videos are so clean. Love it!
hey @Marko whatever place you are at, seems to be pretty cool to live and work. Also i like the way you detailed out the thing in this video.
I would not underestimate Scala. It is that type of language "Once you master it, you will never want anything else". BTW Just visiting Oslo and falling in love with the city Norway overall :-)
Points to consider 1) Its a personal project so I think not more than 2-3 people will work on this hence code maintainability will not be a bigger issue hence complexity of the language will not have that much of a impact 2) Since it will be a data analytics software therefore runtime minimization should be a major priority but I think that also depends upon the dataset, So you should have an idea about the size of the data so that can be a tradeoff with the language complexity
I like your videos, the sound, audio, shots, editing, everything is best, 😊
This is the perfect video! I was thinking about writing my own in-memory database like Redis, but I got stuck at choosing a language. I eventually went with Golang. I guess it's a safe bet to go with Golang - best in both worlds
Hey thank you so much, I'm glad you like it! 😃 Yeah I am leaning towards golang for sure these days, it's got a lot of great things going for it, and I love how simple and minimal the language is! Do you have a link to share with your project, it would be exciting to check it out! 😃
I feel your pain with the build system, but I'd still seriously consider Kotlin for your use case. It's the best of JVM.
what about python with FastAPI?
Hi Marko, this recommendation I'm about to give may not be perfect but I believe it is still pretty good none the less (dunno how to spell it right :P). Anyways, since I see from the video that you really like and need good request paths handling and paths defining I highly recommend Nestjs (backend TypeScript framework) which I know "It's TypeScript which means it's slow and yada yada yada" but lemme tell you it's actually quite fast for a TypeScript framework (especially recently since they made a new feature which makes Nest 20x faster than before, still experimental). It is pretty easy to pick up and maintain the codebase. Also, Nest is made for big projects with a big amount of data flow. It is heavily reliant on dependency injection which is a double edged sword but in my experience I had no problems with it at all in Nest. When it comes to hosting honestly it really depends on you situation. If you need a server on demand with a specific amount speed and reliability then serverless is the way go to but don't take me seriously with this last one cause i still need to learn a thing or two about it.
I hope I helped with your decision and good luck with the project :)
for API in Rust, try `axum` rather than `warp` if you want to see "lesser" macro in Rust
Honestly Mark, you should rather use typescript with some sql modules. Kafka is more for global and complex data analytics. I think that you should try scala as an option but try using some data frame in TS (like pandas in python).
bro, i always watch your videos, and i know that i will learn something from you
Bun is out! It might be the perfect balance for using typescript and getting performance improvements
The negative earnings with the laugh took me out. Such great content to start the day with :)
This project would be great with Elixir as the backend and pheonix live view for your dashboard
😂😂😂😂
Hey Marko, if you are planning to go serverless I would stick to your language of choice, which I think is TypeSscript? 😅
In cloud you can upscale instances on demand. From the start high load is not expected and over time you can upscale instances as needed with tools that cloud provides, so slower Node runtime compared to Go or Rust will not make much of a difference. In the future if Node will be the bottleneck (which is kind of unlikely), rewriting the codebase to different, more performant language will take fraction of the time of initial development.
Few months ago we deployed task scheduling microservices for FinTech company, around 7 cloud functions with Nest.js + TypeScript and we didn't see any performance bottleneck even in high stress testing, after few optimizations. Depending on task configuration it can be quite computation heavy.
If your project is about learning new language tho, I would Go with Go 😆, specifically with Buffalo which is sorf of alternative to Nestjs, because battery included frameworks can speed up development.
At last, love your videos ♥, keep up the good work!
Hey thanks for sharing your experience with js/typescript in production, that does sound very reassuring. And you are absolutely right, in the beginning the performance will absolutely not be of concern. I am not gonna lie that I am excited to learn something new, so I am looking at go in much more detail these days. Thanks again for the amazing comment, and I'm really glad you like the videos! 😃
@@withmarko Great, looking forward to see your progress in the next video! About Go, when you have Java background, how do you like Go's single letter convention for receivers and overall variable naming?
Did you have contact with Java in new versions and spring boot? I think that they are much more fun to use than before
Yo I was waiting for this, keep it up bro.
Hello Markο
Your chill video are amazing!
Really like watching!
can you please share your career journey in detail. which language you learn first and where you learn from??
This video is so well made,how was i not subscribed
Great breakdown of the different choices Marko! Question for you - how do you prevent burnout with programming? As someone who has tried to build projects with similar complexity as this outside of my day job, I find myself fizzling out a few months in. What’s your secret?
If you rely on motivation, at some point that will run out. You must set a goal and be disciplined to achieve it.
We need more vlogs!😀
struct tags was also one thing i didn't like about go, but you'll get used to it
i feel like you should use go but that is just because im bias. keep up these videos marko!
What about PHP? It doesn't have the disadvantages now, that it used to be. It's fast, easy to learn, and there are great frameworks too (like Laravel). However, if you want asynchronous programs, PHP doesn't support asynchronous things out of the box. You can write async programs in it with some libraries but PHP is a synchronous language.
U can't say that PHP is fast bro
for the deployment do it in a local machine by an old dell optiplex and it will do the job
Great vlog, super authentic ... where did you get the beats used ?
No bs I’m literally learning about how to implement node js and blender right now in my bootcamp.
Give the bun ts/js runtime a try. You can stick to ts while getting nice performance.
Also regarding Kubernetes, try Rancher. Quite a lot of YAML pain goes away.
Don’t know if you gonna see this, but what about Python and Django ? It’s easy to use a database with it and you can build an API as well as a front end
From what little I understand about Rust, the unwrap thing is something people do to avoid error checking in tutorials and examples, whereas in production code you really should implement pattern matching (the match keyword I guess?) to handle errors and whatnot; unwrap just says "okay, this will not give us an error, just extract the value"
Anyway, I'm still very much a newbie in Rust so don't take my word for granted here
I agree with your assessment that Go's syntax for serialization is not great. Honestly, I'd say Go generally isn't very expressive or syntactically pleasing. It's chosen because it puts a "maximum level of complexity" on your development experience, in which almost nothing is super nice, but at least you're going to be spending a minimal amount of time debugging the language itself. However, this aspiration of not having to worry about the language sometimes falls short in high performance workloads that benefit from GC-tuning, and any situation with crashes due to improperly-used goroutines. But, overall, most of the code you write directly correlates to application functionality, even if you have to write way more code than in the equivalent Ruby codebase, for example.
Anyway, Go can be a very productive language. It's just not "beautiful".
One thing about Kafka: You have to be sure that with Kafka you have to put in a lot of work manually also still in production if you want to scale up. There is other choices like Azure Service Bus that take away all that workload but you have to pay for that then. Worth looking into that though, I think.
depends...If you use Kafka through a project like Spring Cloud Stream (using Java) then it's a breeze to use. The setup is very minimal
@@mario_luis_dev Thanks a lot for that! Sounds interesting. I will definitely have a look at that!
C# is the best and most elegant general-purpose programming language ever created. IMO
where do you get this background music, suits very well. Can u please share the link?
amazing stuff mark :)
Rust is actually not that bad. I love the file structure of cargo workspaces.
Sounds like a great use case for CQRS when speaking of backups but you already have that part down.
Language wise I would use Java with Quarkus which will make everything really really easy and maybe deploy to OpenShift if that is something you are comfortable with.
Hello mark love watching the videos I know you have probably answered this before but I really like your bag that I see in your day in the life videos and was wondering what bag it is keep up the good work!
Hi Marko, nice video as always
I’d suggest to dockerize your go binaries and deploy as AWS Fargate tasks.
Hey! Love your videos. Where I can get this wallpaper?
Have you had the chance to look at Elixir?
how do you learn all this? I don't understand any of it. all of this is so overwhelming. do people like work in jobs and learn this stuff by working on projects, or are they expected to know all these stuff before applying as a fresher? or they taught in college?
Elixir gen servers and supervisors could be a good fit for this
Rust is difficult, but can pay off in terms of performance or "good" code.
I don't like Go, it's like using Java in 2004 where they first got generics.
TypeScript is clunky as well for the backend in my opinion, but works pretty well with the frontend
Scala has a great standard library with collections. As well it uses some eDSL to create things like tapir or caliban, like you've mentioned with this HTTP DSL for Rust. Sometimes it's useful to embed your own DSL into the language to make things declarable. Also Scala has a effect system like ZIO/cats-effect which came from years of research in Haskell, if you put some time in to these topics you might become a better programmer.
Good luck!
Wow… that is impressive research!
wow learn a lot just listening to you talk
deployments to wide range of people is the hardest
I never seen people who complains about the tools they use more than programers, I mean for real most of these complaints are "I don't like it!"
Here’s the answer: Do what you know, not what you want to optimize. What you know will get you to market. Optimization is not a requirement actually, it’s an improvement and all of these are built with equally performant optims via additional tools. The user doesn’t care what that is. You’ll spend 98% of wasted time just playing with tools and 2% on building the tool only to discover your project evolving into something entirely new only weeks after anyway.
I was in Oslo a week ago also in this Café but i didnt See you
Regarding Google Analytics, you can make a GDPR complaint cookie banner, and enable the data collection only when consent is received. Otherwise, you keep it offline, and therefore, don't break any laws, isn't it?
Hey marko!
Why don't use Google Cloud Run, in the EU zone? (europe-north1in Finland)
please explain me this point:
google analytics is not GDPR compliant, and for this reason you are not able to use heroku ??
Sir, I am a long time subscriber, what is your vscode theme btw
Norway is really very beautiful.
Are you using light code editor😯
What wallpaper do you use?
Hi Marko, I love Rust but you should pick it only if you are into learning the language, not for delivering this product quickly. And about the video, I enjoyed it, but the parts where you wave your hands in front of the screen looked weird because you are clearly not talking in that moment. What do you think?
Hey thanks for the feedback, that makes a lot of sense regarding taking a long time with Rust. Haha, as far as the waiving of the hands, I did actually talk at the same time, if you look closely, whenever I speak there is a little green light in the background flashing, which is the volume indicator on my microphone audio interface. 😃
@@withmarko thanks for reading and commenting! Yeah, I see the mic. I don't know, maybe it's a personal preference of finding it a bit odd, but anyway I enjoyed your video, keep going!
how do you like c# for the backend
Have you considered NestJS? I know is still TS but has some good principles.
Marko Cooodessss 😂
I wonder why "serverless" for you automatically means "edge"?
i'm relatively new in all those devops stuff and doing it only for a little for my own pet project, but as far as I understand at least aws serverless capabilities, you can deploy dynamodb (nosql) or aurora (sql) in the target region as your primary database and be gdpr compatible about data. then you can package your server as a serverless lambda and here you go fully serverless, cheap, scalable (with all the downsides of aws of course) and presumably good with gdpr... no?
besides it i'm curious about if you want to be infrastructure-as-a-code? and if so how can it be achieved with services like hasura? how can you simultaneously and together deploy db on hasura and server elsewhere?
Dokku is the way!
The 2 criticisms to Rust are a bit unfair, 1) unwrap should not be used for normal error handling. Rust is extremely safe if you play by the rules, and let you forget about a good chunk of runtime errors because it forces you to think about them. Unwrap means "yes, I thought about a possible error and I want my whole program to just crash if it occurs". Other programming languages do the same, but implicitly, probably not what you want. 2) I get the potential criticism about the flexibility using macros in theory, but it actually depends on library implementations and in practice I never found it to be a problem. There are so many high quality crates that should you find some cryptic ones I would suggest just avoid using them.
Backend should be done with Rust, frontend with htmx, and ide with neovim.
I respect the shotcalls, but I have to note that even working full time with neovim I still need an *actual* ide from time to time - especially for debugging and profiling, cause the experience is simply better in visual IDEs (shoutout to jetbrains).
Go + self-hosted ftw!
Rust + axum + sea-orm is a🥞
Don't use Gin use Fiber for as your Go framework, it uses a lot of expressjs ideas and is better performant than Gin is. Well done.
please I want to start in the field of back end and what is better in javascript or c# at the moment
c#
dude im watching this at 4am (Greece)
Hello, first I want to congratulate you on your channel, I am your fan. I wanted to ask you for some advice, I'm making an app in Ionic with Angular and I want to have a button that when clicked sends the signal to a microprocessor to open a door, I'm a new student and I'm just studying programming. Thank you and all the good vibes in the world to you 🙏🙏
Hi Marko,i am planning to buy a macbook for study and do you think macbook air m2 or macbook pro 16inch 2019?