@ArjanCodes there is an issue with pydantic that I only realized the fact that validators are classmethods does not mean one should use @classmethod if one does the @validator would not be executed this should be a big NO NO.
Im currently making a datavalidation script using pydantic. Combining custom constrained annotations with the dynamic model creation per variable works like a charm!
By utilizing `BeforeValidator`, `AfterValidator`, and `WrapValidator`, you can construct an Extract, Transform, and Load (ETL) pipeline directly within an 'Annotated' type hint. Additionally, you can achieve aggregated entity resolution by mapping known keys to a JSON field using `AliasChoices`, and serialize the parsed response within a single model from as many data sources as you need. Create trigger conditions using discriminated unions and fully automate processing procedures like OpenAI Vector Stores. 🐍🐍
I know I sound like a broken record in these comments but I really wish msgspec would gain the same traction. Does everything pydantic does but better.
Arjan, as a Dutchy but also as a human being: thank you for all of these videos, including this one. However, I am still looking for a video where you properly explain how to use the debug tools in an IDE properly rather than just printing everything and using "if self.debug=true: print(f"Beautiful variable in this method"{variable}"). Do you perhaps have something in the pipeline for this?
@wesleydevries7415 check out mCoding's recent video on debugging python in IDE's. I would specifically recommend using Pycharm for advanced debugging, it's unparalleled imho. Groeten!
If talking about python, why not to use built-in IDE debugger. For instance, in vs code you'll find debugger shipped with python microsoft extension. I believe it's there. Which is triggered via F5 key. If there's no option to run debugger or you prefer to use prints, logging can be the solution. The minimal logging level to print during runtime (debug, info, warning) can be set by its config. IMO the loguru lib is the easiest solution to use for logging
Wow 😲 my comment referring to a video from another youtuber (mCoding) that has an excellent introduction to IDE python debugging got removed (by you(r team), @ArjanCodes? That doesn't sound like you at all somehow...) Anyway, make sure to check that out! And use Pycharm, it's awesome 😎
Thanks arjan! Again a great video. I didn't knew about pydantic before and after watching this video, i will surely use this in my projects and make this an addon to my dev toolkit. Again thanks for this awesome knowledge! 😀
Thanks was having a problem upgrading from pydantic 1.9 to 2.8 seeminly some of the modules i wanted to use needed 2.8 - the headache was i could not understand why optional no longer worked - now i get it i had to specifically set the Field Value of all Optional Fields to None
I actually also miss doing those myself. We’re making a few changes at the moment to (partly) return to the style of the older videos. So stay tuned 😊.
Thank you Arjan ! I love pydantic with FatsApi, my only fault was putting too much business logic into it and not knowing where the data transformation was taking place :)
Pydantic would be better, if it would have 1) proper serialising and deserialising and b) If it would have a proper way to convert to and from snake_case and CamelCase for python/JSON stuff.
Hi Arjan I have a question regarding python processes. I am diving in to learn elixir. But I am wonderung if the similar functionality we can achieve from python? Especially all processes are like having individual dashboard to monitor the performance and self healing concept. Thanks in advance.
Hello, @ArjanCodes, can I ask you use, classmethod decorator in validator methods even if you don't need need class instance in code ("cls")?? Wouldn't it be better to use staticmethod?
Do you have an example of after creation there is a change in the value? To Validate or change the information. Example: set a new password in example2
Can you create your own types, like the EmailStr, so that they can be used in multiple classes? I feel like this would help reduce the massive class bloat for the final data container and help reusability. (Particularly my use case which would be laden with numpy fields!) Also, is pydantic still really slow when compared like for like against dataclasses and attrs?
That's what the BaseModel stands for, doesn't it ? You can create your own custom string type, or int type, or anything else you wish, and change every behaviour of the type (and also its validation, thanks to pydantic)
@@juanjosefarina ah so you'd basically nest types and it's solved recursively?? Ty! Definitely going to take a look and see how it runs on my embedded stuff. I'm using dataclasses at the moment which are nice and quick, even though I'm locked on 3.8 so no slots.
For some reason making compositions and accessing attributes from nested models just doesn’t seem to work the way it’s supposed to. I know my models work because I can query them in any other validation library.
With lots of models and some even nested ones pedantic is extremely slowing down our cold start time in the lambdas. Also with 3.12 + pydantic2 + fastapi and debug mode it sometimes takes up to a minute to start the application:/
It's very flawed, uv is a much better solution. Also poetry did some very questionable things with versioning. Like randomly raise an error (literally pick a random number and throw an exception instead of deprecation warning)
It's very flawed, uv is a much better solution. Also poetry did some very questionable things with versioning. Like randomly raise an error (literally pick a random number and throw an exception instead of deprecation warning)
If you use data from mongodb is there better way to create instances of pydatic classes then getting binary json as dictionary and dictionary unpack into pydantic instance? I looked into creating pandas dataframes with PyMongoArrow but I'm not sure I need that clunk.
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
@ArjanCodes there is an issue with pydantic that I only realized the fact that validators are classmethods does not mean one should use @classmethod if one does the @validator would not be executed this should be a big NO NO.
Im currently making a datavalidation script using pydantic. Combining custom constrained annotations with the dynamic model creation per variable works like a charm!
Pydantic is a lifesaver when dealing with messy/nested JSON data. I love it.
Does dataclass provide same functionality with json?
@@coladockuse sqlmodel. Thank me later
Totally agree!!
By utilizing `BeforeValidator`, `AfterValidator`, and `WrapValidator`, you can construct an Extract, Transform, and Load (ETL) pipeline directly within an 'Annotated' type hint. Additionally, you can achieve aggregated entity resolution by mapping known keys to a JSON field using `AliasChoices`, and serialize the parsed response within a single model from as many data sources as you need. Create trigger conditions using discriminated unions and fully automate processing procedures like OpenAI Vector Stores. 🐍🐍
Every time I see your video, I get new ideas about the code structure. Thanks a lot for your great job.
Thank you for the kind comment, Michael :)
I eagerly await the release of a new upload from Arjan every week. Never disappoints me.
pydantic is one of the best Python packages I've ever used.
I know I sound like a broken record in these comments but I really wish msgspec would gain the same traction. Does everything pydantic does but better.
Arjan, as a Dutchy but also as a human being: thank you for all of these videos, including this one.
However, I am still looking for a video where you properly explain how to use the debug tools in an IDE properly rather than just printing everything and using "if self.debug=true: print(f"Beautiful variable in this method"{variable}").
Do you perhaps have something in the pipeline for this?
@wesleydevries7415 check out mCoding's recent video on debugging python in IDE's. I would specifically recommend using Pycharm for advanced debugging, it's unparalleled imho. Groeten!
😊
If talking about python, why not to use built-in IDE debugger. For instance, in vs code you'll find debugger shipped with python microsoft extension. I believe it's there. Which is triggered via F5 key. If there's no option to run debugger or you prefer to use prints, logging can be the solution. The minimal logging level to print during runtime (debug, info, warning) can be set by its config. IMO the loguru lib is the easiest solution to use for logging
Wow 😲 my comment referring to a video from another youtuber (mCoding) that has an excellent introduction to IDE python debugging got removed (by you(r team), @ArjanCodes? That doesn't sound like you at all somehow...)
Anyway, make sure to check that out! And use Pycharm, it's awesome 😎
Using breakpoints and the debug console of VS Code ? That gives you pretty much full control over runtime code and what is going on
Love pydantic, makes life much easier
Thanks arjan! Again a great video. I didn't knew about pydantic before and after watching this video, i will surely use this in my projects and make this an addon to my dev toolkit.
Again thanks for this awesome knowledge! 😀
I'm really glad to hear the video had a positive impact on your journey, Devan! Best wishes :)
Just saw for the first time that it's possible to use python primitive types in the dictionary as keys... Learning never ends. That's neat, thanks!
Glad you found it helpful!
Pydantic 2 is much easier and cleaner than version 1, and thanks for clarifying this
Awesome explanation as always Arjan! Thank you!!!
Thank you for the kind comment, Bruno! Glad you liked it :)
Ninja is also interesting to know about, in my opinion. It really is a life-saver for those of us who live in the Django world.
Is django a right tool for making web APIs?
@@matis9783 I recommend you use FastAPI instead of Django for APIs
@@matis9783 if you ask me: with Ninja it is.
Arian great tutorial on Pydantic.
Very informative. Can you/did you use pydantic for validation of sql queries?
Thanks was having a problem upgrading from pydantic 1.9 to 2.8 seeminly some of the modules i wanted to use needed 2.8 - the headache was i could not understand why optional no longer worked - now i get it i had to specifically set the Field Value of all Optional Fields to None
Miss your more long form, typing the code included videos 😊. The click clack is unbeatable.
But it feels like those videos stick way more!
I actually also miss doing those myself. We’re making a few changes at the moment to (partly) return to the style of the older videos. So stay tuned 😊.
Thank you Arjan ! I love pydantic with FatsApi, my only fault was putting too much business logic into it and not knowing where the data transformation was taking place :)
I'm glad to hear that the video was useful, David!
Pydantic would be better, if it would have 1) proper serialising and deserialising and b) If it would have a proper way to convert to and from snake_case and CamelCase for python/JSON stuff.
You can use the field method alias parameter to assign the different field name types
@@santoshmagadum7086 I know, but that's a workaround, not a good design.
Fantastic video. Thank you.
Glad you liked it, Juan!
used it recently to get environment variables.
I think you should make a video about django-ninja
It is django with pydantic and it's mind-blowing
Hi Arjan I have a question regarding python processes. I am diving in to learn elixir. But I am wonderung if the similar functionality we can achieve from python? Especially all processes are like having individual dashboard to monitor the performance and self healing concept. Thanks in advance.
nice video as always👍🏻
Thank you so much!
Hello, @ArjanCodes,
can I ask you use, classmethod decorator in validator methods even if you don't need need class instance in code ("cls")?? Wouldn't it be better to use staticmethod?
What about the Marshmallow library In python, can you please post a video about that!!..
Do you have an example of after creation there is a change in the value? To Validate or change the information. Example: set a new password in example2
Can you create your own types, like the EmailStr, so that they can be used in multiple classes? I feel like this would help reduce the massive class bloat for the final data container and help reusability. (Particularly my use case which would be laden with numpy fields!)
Also, is pydantic still really slow when compared like for like against dataclasses and attrs?
That's what the BaseModel stands for, doesn't it ? You can create your own custom string type, or int type, or anything else you wish, and change every behaviour of the type (and also its validation, thanks to pydantic)
@@juanjosefarina ah so you'd basically nest types and it's solved recursively?? Ty! Definitely going to take a look and see how it runs on my embedded stuff. I'm using dataclasses at the moment which are nice and quick, even though I'm locked on 3.8 so no slots.
How you handle uniques validations in combination with database in fastapi?
Can a field be a descriptor?
6:43 line 58 why use a list comprehension when you can just provide a generator expression to join()?
Thank you....
i use attrs !
thx
7:02 my teamlead will "kill" me for code like on lines 53-55, it's unreadable. Why do not you use a match statement?
For some reason making compositions and accessing attributes from nested models just doesn’t seem to work the way it’s supposed to. I know my models work because I can query them in any other validation library.
you should had made analogy with zod
The new pydantic version is deprecating example field, now you need to use json_schema_extra={"example": "123"}
Good to know!
What about schematics for python
I would like to know your opinion about SQLmodel for fastapi
I like the idea of the library, but I’m waiting to do a video about it until it catches up to the latest version of SQLAlchemy and Pydantic.
With lots of models and some even nested ones pedantic is extremely slowing down our cold start time in the lambdas. Also with 3.12 + pydantic2 + fastapi and debug mode it sometimes takes up to a minute to start the application:/
12:11 Oh and ellipses are no longer necessary for required fields. I think they're even discouraged
i suggest compiled fastjsonschema for deeply nested messy data
your discord is not loading
how you gonna write the test cases for them ?
What do you mean how ? One way was pretty much everything he showed in the video, but you should be using pytest or other testing library.
Pydantic is epic, if it was fighting elections. He would win single handedly, cause there's no opposition.
why use pydantic when you can... just check the types with the isinstance() function?
Miller Kenneth Jackson Brenda Brown Timothy
Good video! The North face is racist
But you shouldn't be using poetry.
Why?
It's very flawed, uv is a much better solution. Also poetry did some very questionable things with versioning. Like randomly raise an error (literally pick a random number and throw an exception instead of deprecation warning)
It's very flawed, uv is a much better solution. Also poetry did some very questionable things with versioning. Like randomly raise an error (literally pick a random number and throw an exception instead of deprecation warning)
If you use data from mongodb is there better way to create instances of pydatic classes then getting binary json as dictionary and dictionary unpack into pydantic instance? I looked into creating pandas dataframes with PyMongoArrow but I'm not sure I need that clunk.
you can try beanie-odm☺