Comment regarding pagination vs cursoring. Pagination grants MUCH more flexability when consuming data, and a well built pagination endpoint also need not necessarily be slow. Cursor based will often force the consumer to go through everything to get a specific piece of data. A well built or poorly built implementation in **either case** can cause issues and slowdown.
Please keep in mind that using filters inside the header and not the body might result in many url length related limitations. On the other hand using filters in body will not allow to preview directly the result in a simple browser GET navigation. Most likely a decent solution would support both simple filtering via a query string format inside url but also a more advance filtering via body. This is important when you application has complex filters or multiple filters. EG1: multiple filters with multiple options where you might hit a limit unless you defined an encoding to represent a set of multiple filters being selected. EG2: a business oriented app where the user needs customisable filters and grouping of filters with AND / OR between them like (filter1=A or (filter2 = b and filter3=c)). Users might end up with really long filters that might not be easily serialisable in an URL , keep human readability AND size constraints.
Two other tips I want to add regarding Documentation. 1) Make sure you have good, language agnostic documentation. Most API's provide libraries for common programming languages which is great, but there are a lot of languages out there and you can't come up with libraries for all of them, or for future languages. My recommendation is to first write you language agnostic documentation, and then you write your libraries using that documentation, and if there's some information you need that's not in that documentation, then update it. 2) Existing documentation doesn't mean you can be lazy. I work with a lot of FHIR based API's, and despite the fact they all supposedly follow the same standard, there are always a thousand small differences and changes, and very few of them are listed in the vendor's documentation. Remember: Good documentation is an investment. It means less support work for you or your staff, and it means your customers are able to use your API faster, which means they start paying you faster. Bad documentation makes it more difficult for people to use your API, which means they're more likely to give up or look for an easier to use alternative. Rule of thumb for good documentation: If you are the person who is intimately familiar with the API, and you look at the documentation and think "Yeah, that's good enough", no it's not, it's shit. Good documentation should be specific, should be redundant, should have examples. Good documentation, when you the developer of the API, look at it, should make you say: "Do we really need this much detail? Surely it's too much?"
I will add that these are great advices when you are building a public api. If you are building an api that is basically a backend for a single frontend, you can go easy on these standards and build what you actually needs instead of following blindly the norm.
00:01 Crafting effective, secure RESTful APIs 00:45 Ensure reliability through idempotent APIs 01:32 Use versioning to introduce new API versions without impacting current consumers 02:13 Versioning for backward compatibility 02:56 Use clear query strings for sorting and filtering API data 03:41 Security should not be an afterthought in API design 04:22 Clear linking and direct paths improve API usability 05:06 Good APIs protect infrastructure and encourage fair use
Tip 1 should be replaced with consistency. Plural or singular, but stick to the choice made. I'd argue "cart/122" is more elegant than "carts/123". If our pattern is cart/, then carts/123 deals with one cart, so singular would make more sense. Consider "cart/" with no id as the list of carts.
I disagree. cart alone is misleading. It may read better when you add a cart number to it, but it doesn't make sense for what it actually does. A "GET /cart/123" might make sense if you're reading it literally. But what do you suspect doing "GET /cart" returns? I would assume it gives me just 1 cart, but it should actually give me a list of all carts (otherwise why do would you need an ID if there weren't any other carts?)
The top 7 tips: 1) Use clear naming. 2) Ensure reliability through idempotency. 3) Add versioning for backward compatibility. 4) Add pagination for response. 5) Use clear query strings for sorting. 6) Security should not be an afterthought. Use api-headers for passing in tokens. 7) Keep cross-resource references simple. 8) Rate limiting
Do not use get for filteting as it is impractical, instead use a /list or /filter endpoint and post a complex filter object, yielding you result set. In practice in bizz apps filters/sorts ars very complex and it is unusabld via query params. Example: POST /doctor/list ...json filter object. You can get an object by canonjcal id, but filtering for a list if object representations is much better done via special list resource.
Another indication to apply the fifth suggestion is that when you share a link with those parameters, the recipient will also have it sorted just as you did.
Versioning can be also done using headers. Custom, like x-version-api, or use an existing one. Or even query string params (not the best way, but still).
What would you recommend when the type of parameter in a query (usually GET) is an array with no limits? Let’s say an array of GUIDs. Having a GET endpoint would make it hard to read by humans (since every item is “concatenated” using the same key) and could potentially have issues with URL length (although more and more modern browsers don’t have so much limitation but there could be network devices in between that could still have length limitations). In such cases I have been using PUT, with the array of items being passed in the request body.
Really great tips and ideas. And +1 good documentation about authentication methods and possible entities and their fields with its types are a major plus.
I was hoping you would say something that makes me totally rethink of making an API.. Sadly, (or happily) I already knew these techniques.. Only slight difference is the usage of a header to do versioning, rather than url based.. I got that trick from github.. but to be honest I am on the fence if I like path based or not.. Path based is easier to understand from a OpenApi spec document.. Choices Choices...
versioning on the header level is interesting, never thought of that. i might want to try it as it will relief me from messy url's and makes documentation more easier 🎉🎉
Actually get methods are not favored at all for fetching items. POST is typically used. Take a look at UA-cam's API calls and you'll see. The reason is, a) it is inherently less secure to have the filtering, querying params in the URL, which can give away PII. b) Post methods can provide these methods in the body and hence provide more granularity for search parameters. It is possible to cache POST method results too but it requires a bit more diligence.
Yeah and the api as used by the youtube website is pretty much incomprehensible whereas the actual youtube data api for third party developers never uses POST requests for fetching data.
Regarding tip #1.. If we have products endpoint I would like to make product/:id instead of products/:id. /products will be list of products and if you want one you do /product/:id. Cart is not a good example since cart is one per user so it can be without id. Just validate user token to get user id😊
PUT is only idempotent when no optimistic lock mechanism like a version-property is used. In real world examples you should use an optimistic lock to prevent race conditions. When two users load the same resource and edit them the first user runs the PUT to update the resource. Afterwards the second users also updates the resource with his changes, but the data, which will be updated is not the data he has loaded before, because the changes of the first user were already applied. This could lead to inconsistency or loosing information. When using optimistic lock, the second user gets an error (normally HTTP status 409 conflict) that the resource was already changed by the first user. And this also implies that running the same request twice, the second request will fail because of version mismatch.
I do disagree with the first tip for exactly the reason mentioned. Plurals would imply I'm dealing with a collection (== multiple) carts, not a single cart. In the example given, however, I'm only dealing with a single cart (the cart with ID 123), so I think singular ("/cart/:id") would be better. It it was to get multiple carts (eg. 123 and 456), then yes, plurals would make more sense (eg. "/card/123,456").
The video was fantastic, packed with insightful content about APIs. I do have a suggestion, though: perhaps consider having a junior native speaker read out your scripts. This could make the presentation more catchy, allowing you to dedicate more time to content creation and research.
Please don't version your API in your URI allocation scheme, it's bad practice. What version of an API is agreed on between client and server doesn't need to be advertised in an URL. It's an API -- the underlying implementation may change, but the interface doesn't. If /v1/cart changes to /v2/carts, well, why the version?
Well the interfaces do change. I work in fintech space and here is the basic example: In v1: payment mode: net banking Bank name: Bank of America In v2: payment mode: NB Bank name: BOA Note: there are many keys changes in the payload this is just the subset Here, internally the functionality is same as we are making the payment but interfaces is changing In practice: all our existing customers who are using v1 keep using v1, all the new customers are being onboarded with v2 version. I have integrated many banks apis in our application and all uses version in their url.
@@RohitYadav-cu5sh All your example is changing, is the payload i.e. the message entity. Use different Content-Types, or if xml, a different schema. But the URIs shouldn't change. There's a detrimental effect on cache behavior when you're creating multiple URI aliases for the same resource.
I know everyone does it and calls it REST, but Roy Fielding has written and lectured against API versioning in the URI allocation scheme, as a fundamental misunderstanding of the architectural style.
The "Accept" header is your friend. Let's say the data format is changed from application/xml to application/json. Deprecated (v1) URI's will only Accept: application/xml. New URI's will only Accept: application/json. URI's which have not changed between versions will Accept: application/json, application/xml -- in that order. You seem to have a custom media type, version that. Then a client is free to use whichever API version works for their application, even both, and this is all transparent (i.e. self-descriptive messaging) in HEAD requests. See Dr. Fielding's thesis for all the explanations of why API versioning is best done by versioning media types, i.e. application/{vnd.apiv1.json|vnd.apiv2.json}, you'll be a better developer for it even if you continue to give the "corporate" REST API answer in job interviews conducted by those who only know buzzword REST.
Comment regarding pagination vs cursoring. Pagination grants MUCH more flexability when consuming data, and a well built pagination endpoint also need not necessarily be slow. Cursor based will often force the consumer to go through everything to get a specific piece of data.
A well built or poorly built implementation in **either case** can cause issues and slowdown.
Please keep in mind that using filters inside the header and not the body might result in many url length related limitations. On the other hand using filters in body will not allow to preview directly the result in a simple browser GET navigation. Most likely a decent solution would support both simple filtering via a query string format inside url but also a more advance filtering via body. This is important when you application has complex filters or multiple filters. EG1: multiple filters with multiple options where you might hit a limit unless you defined an encoding to represent a set of multiple filters being selected. EG2: a business oriented app where the user needs customisable filters and grouping of filters with AND / OR between them like (filter1=A or (filter2 = b and filter3=c)). Users might end up with really long filters that might not be easily serialisable in an URL , keep human readability AND size constraints.
Two other tips I want to add regarding Documentation.
1) Make sure you have good, language agnostic documentation. Most API's provide libraries for common programming languages which is great, but there are a lot of languages out there and you can't come up with libraries for all of them, or for future languages. My recommendation is to first write you language agnostic documentation, and then you write your libraries using that documentation, and if there's some information you need that's not in that documentation, then update it.
2) Existing documentation doesn't mean you can be lazy. I work with a lot of FHIR based API's, and despite the fact they all supposedly follow the same standard, there are always a thousand small differences and changes, and very few of them are listed in the vendor's documentation.
Remember: Good documentation is an investment. It means less support work for you or your staff, and it means your customers are able to use your API faster, which means they start paying you faster. Bad documentation makes it more difficult for people to use your API, which means they're more likely to give up or look for an easier to use alternative.
Rule of thumb for good documentation: If you are the person who is intimately familiar with the API, and you look at the documentation and think "Yeah, that's good enough", no it's not, it's shit.
Good documentation should be specific, should be redundant, should have examples.
Good documentation, when you the developer of the API, look at it, should make you say: "Do we really need this much detail? Surely it's too much?"
Ah man, feel sorry for you, FHIR is a total disaster
I will add that these are great advices when you are building a public api. If you are building an api that is basically a backend for a single frontend, you can go easy on these standards and build what you actually needs instead of following blindly the norm.
agree
00:01 Crafting effective, secure RESTful APIs
00:45 Ensure reliability through idempotent APIs
01:32 Use versioning to introduce new API versions without impacting current consumers
02:13 Versioning for backward compatibility
02:56 Use clear query strings for sorting and filtering API data
03:41 Security should not be an afterthought in API design
04:22 Clear linking and direct paths improve API usability
05:06 Good APIs protect infrastructure and encourage fair use
I was just asked about the idempotancy of different methods during the interview! Well explained!
Tip 1 should be replaced with consistency. Plural or singular, but stick to the choice made.
I'd argue "cart/122" is more elegant than "carts/123". If our pattern is cart/, then carts/123 deals with one cart, so singular would make more sense.
Consider "cart/" with no id as the list of carts.
I disagree. cart alone is misleading. It may read better when you add a cart number to it, but it doesn't make sense for what it actually does.
A "GET /cart/123" might make sense if you're reading it literally. But what do you suspect doing "GET /cart" returns? I would assume it gives me just 1 cart, but it should actually give me a list of all carts (otherwise why do would you need an ID if there weren't any other carts?)
@@nicdgonzalez🥚xactly
The top 7 tips:
1) Use clear naming.
2) Ensure reliability through idempotency.
3) Add versioning for backward compatibility.
4) Add pagination for response.
5) Use clear query strings for sorting.
6) Security should not be an afterthought. Use api-headers for passing in tokens.
7) Keep cross-resource references simple.
8) Rate limiting
Do not use get for filteting as it is impractical, instead use a /list or /filter endpoint and post a complex filter object, yielding you result set. In practice in bizz apps filters/sorts ars very complex and it is unusabld via query params. Example: POST /doctor/list ...json filter object. You can get an object by canonjcal id, but filtering for a list if object representations is much better done via special list resource.
the only problem is if you try to share the link to another person, not containing the same filter
As always very well explained, thank you!
Another indication to apply the fifth suggestion is that when you share a link with those parameters, the recipient will also have it sorted just as you did.
Great video! Some key points were highlighted that I hadn't heard anyone else discuss. Thank you!
Versioning can be also done using headers. Custom, like x-version-api, or use an existing one. Or even query string params (not the best way, but still).
Thank you this was super useful!
thanks. may be you can give examples for non restful endpoints.
These visuals are so fantastic. Love your videos and subscribed.
Very clean explanation.
What would you recommend when the type of parameter in a query (usually GET) is an array with no limits? Let’s say an array of GUIDs. Having a GET endpoint would make it hard to read by humans (since every item is “concatenated” using the same key) and could potentially have issues with URL length (although more and more modern browsers don’t have so much limitation but there could be network devices in between that could still have length limitations).
In such cases I have been using PUT, with the array of items being passed in the request body.
Thanks for the video. But I am curious, what tools did you use to create your own diagrams and animations in this video?
Thanks! I was guilty of not using headers for keys. Great point that keys would be in url logs.
Really great tips and ideas. And +1 good documentation about authentication methods and possible entities and their fields with its types are a major plus.
Versioning hell is great!
FWIW - Headers often work better for versioning rather than embedding that info in the URL.
Big fan of your book sir. I am new to Architecture. 🙌
Few more:
Standardize your API response.
Return proper error codes and messages.
Can you talk about API request validation?
this is helpful! Thanks!
Thank you for such an informative video.
Great video
you seem happy in this video =)), talk faster than usual
wonderful, thanks a ton!
thank you
I was hoping you would say something that makes me totally rethink of making an API..
Sadly, (or happily) I already knew these techniques..
Only slight difference is the usage of a header to do versioning, rather than url based..
I got that trick from github.. but to be honest I am on the fence if I like path based or not.. Path based is easier to understand from a OpenApi spec document..
Choices Choices...
versioning on the header level is interesting, never thought of that. i might want to try it as it will relief me from messy url's and makes documentation more easier 🎉🎉
hello could you tell me what you think of the ideas I had if they are feasible
thank you!
Actually get methods are not favored at all for fetching items. POST is typically used. Take a look at UA-cam's API calls and you'll see. The reason is, a) it is inherently less secure to have the filtering, querying params in the URL, which can give away PII. b) Post methods can provide these methods in the body and hence provide more granularity for search parameters. It is possible to cache POST method results too but it requires a bit more diligence.
Yeah and the api as used by the youtube website is pretty much incomprehensible whereas the actual youtube data api for third party developers never uses POST requests for fetching data.
Good presentation , May i know what tool you are using to presentation
IOT sir chadastham valla ochinollaki swagatham suswagatham
#5 vs #7 How about `/users/sort_by/registered` ?
When does it come to secure APIs? I watched the half of the video, but not a word about safety
:55 Yes! Another good video, thanks! 😎✌️
Versioning is such a controversial topic in API design. As a matter of fact, on a lighter note, REST is controversial to begin with. 😀
In regards to idempotancy, curious to know your opinion for when the API is requesting live data and the response keeps changing every second.
url in your about section to blog is broken
Awesome.
No comments on Level1 - 3 and HATEOAS?
This channel needs subtitles.
Great
awesome
Regarding tip #1..
If we have products endpoint I would like to make product/:id instead of products/:id. /products will be list of products and if you want one you do /product/:id.
Cart is not a good example since cart is one per user so it can be without id. Just validate user token to get user id😊
/product/:id is not restful
who is here after raghava sir message
Me...
Too fast and hard to understand.
In tips 2: why PUT idempotence yes, but patch no
Thank u
PUT means updating all the fields in an entry, PATCH means updating selective fields in an entry
PUT is only idempotent when no optimistic lock mechanism like a version-property is used. In real world examples you should use an optimistic lock to prevent race conditions.
When two users load the same resource and edit them the first user runs the PUT to update the resource. Afterwards the second users also updates the resource with his changes, but the data, which will be updated is not the data he has loaded before, because the changes of the first user were already applied. This could lead to inconsistency or loosing information.
When using optimistic lock, the second user gets an error (normally HTTP status 409 conflict) that the resource was already changed by the first user.
And this also implies that running the same request twice, the second request will fail because of version mismatch.
Patch can be used to add an item to an array, calling it multiple times would add multiple items. Put would overwrite the same items everytime.
I do disagree with the first tip for exactly the reason mentioned.
Plurals would imply I'm dealing with a collection (== multiple) carts, not a single cart.
In the example given, however, I'm only dealing with a single cart (the cart with ID 123), so I think singular ("/cart/:id") would be better.
It it was to get multiple carts (eg. 123 and 456), then yes, plurals would make more sense (eg. "/card/123,456").
How about versioning per endpoint?
Yeah I tend to go for the /carts/v1 approach tbh
Do POST need to be idempotent????
No. That's why PUT exists
The video was fantastic, packed with insightful content about APIs. I do have a suggestion, though: perhaps consider having a junior native speaker read out your scripts. This could make the presentation more catchy, allowing you to dedicate more time to content creation and research.
15 inches 😧
rate limiting does not help against ddos attacks. What content is this?
Would you mind elaborating?
Personally, as a developer I'd rather have nearly everything be in a query string than to have to codify building out separate URLs for each use case.
plain generic info...
accent is tricky to understand please talk slow
Love your videos.
personally I do not like plurals into API...🥸
Lol i misread the title as Effective Sale API
Please don't version your API in your URI allocation scheme, it's bad practice. What version of an API is agreed on between client and server doesn't need to be advertised in an URL. It's an API -- the underlying implementation may change, but the interface doesn't. If /v1/cart changes to /v2/carts, well, why the version?
Well the interfaces do change.
I work in fintech space and here is the basic example:
In v1:
payment mode: net banking
Bank name: Bank of America
In v2:
payment mode: NB
Bank name: BOA
Note: there are many keys changes in the payload this is just the subset
Here, internally the functionality is same as we are making the payment but interfaces is changing
In practice: all our existing customers who are using v1 keep using v1, all the new customers are being onboarded with v2 version.
I have integrated many banks apis in our application and all uses version in their url.
@@RohitYadav-cu5sh All your example is changing, is the payload i.e. the message entity. Use different Content-Types, or if xml, a different schema. But the URIs shouldn't change. There's a detrimental effect on cache behavior when you're creating multiple URI aliases for the same resource.
I know everyone does it and calls it REST, but Roy Fielding has written and lectured against API versioning in the URI allocation scheme, as a fundamental misunderstanding of the architectural style.
The "Accept" header is your friend. Let's say the data format is changed from application/xml to application/json. Deprecated (v1) URI's will only Accept: application/xml. New URI's will only Accept: application/json. URI's which have not changed between versions will Accept: application/json, application/xml -- in that order. You seem to have a custom media type, version that. Then a client is free to use whichever API version works for their application, even both, and this is all transparent (i.e. self-descriptive messaging) in HEAD requests. See Dr. Fielding's thesis for all the explanations of why API versioning is best done by versioning media types, i.e. application/{vnd.apiv1.json|vnd.apiv2.json}, you'll be a better developer for it even if you continue to give the "corporate" REST API answer in job interviews conducted by those who only know buzzword REST.
U missed the backwards compatibility. Every version should support backwards compatibility.
Thank you @ByteByteGo ! I think this might be one of your best videos yet, you packed so much useful knowledge into this one, Thank you!