Benchmarking GraphQL Node.js Servers
Вставка
- Опубліковано 5 вер 2024
- Learn how to improve the performance of your GraphQL Node.js servers. Includes using graphql-jit, avoiding apollo-trace, and swapping apollo-server out for express-graphql
Code: github.com/ben...
----
Follow me online: voidpet.com/be...
#benawad
Forgot to mention that I still plan on using type-graphql. It will be getting a raw setting next that should eliminate the runtime overhead
Update: twitter.com/benawad/status/1214272872706916352
Update2: I'm using express-gql twitter.com/benawad/status/1215354990161776641
Also I don't think you *need* to change anything you're doing, but if you're looking for better performance these are some options.
Not a fan of Go, but with github.com/99designs/gqlgen one would achieve even better results performance-wise.
typegraphql + typeorm ARE SO good that I would rather pay for more CPU instead of getting rid of them
This info is very timely since I'm new to GraphQL. I appreciate learning about performance considerations early before making any poor stack choices!
Not all choices should be made solely on performance factors.
Thanks for taking the time to make this
This is quite interesting and the most comprehensive breakdown of GraphQL I have seen. I am a big fan of GraphQL but it also reminds me that it was created to solve specific problems and REST should be looked at as the first option because I personally believe that performance and user experience trumps all. Also speaking of web caches they are much easier to implement in REST. One other thing, I have never done the type of bench-marking you have done here with GQL servers; however, I have found I get much better performance metrics when I use non JavaScript GraphQL servers, mainly Sangria in Scala and Absinthe in Elixir.
Thomas Johnson I agree. GraphQL at its core is simple, it’s just another HTTP request. However at its core, it’s too limited. So you essentially have to add infinite plugins to get thungs to work as expected. Apollo client is like 7 different npm packages and quite complex in terms of caching opposed to using just axios or even fetch. Server side is the same issue having to resolve each field manually versus just returning static JSON. You really need to weigh your options. Not to mention GraphQL takes more work than serving JSON over http
Great video Ben. Thanks for taking the time to do this. Currently working speed improvements using gql and new relic. Good to know that apollo tracing has such an impact, in case anyone is trying to analyze performance.
Thanks for sharing man! I like these breakdown videos.
I was just wondering about this the other day. Thank you
Great presentation Ben. Heaps of detail. Thanks :)
Great video, I decided to use Koa instead of express just because is lighter, now I know is indeed better (at least for my project)
Excellent comparison, thanks for sharing!
Benchmark runs at 16:00
You saved it!
Thanks! Really interesting benchmark.
Excellent video Ben. Very detailed and explanatory.
Just one thing. What's the point of using Faker for generating dummy data if you're seeding it? Isn't it just the same as having a single mock?
Keep up the good work.
1. it made it easy to create the single mock
2. it's easy to unseed
This is very helpful and well done. Have you used or compared aws appsync service? How does it compare with using apollo server. Thanks again.
Personally if i care about performance then i would not use NodeJS and GQL. Insted... Go and ProtoBufs (gRPC) if payloads are small.
Performance always matters. Just a question of how much
Yeah good luck using GraphQL properly in Go.
Sorry, but gRPC for an API gateway instead of REST/GraphQL is kinda pointless if you have a frontend team.
Sure thing grpc-web exists, but isn't really efficient.
Now using gRPC as communication between services is great.
@@uziboozy4540 Depends on the frontend. If it's plain web then there is no native RPC implementations (but doable with performance hit), if its Flutter/Dart and kind, then no problem at all. And gRPC can be easily used to automatically expose REST endpoints with plugins like grpc-gateway. + Swagger plugin and you get really well documented API without much effort.
The thing is that i often see GraphQL used for inter-service communication, which looks silly to me. It just wastes CPU for json parsing.
Personally i am waiting when gRPC will be available over HTTP/3 Quic. I will be the most happiest person in the world. :D
@@Oswee Yeah but there's the problem, not everyone will be using HTTP/3 even years after it's been integrated into Chrome.
Some people are stuck with old versions of browsers.
You could easily fix this by proxying the request depending on their supported HTTP version.
@@uziboozy4540 Let's pray for better! :D
Great video, would love to see your refactor path from apollo-server to express-graphql. Going to be doing myn soon and be nice to see if theres any gotchas ;)
I ended up making my own lib github.com/benawad/express-gql
the transition was simple, but I wasn't using any special apollo features
No point comparing Apollo if you’re not planning to use their niche features: batch queries and persisted queries.
Works the same way when you don’t plan to heavily plugging middlewares, express makes no sense here.
With koa in the picture, might as well adds zeit/micro and hapi?
I'd say most people use Apollo without utilizing batch queries/persisted queries so I think it's worth mentioning
PRs welcome ;)
Hey Ben, Great video and explanation! Thanks for all the info, thought and work that went developing this repo. You mentioned that you will be converting to express-graphql from Apollo-server. Do you think you will do a video on that process?
I'm actually thinking about trying fastify-gql first: twitter.com/benawad/status/1214272872706916352
I'm going to give it a try and I'll do a video on it if it makes sense
@Ben Awad how do they compare when run on serverless functions? also how do they compare against golang counterparts, also running on serverless functions?
So we should move over to REST :)
I feel like caching plays a big part in a lot of production settings. REST is also easier to cache than GraphQL. Apollo Server has a bunch built in to help cache results.
yes keeping things simple ussualy leads to faster things
Hey Ben, you were one of my inspirations to start a UA-cam channel. Any advice on getting started?
@@TheL0rdOfTheStrings +1 this is a great resource
Hi Ben,
Great comparison.
Have you tried pre-compiling your graphql-schema at build time?
String-based schema has to be parsed into a graphql-document (js object). This adds a lot of overhead.
Maybe it could help improve the performance.
How would I pre-compile it?
@@bawad You can compile it yourself just using graphql-tag, or you can use a babel-loader or webpack plugin, for example
github.com/gajus/babel-plugin-graphql-tag
It's primarily meant for client-side graphql queries, but it works for server as well.
@@Ugron1 I see, I think apollo-server lets you do gql`` now
Very nice article.
I'm a graphql newbe, but I think your tests should have called a db. the relative time differences would be much more meaningful. say a 50% time diff without a db call may be just 10% diff when you call a db. if the request calls db multiple times, the diff in time it spends in graphql framework may be negligible. if code spends most time doing db calls, then the framework choice makes not much difference in performance.
It's definitely more fruitful to optimize db queries but I wanted to look at the speed of GraphQL specifically
Wow this is awesome. Thanks man!
Thanks man, you're super smart.
Apollo also has a cache system. I think is similar to graphql-jit if you implement it as well
Not all queries are cacheable though. So you have to be mindful when picking what queries to cache. You can find the docs here:
www.apollographql.com/docs/apollo-server/performance/caching/
www.apollographql.com/docs/apollo-server/performance/apq/
@@sergeimeza4663 twitter.com/scotttball/status/1214301210712240128
Awesome video.. Thank you
Thanks! Pretty good content
Yeah, for sure. The schema isn't new for every request, or at least it shouldn't be and only needs validation on changes. I don't get that bit where the express version is validating schema on every request.
Maybe it's the request being validated against the schema?
@@scottamolinari It seems to have been a concious decisison, at one point. At least, they are testing for it - with quite explicit wording regarding the at-request-time behavior.
github.com/graphql/express-graphql/blob/master/src/__tests__/usage-test.js#L100
Thanks Ben!
Hello BenI You are definitely moving in right direction! Can you please compare bare AWS API gateway lambda without express library? for me express itself, and especially after you add CORS middleware is an overhead.
A lot of programmers do not know that, and added express.js library into AWS lambda.
I'm not interested in using AWS lambda, but your welcome to add a benchmark for it
Ben Awad Hi ben, may i know why?
@@wj6946 I find a regular server easier than serverless atm
@@bawad how about now?
Amazing! Thanks!
gRPC is the future, fast & easy & love
can it totally replace all the benefits of graphql like no overfetching?
there is a project that can create graphql schemas from grpc protobuf.
should we use that or straight grpc.
how does the clientside interaction from grpc look? is it as developed as graphql? even on the browser?
I wasn’t aware it was browser supported?
I'd be curious how subZero (postgREST) (subZero implements a postgRESTgraphql alternative) would work out in the test. I would imagine it would be extremely fast with stable memory usage. But I don't really know.
Can we swap apollo-server for express-graphql even for the gateway?
Great video, cheers. :)
Let's be honest if we wanted performance why would you use Node server? Wouldn't Go be a better option? Is it really better to drop typegraphql for a couple hundred less requests when you get the better dev experience that comes with it. I struggle to know when to draw the line between performance and usability
I agree, but it's helpful to know how to get the most out of node.js
Hi Ben, love your channel! What are your thoughts on Postgraphile?
I don't speak English well Ben would like to know if you used postgraphile and what he thinks about this
I like it, but don't use it because I don't want to put logic in postgres
plan on adding any non node servers to the benchmark?
I have no plans to add non node servers, but happy to accept a pr for them
You can have a look at github.com/the-benchmarker/graphql-benchmarks (based on famous Web frameworks benchmarks) which covers also other languages/ecosystems. At the moment it covers c, go, ruby, and 4 node/javascript frameworks.
I think there is a danger of prematurely optimising your code based on these results. You can scale your node servers horizontally should you hit any scale. When you start hitting thousands of requests a second it will typically be your database which becomes the bottleneck. Things like not building the schema on each request is good advice. I would have thought a bunch could be done with caching to reduce the amount of work being done on identical requests.
totally agree
Does anyone have any experience with performance of a Java based implementation?
Hi Ben, love your channel! What are your thoughts on Prisma?
twitter.com/benawad/status/1199344762362314757
@@bawad Agreed. I'm working on a project I inherited built on Prisma 1 right now, and I'm going to try to get the team to migrate off it as there are many things I don't like about it (to be fair, I'm not as well-versed in the GraphQL ecosystem as you though).
What are your thoughts? Do you have an opinion? Or will Ben's be yours?
@@whipstitchwebwork1383 On what, Prisma 1? I don't like it because it's too inflexible (eg. can't support keys consisting of multiple columns), requires a whole separate Prisma server, and is going to be obsolete due to v2 which is totally different.
Also I think GraphQL is overrated, particularly since the tooling around it is still pretty immature. I think a lot of developers are adopting GraphQL just because they think it looks cool on their resume, without actually evaluating whether the tech will benefit their application and neglecting other things (eg. good code structure, testing).
I'll qualify this by saying I come from more of a frontend background (my last time doing backend was in 2008 before ORMs were a thing). Not sure how I feel about ORMs tbh as it feels like I'm just learning the ORM. That's why I'm leaning more towards something like knexjs.
So, would this whole performance issue be inexistent if we were using a strongly typed language? Talking of languages like Java, Python... I'm thinking that if that is the case we better not use JS when deciding to build apis with GraphQL...
No, one of the requirements of the GraphQL spec is to check the types at runtime
@@bawad you're right
Edit: what do you think of comparing a few GraphQL implementations against each other?
I'm not interested in using another language besides node.js at the moment or I would
i tried uWebSockets with graphql-tools
6113.2 │ uws-graphql-jit
5134.0 │ fastify-REST
4877.2 │ fastify-gql+graphql-jit
4083.4 │ express-REST
2463.0 │ koa+graphql-jit
2235.6 │ express-graphql+graphql-jit
2180.2 │ fastify-gql
1981.4 │ express-graphql+graphql-jit+type-graphql
1927.2 │ apollo-server-fastify+graphql-jit
1810.2 │ uws-graphql
Awesome
Should not Apollo Server cache those requests? Did you see any effect of that?
twitter.com/scotttball/status/1214301210712240128
Have you looked into Hasura?
I ask because it seems like you are using postgres already, and from the looks of what you were building it seems like it could all easily be accomplished with hasura.
yes twitter.com/benawad/status/1199348053204852737
Very cool Ben. It would be interested to see how Hasura compares although it wouldn't be an apples to apples comparison given Hasura is pre-compiled Haskell. The other thing about Hasura is that if you need to apply any middleware or custom logic, then you have to call a remote procedure (like serverless) which has its own performance implications.
@@LetterSignedBy51SpiesWasA-Coup I was mainly referring to his example recipes website, not sure if I can think of custom logic he would need to do what he wants. Most of my custom remote logic is for writes, and not reads. And you can do a lot with views, and sql functions.
If majority of your GraphQL schema is just mapping to the DB I think that Hasura would be a great choice. But I agree is it very nice to see a breakdown of the debugging process. I have enjoyed the recent graphql videos.
@@icodefor I agree. For a postgres backend, Hasura would be my first choice because of all the GraphQL resolving you get for free and they claim to be "blazing-fast" and I believe them.
Hello Ben, which graphql library would you recommend that works well with nestjs. Im currently avergaing 200ms per every query (I mean every query ). Or if there any tweak to mitigate graphql overhead.
do you know what your bottleneck is? Is it nest or something else your doing?
@@bawad its definitely not nest, I tried using nest with a rest API and performed quiet well. Must be the graphql library
Might be your db too
@@bawad which tools can I use to actually trace the request all the way to the db?
What do you mean by tracing at resolver level ?
What's tracing ? ( Sorry im new to web tech)
Any links for reading would be great
@@jinxblaze a trace look like this twitter.com/benawad/status/1212392789637521410/photo/1
it measures the time it takes for different things in a request
You can make videos about using fastify-gql =))
We'll see
@@bawad thanks
But have you tried pure node HTTP server or uWebSockets.JS project? It's definitely faster than using fastify/ express.
I haven't
Its also more verbose, so more maintenance in the long term
@@alexnezhynsky9707 not really. If it's just a pure GraphQL server, it's fairly easy to replace underlying server. You may not need extra overhead from express if you need performance.
you displayed latancy is it average? Otherwise its wrong to use average for latancy.
Not sure, why is using the average bad?
@@bawad Maybe to be more precise and remove outliers a comparison in p99 is more precise. I guess this @shirshak point
do you use apollo client or redux or something else ?
Apollo client
This project looks like a pretty nice Bachelor's thesis
Well, which graphql nodejs server do you recommend to use right now?
fastify-gql
@@bawad Can we use fastify-gql with type-graphql?
@@cimsir_tl yep. just pass along the schema generated by type graphql to Fastify
@@oscarpena6895 i did it! Server has only hello query but you can see in this url:
type-graphql-fastify-gql.herokuapp.com/playground.html
github: github.com/berkaycimsir/Fastify-Gql-With-Type-Graphql-Example
Can you make a video or test Facebook's Relay Graphql? relay.dev/. Because every one talking that it is the fastest graphql implementation available.
It can be used serverside as well.
What are your thoughts on Hasura?
twitter.com/benawad/status/1199348053204852737
That's strange we have much different results for fastify-gql+graphql-jit+type-graphql, i have 5538 compared to 5481 non-jit and 6224 fastify-gql-jit, and you've only 1269 for fastify-gql+graphql-jit+type-graphql, vs 5437 fastify-gql-jit, and there's no result for non-jit
So for me results are pretty similar, while your fastify-gql+graphql-jit+type-graphql 5x time slower :C
I'll rerun fastify-gql+graphql-jit+type-graphql
I forgot I had something running in the background when that one ran.
@@bawad thank you
Have you looked at FaunaDB yet?
nope, no plans atm, I'm liking postgres
What about koa +grapql jit
just added it, check the readme
@@bawad nice
Gatsby graphql?
Doesn't count. Gatsby could use anything since it's all done at buildtime.
Hey man... why don’t you have a website
benawad.com
Ben Awad, this my first time seeing a bare bones static site as a portfolio
that's because it's not a portfolio
Ben Awad, if it works don’t change it, so good stuff man! And keep on pushing along
Ben, have you tried DGraph?
Then try grpc now.
same benchmark for rust+juniper+actix in release mode
5 connections
┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬───────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼───────────┤
│ Latency │ 0 ms │ 0 ms │ 3 ms │ 10 ms │ 0.32 ms │ 1.74 ms │ 112.54 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴───────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec │ 5095 │ 5095 │ 5279 │ 5583 │ 5322 │ 173.99 │ 5095 │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 1.08 MB │ 1.08 MB │ 1.12 MB │ 1.18 MB │ 1.13 MB │ 36.8 kB │ 1.08 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
I found I was logging the requests in actix, so I removed it and got insane bench marks
┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬───────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼───────────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms │ 0 ms │ 0.01 ms │ 0.36 ms │ 111.62 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴───────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec │ 18431 │ 18431 │ 22863 │ 22927 │ 21982.4 │ 1779.49 │ 18421 │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.91 MB │ 3.91 MB │ 4.85 MB │ 4.86 MB │ 4.66 MB │ 378 kB │ 3.91 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
holy crap, rust op
Wow look at rest
I've been using GraphQL in production for around a year and my conclusion is that it's total shit. It's not an improvement over REST. If you're using REST and thinking about switching to GraphQL I'd strongly advise against it. Stick with what works well. GraphQL is a poorly thought out API standard.
What didn't you like?
why are you doing these benchmarks? I remember you had problems with loading speed, is this related? what solved the speed issue last time for your app? because 1000 or 4000 requests per second seems irrelevant to me. If you have 10, yes, ten, requests per second, that means on average, already almost a million requests per day. That means you have a huge pimp app, and you can afford a second server, hell, 5 new servers. Even lets say, 400 requests per second is more than enough clearly. Who on earth gets 400 requests a second? twitter? I like the benchmark but does it really provide value? because on paper, 200 requests / s vs 4000 r/s should basically feel the same. One takes 5ms to answer the other takes 0.25ms to answer, difference negligible.
> why are you doing these benchmarks?
1. It's fun
2. I was curious what the cost of graphql was
3. I wanted to see how different libs compared
> I remember you had problems with loading speed, is this related?
ish, I wanted to see how type-graphql compared to apollo style schema
> what solved the speed issue last time for your app?
upgrading type-graphql versions helped, and I will be fetching less data
> I like the benchmark but does it really provide value?
It gives me an idea of what libraries to try to see what effect it has on prod
My thoughts exactly. If an app is loading slow those ms would be the last thing I'd worry about. Kinda like if I built a chair with legs that were 2inches bigger on the left and sliding a piece of paper under the right legs. Did it help lessen the gap? Sure, in theory. But in practical terms? Not a bit.
@@bawad well it was insightful and fun indeed, I enjoyed the video. I just thought I'm not so sure if these optimizations really yield any value. Well, unless you get more than 100 requests per seconds. What would be interesting is how to properly leverage caching with your stack. Like how can you setup caching that automatically expires after a couple seconds and what to use, maybe loading stuff in the background etc
Twitter has about 150m daily users. Web traffic isn't evenly distributed across the day because of timezones, i.e. "peak hours". I think it's a safe assumption that about 50 million people could be using Twitter within one hour. Assuming each one of those people makes 5 requests to Twitter in that hour the average load will be about 70k requests/s. So, no, 400 requests/s is not a reasonable estimate for Twitter.
Of course, Twitter's architecture is probably very complex. The bottlenecks are probably not (assuming they use GraphQL) parsing GQL queries and walking the tree.
Even for small startups, side projects, hobbyists, etc there are other considerations. One might host several sites on a modest VPS. One might want to minimize the ecological footprint of their site for ethical reasons. Etc.
@@DehXable Great math demo and all, but stay in reality. I don't think Twitter is using graphql at scale and certainly not watching Ben's UA-cam channel for advice. For those that are this is the last thing they should be worried about. On small apps (which is prolly his audience's wheelhouse or going to be once their edu is done) graphql is fantastic and it's ease and benefits should not be passed over because of this. No way. Imagine just learning about querying data and you watch this and decide not to incorporate graphql in your stack because it's SLOW? What a negligent act that would be to clients. It should be a go-to choice for most small and mid sized apps. Why it was created in the first place...parse away.
"ecological reasons" that start up will fail.