User Research on Destiny
Вставка
- Опубліковано 7 кві 2017
- In this 2015 GDC session, Bungie's John Hopson discusses the novel types of research that went into the planning of Destiny, including how to best introduce a new IP to players to studying the dynamics of typical raid groups in order to build raids.
GDC talks cover a range of developmental topics including game design, programming, audio, visual arts, business management, production, online games, and much more. We post a fresh GDC video every weekday. Subscribe to the channel to stay on top of regular updates, and check out GDC Vault for thousands of more in-depth talks from our archives.
Follow us on Twitter
/ official_gdc
Check out our Facebook page for GDC exclusives
/ gamedevelopersconference
Visit our site: ubm.io/2ctNvqZ
Here from that labat video
Another reminder for anyone else from Labat he states he does not want John to receive any harassment regarding this issue.
I agree that the commodification of the user was a snowball nobody wanted to be rolled down the hill. I agree evil data is why we have games that drive addiction rather than a returning player.
We are already making the best push we can in the other direction by letting greedy games die. More and more Indies are showing up AAA teams and blowing them out of the water.
@@SPITFIYAHproductions Yea there were hundreds of thousands of game devs one of em was gonna mix psychology with game development sooner or later it's not something you can prevent.
What if they MAXIMIZE fun? Wouldn't that do what they want and remain profitable?@@TheHerrDoktorr
excellent talk. thanks so much for posting
The feeling I have coming away from this talk is that they did a bunch of research but frequently made decisions based on their own assumptions, which kind of defeats the point of all the research to begin with. It seems like most of the times where the user research drove a change was in regards to visibility/readability, such as knowing when a character is a player or NPC, knowing that you're deleting your gear, knowing where damage is coming from, knowing which class the armor looked like it belonged to, etc.
But once you step away from that most of the issues they encounter seem to take them by surprise, such as the loot cave. User research and algorithms told them it could be a problem. They knew it was a very real possibility. But they decided to ignore it based on an assumption about player habits. Similarly, they made gear not matter in MP because they assumed that lab players would be the same as live players. They assumed that RNG wouldn't be a big deal to players, they assumed that players would be able to navigate the post-20 game well, they assumed that players would do other things if a campaign mission was too hard, etc. Obviously we all make hypotheses, but it surprises me how much they ignored or misinterpreted their data in favor of their assumption.
That makes me question obvious design flaws like the bullet sponge bosses, or high tier engrams frequently giving low tier loot. With about six more months of user research than normal and repeat testers there's no way stuff like that didn't come up. Did they just ignore it? Did they assume that it would work out, even over the long periods of time that people would be grinding for gear?
Additionally, I think even in this talk he offers some assumptions that are incorrect. For example, in response to the sustained pseudo-complaints about RNG during the Q&A he said that someone *has* to be that unlucky. That certainly isn't the case at all, since many other RNG-heavy games have systems in place that address this, often by increasing your chances of a good drop after a certain number of bad ones. There definitely is a philosophical debate to be had about whether or not you're in favor of those types of systems, but they are solutions to what he even admits is a very common complaint, and it strikes me as odd that they'd just ignore them completely.
It seems like they they had executives and focus groups take priority over actual designers. Also, who honestly thinks mmo players don't strive to own every exotic/rare drop?
They are professionals with 4 very succsesfull MP games behind them, it is good that they didn't let user research to make their game for them, it is good that they took risks. And it was to be expected because they are damn good at this. Lesser devs would fall into a trap of letting user research make games for them, i'm looking at you, Ubisoft.
"That makes me question obvious design flaws like the bullet sponge bosses" Are you a game designer? How do you know that it is a flaw? Bosses with a big pool of health that constantly shoot at you is a basic test of skill, i see no diffirence in basic design between Sekrion, Nexus Mind and the second boss in Contra (1987). Bosses are supposed to test skill, and take some time to kill.
If Destiny strkies have been a faliure then they would not be so goddamn popular. Just check out Destiny play statistics websites, PvE is more popular than PVP. That is why in Destiny 2 they changed PvP more than PvE.
+ndesuuu
I think you're making the flawed assumption that because they're professionals and their games have had many players that Destiny had no flaws, or that the flaws were intentional and thus justified. Even professionals make mistakes, as they frequently covered in their talk, and their work isn't exempt from criticism by non-professionals. I don't have to be a best selling author to tell you that Twilight had flaws. I don't have to be a director to pan Batman vs Superman. Likewise, I don't have to be a game designer to know when something isn't as fun as it could be. That a game was made by game designers and was popular doesn't mean that it can't be criticized or that all criticism is invalid. I think it's pretty hypocritical of you to call out Ubisoft for making mistakes, but Bungie apparently can't because they're professionals who made popular games. Are Ubisoft devs not professionals who have made many popular games? Way more than Bungie, I'd say, even if you don't like them yourself. Keep in mind that this very video is all about Bungie reaching out to regular non-professional gamers to have them tell the professionals what they're doing wrong, so even Bungie doesn't agree with you here.
The issue with the bullet sponge bosses (at least at launch because I never played DLC content) is that it's NOT a test of skill, it's just a pointless time gate. You hardly have to aim because the boss takes up half your screen, you just look in a general direction and push a button. Sometimes you have to move to avoid an attack or grab ammo, but you're doing literally nothing but point and click, and it's so easy that the only time you're in danger of dying is when you take risks to speed things up because it's so dull. There's no mechanical difficulty, it's just gametime padding. When you get to 10+ minutes of shooting the same guy with the occasional wave of enemies to keep your ammo supply up it gets real old. There's nothing interesting happening, you're just running through the motions waiting for the guy to die already. If it truly is a test of skill like you say it is, then how long should it take to prove that you're able to competently shoot a guy as big as a house? Five minutes? Ten? An hour? Where do you draw the line? And if you've proven that you can pass that 'test' for one boss, then why make EVERY Strike boss a bullet sponge? If I passed once, why do I have to be tested again and again and again in such a mundane manner? It was only during the raid when things started getting interesting, and even then the mechanics were only on par with a bog-standard dungeon from an actual MMO. The fact is that Bungie could have made Strikes more mechanically interesting, but they didn't, and I think that's something we can criticize.
en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
Nightfall strikes are hard. In the first weeks nightfalls were EXTREAMLY hard and only a few players have completed them. Bosses have very small crit spot that are very hard to hit. Accuracy is rewarded. There is a waves of enemies in between. The player has prove that he has the skill by consistently defeating enemies and killing the boss in the process.
The fact that you don't know about crit spots baffles me, have we played the same game? It also sounds like you played Strike playlist when you were too overleveled for it.
So you don't know about basic game mechanics yet you produce walls of so called "critique", that's Dunning-Kruger effect all the way.
"Are Ubisoft devs not professionals who have made many popular games?" Of course they are! Ubi devs are great, but they do spent too much time in user testing lab, some of their games feel like they lack a certain edge that human touch can provide. But i do respect Ubi for all their work, i was just poking them. Because their games would be even better with some personality, for example "Blood Dragon" had a personality in it's style, but it lacked the personality in it's gameplay, it was just too easy for it's "hardcore" style. Do the word "easy" comes to mind when someone sees 16 bit cutscenes?
"Keep in mind that this very video is all about Bungie reaching out to regular non-professional gamers to have them tell the professionals what they're doing wrong, so even Bungie doesn't agree with you here." They are reasearching and testing, in the lab. Gamers are not telling them what they think is wrong like you do. As you can see in this video and many others about user research: they do experiments, users are observed and monitored. It is a test, not ask. Basically you can say that didn't liked the game all you want, but if your heart was racing during expected moments then it's all good.
+ndesuuu
Nightfall strikes are only hard when people are so new that they don't have the gear for them. Mechanically they're the same as the regular strikes, they just have enemies with more health who do more damage. That can be overcome pretty easily with gear, which is exactly the kind of brainless difficulty that I'm advocating against. I was doing Nightfalls with my friend in mediocre gear within the first week or two of the game releasing -- they're punishing, but that's not the same thing as being difficult. And I know a lot of people do them solo with various gear restrictions to try to actually make them challenging. If they can be done easily enough without a full group and with self-imposed restrictions then I think that reinforces my point.
Of course I know about weak spots. I never said that I didn't. It's just so basic I assumed it wasn't worth mentioning. In my opinion shooting heads, killing enemies, and not dying should be the bare minimum of what's expected of you in a shooter, not the actual challenge itself. Sure, maybe the game should want you to "prove your skill" at not dying while killing things early on, but do we really need to prove that same basic skill for every boss over and over again even into the late game? How do you not get tired of that? Maybe I'm repeating myself here, but I want something more. I want actual mechanics. I want the current raid-level mechanics to be in strikes, and I want the raid's mechanics to be more difficult. I know the game isn't an MMO, but it feels like such a missed opportunity to half-ass taking the good ideas from the genre. There's nothing stopping them from giving the strikes something more interesting than beefy enemies. Even the most basic beginner dungeon in MMOs has something more interesting going on than the monotonous strikes in Destiny. Heck, even DOOM 2016 has mechanics in its boss fights!
They certainly do a lot of observation during their user research, but I'm not sure why you think that they don't get verbal feedback from their playtesters. You say it's "test, not ask", but we know from this very talk that there are moderators watching each player and asking them questions about their experience. We see one such example at roughly 29:45. There is a difference between the substance of feedback during play and after when the player has had time to think about what they just experienced, but there's no way they didn't sit down with these people later and take notes on their critiques. It'd be a huge missed opportunity with how much money they poured into testing and repeat testers to not listen to what they had to say.
Heaven forbid we only play as much as we actually want and the activities we actually want.
No, they’re not forcing us to be omnivores, they’re just psychologically manipulating us. Which is better, because that way if we don’t have fun it’s OUR fault! Or there’s room to argue that, at any rate.
Great video, but would of been easier if it had captions!
This guy is awesome! :)
Shame they didn't test how players would transition from base game of uncoordinated coop to requiring coordinated group play for end game.
It would be interesting to know how many players completed raids from total player population. Seems like they made content for the whales, but i'm probably part of a jilted minority.
When I first started, a huge amount of players I partied up with (at least 70%) were horrible, didn't listen, some didn't have mics and messaged "can still hear your mics" yet not get anything right.
It was a horrible experience until I found a group a week later on a forum website after waiting for 9 hours for someone to reply to the topic.
The only point in Destiny after that where I ever failed a raid and people left was when half our raid team was busy for that week.
I did 3 raids a week (we all had 3 classes) and they each took about two hours. When playing with people who were just bad players, sometimes a raid could take 1 to 3 people quitting, taking breaks to use DestinyLFG to look for someone else to fill.
If four players left the party, the 5th always left but that rarely ever happened, usually just one or two people left.
Playing it without a group or at least keeping good players on your friends list from previous raids.. is impossible sometimes. You'll have people blaming everyone for their own mistakes and then players who instantly leave when someone made a minor mistake, even if it never caused a wipe and we advanced onwards.
I should have a video of me leading a raids final boss but I can't remember whether it has me calling out the phases or not.
The loot cave was fun!
what user research?
I put 500 hours into destiny!
wait!?!?
it had story missions?
Gotta pump those numbers up! Those are rookie numbers!
Only 500 scrub?
Lol 500.
Awwwww. It's a cute little baby
I got the impression that they're viewing human beings as alien objects, completely clueless as to what makes them tick psychologically on the most basic level. This all is a rather ineffective and primitive method of user research. : /
I guess this is yet another case where corporate life turns humans into emotionless drones.
slotmachineboiis
bad talc