As a big fan of logic, I must say the way you structure and communicate your thinking is extraordinary. I even notice certain terms and definitions that stay consistent across the videos without ever feeling it’s for the sake of technical buzzwordery. A conversation with you must be a fascinating experience!
Thank you! I think that I have a mostly rational, mostly consistent approach to software development. I think that helps with explaining my ideas, because my view, at least inside my head, is consistent. I believe that I have "strong opinions, lightly held". I am easy to convince that I am wrong, if you have evidence or a stronger theory than mine, but I try to not change my mind based on emotion, rhetoric, or sometimes abuse :) I think you may overstate my conversational skills, I am good on a few nerdy topics and rubbish at chat :)
Your content is similar to what I was taught at the UQAM & ETS in Montreal during my Computer Science master degree. It's really high quality content man.
I once worked with a Senior QA in a small test team that that was adamant about adding implementation details in test cases, for software that could change from release to release. Let's just say, we didn't get along :P
It was beneficial to gain an understanding of the importance of providing a detailed software specification that outlines the functionality of the software, rather than focusing on the methodology used to achieve these functions. 👍🏻👍🏻
Subscribing! Just watched three videos. There's always lots of content of simple "follow what I do" coding tutorials, but these kinds of knowledge-based, approachable mini-lectures actually help you become a better programmer in practice.
Thank you for this video! As a relative newcomer to the industry (in 2014), I am fascinated to see the parallel between in this video (2:42) "BDD meant test-driven development done right" and in Ian Cooper's talk "TDD, Where Did It All Go Wrong" (also available on UA-cam) where at 22:00 (in that video, not this one) Ian Cooper shows that Kent Beck's original formulation of TDD was always about behavior to begin with! Thank you again, I really appreciate being able to piece together some of the history of how we got the terms and practices that we have today.
in aerospace, requirements validation based vehicle mission scope is required. everything else flows down from the requirements during development. subsystems are then functionally broken down and contracted out. useful info. thanks for sharing.
@@ContinuousDelivery Thanks - I am really keen to understand the non-functional relationship here also. As we know, this can come in many shapes and sizes and is not just the various flavours of performance testing.
@@pauld1987 My preference is to try and capture NFRs as FRs. A NFR is just something a user cares about in disguise. So try and get to the idea that the user cares about. For example, when building an exchange we had stories like "A Trader submits a trade and the order-history service fails, but when the order-history service recovers, the order is still correctly recorded".
@@ContinuousDelivery I wouldnt argue against the point you make. I guess the devil is in the detail and the key point here is who the users are. For example, would a business user care where I fail over from and to? They would care about things like RTO and RPO I agree. Been quite a while since I have had an interesting testing conversation.. your videos have certainly got the juices flowing again.. thank you
He is essentially just describing a way to fulfill the DO-178 objectives without all of the mandatory rigor. DO-178 requires multiple layers of requirements written in plain language from which all code and tests must be written. I think following DO-178 would help a lot of software teams even if they don’t fulfill all objectives (specifically the independence objectives).
As I say in the video, I think that the three Amigos, and Natural language thing is great, but knowing the history, at least for me, also broadens the applicability.
Hello Robert, I am an aspiring Scrum master and in time I may be required to coach my teams and that may include my PO and this concept of BDD may come handy. I would need a favour. Can you please share with me a screenshot or of a user story case that fully captures the BBD ?
I use "should" in my test cases too, but I tend to have more context. They're usually of the form: methodXYZ_shouldDoThis_WhenThatIsTheCase(). When I start testing a new method, I start a "template" with: @Test public void test() throws Exception { // Given // When // Then } And I start filling in details. Once that test is done, I repeat with another one. Lately I've been writing a lot of characterization/approval tests. I sometimes find behaviors that feel wrong to me. Since I'm usually not an expert in the the code, I don't want to change the behavior. But I use the test to confirm the questionable behavior. I'll change the name of the test slightly by appending SHOULD_IT to the test name such as: methodXYZ_shouldDoThis_WhenThatIsTheCase_SHOULD_IT(). I'll consult with the team and/or create a ticket. The SHOULD_IT tests helps to reproduce the questionable behavior. If it is the desired behavior, then remove "SHOULD_IT" from the test name and we're done. If it is incorrect behavior, then copy the SHOULD_IT test case, change it to confirm the correct behavior, which should fail when executed. Update the code so that the new test passes and the SHOULD_IT test case fails. Then remove the whole SHOULD_IT test case.
Spec /= Test rather Client-Supplier relation specification (rules based on logical assertions) Scenario /= Test Case rather Client accesses Supplier Given / When / Then --> Precondition / Process / Post-condition = {P}C{Q} The Design-by-Contract assertions are the "tests" we are looking for.
Your explanations are helpful, and revealing. In the case of BDD, I am more solidly believing that BDD = Design by Contract. The most interesting "behaviors" are always between Clients and Suppliers (both human and machine). This relationship is always governed by assertions that define the Rights and Obligations of Clients and Suppliers. Software defects are the experience of a Right or Obligation that is unmet (or violated assertion). A defect in the Client happens because it does not meet its Obligations to the Rights of the Supplier when called (preconditions). A defect in the Supplier happens because the Supplier does not meet its Obligations to the Rights of the Client-that is-to produce precisely what the Client needs and when it needs it (post-conditions). Only the Client-Supplier relationship matters. As long as the Suppliers meet the needs of the Clients, then a system is without defect. Only when the rules (Right and Obligations) of this relationship are not met or followed do we have an interesting defect. We are left with but a few observations: 1. We might have an unspecified Right or Obligation. If we miss such a specification, we open ourselves to undetected defects. 2. We might have an incorrect specification of a Right or Obligation. This is a defect that will allow improper behavior and outcomes. 3. We have complete, timely Client-Supplier rules, which can detect failures in the Client (broken precondition rule) or Supplier (broken post-condition rule) directly. Therefore, Design by Contact is the fulfillment of BDD and is the specification language you are looking for.
His idea of removing things like providing where the response comes to is new, to me at least. This is a really good and thoughtful way to approach it. Makes the business fit test more durable outside the particular implementation.
@11:00 More durable? Perhaps, but certainly not reusable across multiple scenarios. While I do prefer more abstracted scenario write ups, in practice the steps of those scenarios often prove completely unusable across multiple scenarios. There is a place for specific, granular steps - especially in a browser testing.
I am afradi that I disagree. I think that I explain what I mean by the separation more clearly in this video ua-cam.com/video/JDD5EEJgpHU/v-deo.html I think it important to separate "what" from "how", of course the testing has to evaluate at the level of "how" the system works, but the test cases don't. Abstracting them so that they capture "what the system should do" from a user's perspective gives us a lot more freedom for the system to change without breaking the test cases. It may break the tests, but that will be fixed in the, shared, plumbing rather that "test-case by test-case".
@Continuous Delivery I'm afraid reality has a habit of intruding on the idea you're aiming for. For example, my application has a transaction page with a table on it. The step to check that table is "I expect the transaction table to show these values:" with a Cucumber data table. But our application is used by multiple brands, each with their own layouts to the transaction table. As a result the step definition now has to clarify which transaction table. Still, the real problem with your approach is the step definitions can never be used outside the scenario they appear in. This makes tools such as Jira's BDD feature file builder useless.
@@michaelmorris4515 I am afraid that none of your assumptions here are true. First, this is not a theoretical approach, I, and many others, have used this on big complex real-world systems. One of my clients used this approach to test medical devices in hospitals. Another uses it to test scientific instruments. I built one of the world's highest-performance financial exchanges using this approach, and I found out this week, that the tests are still working and providing value 13 years later. I think that your example focuses on the technicalities rather than the behaviour. "I expect the transaction table to show these values" sounds to me like you are leaking implementation detail into your test cases, and that is why they are fragile. What is it that the user really wants? Do they really care about "transaction tables" when they walked up to the computer to do a job, were they thinking "what I need to do is make sure that the transaction table shows these entries"? I doubt it. I can't give you a real example, because I don't know what your app does, but I'd try an capture the intent that the user had. So forgive me for making something up, but lets say that in your case a "transaction" represents selling something, and your "transaction table" represents a list of things, or services, sold. Then I can think of a few scenarios that matter to a user. "I want to be able to buy something and see that I have bought it" (it ends up in the "transaction table"). "I'd like to be able to buy a few things and see a list of the things that I bought" (they all end up in the transaction table). and so on.
@@ContinuousDelivery What am I missing here? I could rewrite the step to simply be "I expect the transaction table is correct" and not mention the values in the feature file at all - but the step can't be freely reused at that point (well it can, but not without revising the implementation code each time). What I'm perceiving that I don't like is the inability to reuse the code at all.
@@michaelmorris4515 My approach is to build a Domain Specific Language (DSL) that allows you to capture the behavioural intent of your users. It is perfectly acceptable to capture parameters at that level, as long as they make sense to the user. For example, if I was writing a test for buying books on Amazon, I could imagine adding the facility in my DSL to specify the name of the book, the author of the book and how many copies I wanted. My DSL would probably default these things if I didn't supply them, but I Could be as precises as I like if I need to test detail. But, the language is ONLY the language of the user, not of the system. The technical detail of 'how' the system works is in lower-layer of the test infrastructure, I call these "protocol drivers" which take ideas like "buyABook" and translate them into "put this value in this field, put that value in that field and press this button". That translation is generic for the concept of "buyingBooks" and so is shared with every test that "buysBooks".
Thanks for these videos Dave, really useful stuff, have always seen the value in tests but, unit testing everything used to drive me mad. Since I moved to BDD my code has improved quite a bit, I am still a newb at it though so these videos really help out. Cheers
Just a little correction: in Python, you can actually drop the 'test_' part at the beginning of the test function/method name, you just to make a little configuration of the test framework you used. Here is an exemple with pytest, in a pytest.ini file at the root dir: [pytest] python_functions = should_*
Very good presentation; I thankfully listen to many of Dave's talks. But the ones about BDD don't take away the doubts that I feel around BDD. (1) Why be so strict on not describing the 'how'? It might be very valuable to the users to know and be able to visualise beforehand how they will be interacting with the system. For a GUI it might be essential that it has clickable, not voice-operated controls. For other users, in other circumstances, it might be the other way round. Sometimes, a 'how' requirement screams to be made explicit, and then it is just practical to mention it, instead of working around it in abstract phrases. (2) Examples are simple, trivial, not realistic scenarios The math example about adding two numbers is about the behaviour of a calculator. Why apply BDD and a special template (Gherkin), when we already have math? It adds another level that needs to be described and understood (and thus can be misunderstood). It's a huge leap to take between a calculator and modern GUIs! Most examples I find on the WWW are at this level; they tend to make simple scenarios more complex. (3) In the video, 2 examples in Gherkin style are given that are readable, but then, surprise, they appear to be anti-patterns. I agree; these are exactly the kind of examples that keep me from using BDD. But where are the good examples? The talk ends with an example about starting a car, which is a 'good' example. It's a bit ironic that this example is less readable for humans; it is written in pseudo-code. Also, it is almost self-referencing: 'if the car starts, the engine starts' This is the level of formal communication that we need to interact with computers, but for humans it is not informative. (4) I adhere to the approach to have 'executable specs' instead of 'test cases' But they can never fully replace good old requirements, where the system behaviour is described with all the subtleties that natural language offers. Different phases of software development ask for different levels of specification on the natural - formal dimension. (5) The ubiquitous language For some large companies that maintain long-lived complex products, it might be worth the effort, but for many smaller industries, it is a huge investment to develop and maintain a domain-specific language. (6) BDD has the promise to bridge the gap between business stakeholders and software developers BDD offers a template (Gherkin in this case) in which the requirements should be moulded. Business stakeholders use natural language, diagrams and arm movements to describe what they expect the system to do. It is a huge effort to mould their high-level ideas into the boundaries of the BDD style. Is it worth it to go through this?
for #1. When you create an API, you make a specification for your service/modules/functions/classes. Once you establish a contract with the user, you are not allowed to change it for free. So if you make "how" a part of your API, you will not be able to change it without ruining everything. Moreover, it would require for you to also test the "how". However, there ARE cases when there are side effects that the user has to be aware of. For example, when you are implementing an API for a *stateful* service: that is, an invocation of one function will affect its/other functions' next invocation.
Excellent Excellent points, and I though it was only one who had doubts 1 and 4.The points made in the video sound very cool in theory, but in practice they can only serve as high level guidelines which would become very very optional e.g. not to include the slightest of How part in the spec design. Also the BDD tests will never be able to replace the good old detailed requirement documents.
There seems to be a tension here. I don't understand how you can simultaneously say that you would want your BDD specs to abstract away from commitment to any concrete implementation specifics. (Eg. brain interface to a calculator rather than keyboard). AND that BDD is applicable at smaller scale of individual methods or classes. At some point, down in those finer granularity details of your system, you WILL find commitments to particular implementations. And if you are to TDD or BDD at that level of granularity, the tests will have to talk about those specific commitments. There will have to be references to text-fields and mouse-clicks etc. because these are the things that the programmers are actually working with. It seems to me that it's likely to be precisely BECAUSE people hear someone like you talk about writing your BDD specs abstracted away from these specifics, that they imagine that BDD isn't relevant to the finer grained details.
For me the big idea here, for high-level functional tests or low-level unit tests, is to separate what the system (or code) does, from how it does it. BDD style unit tests test that the code does what you want it to do, while, as far as possible, not being tied to how it does it. This means that there is a difference between the external representation of our code, and the code itself. One of the reasons that people struggle is that they don't think enough about this difference, every function signature should hide something of what happens behind it. The test tests the use of the code, not its implementation. The outcome, not the mechanism.
@@ContinuousDelivery Sure. I agree with that. I'm just pointing out that at some level / granularity of the specification, you WILL end up having to say that the outcome is actually some text that appears in a text-box. And if you put too much emphasis, every time you present BDD, on saying that a spec shouldn't talk about text-boxes, then of course, some people will get the impression that BDD is only for the highest level specifications and not relevant to the methods that actually do have to make text-boxes. The broader issue, in my experience, is that most people understand a clean separation between what and how in principle. And are willing to try for it. But that in messy reality they discover a lot of "intermediate" layers. For example, people want the same button and text-box interface to an activity on both Android, iPhone and the desktop Electron version of the app. But not the micro-service API. So you have to detach the action from the UI. But also have to create a layer of common-to-all-button-and-textbox-UIs stuff, that can be reused across the three cases. Which things are the "whats" within that layer? Or another example is producing some kind of moving average graph of your data. Is that functionality part of the model / business logic / "what" of the app? Or merely part of the view / presentation / contingent "how we display this" of the app? What about the size of the window you are averaging? Is that data just stored temporarily in the view layer? Or is it an essential part of the model? I think this is where people tend to get into more trouble and need more help. Not so much they can't understand the basic principle of separating "how" from "what". But when you get into these intermediate layers, identifying which things in that layer, are the necessary "whats", vs which things are merely the unnecessary "hows".
(Oh, and BTW : I'm not just trying to be awkward, I'd love to see a video on your thoughts tackling that topic ... separating hows from whats in intermediate layers)
Hello Dave, First of all thank you for your very educative, very precise and logical videos. They have been immensely helpful. I am an aspiring Scrum master and in time I may be required to coach my teams that may include my PO and this concept of BDD may come in handy. I would need a favour. Can you please share with me a screenshot or two of a user story case that fully captures the BBD? Thank you.
This is really interesting. This is the way we use to do development testing, although manually, way back in the day. You used client/business language, you did testing based on business user stories. In the past you paid QA teams to run through it. Then you could buy really expensive software that could automate clicks and check for results in the front-end or the underlying database. It seems we are going full-circle here to back to what we did in the 80s and 90s. I have heard of some automated tools that can do this but most are web-based even though a lot of corporate software is still windows-forms style applications. What are some current tools to do this sort of thing?
Hello Dave. Thank you for your videos. A colleague of mine and I have helped teams in the company we work for adopt BDD as collaborative approach of delivering software. At the beginning we misunderstood it, yet through continuous learning, we have managed to get it right. Still now and then people ask questions which I cannot answer because my experience with BDD does not go behind borders of our organisation. I have thought you would could be of more help. Could you comment on them shortly, please? Thank you. 1. Question: How do you see the scope of applicability of BDD? What are the types of software development problems for which you would not recommend the BDD approach? 2. Trying to generalize the applicability of BDD beyond problems involving human behaviors, one could consider instead of humans some abstract agents (e.g. machines or processes). Instead of user stories we would have then, say, some “process stories”: representations of how the software should work from the point of view of some processes. Wouldn’t this lead to the generic notion of “software feature”? From this perspective it might seem that Feature Driven Development (FDD) is in a way a generalization of BDD (organizing the work according to predefined feature slices). Any comments on this perspective?
1. I think that BDD is a broadly applicable idea. Fundamentally it is about using automated tests as specifications rather than as tests. To do this the prime directive of BDD, IMO, is to ensure that our specs only say "what" the system should do, without saying anything about "how" it does it. This works for nearly all automated tests. There are times when BDD isn't enough, but I think it is always applicable. It is not good as the only way to test very graphical systems, you can't really write a behavioural spec for "does it look nice". You can write a spec for "If I shoot the bad guy he explodes" that is a behaviour, but what it looks like when he explodes is not really. So I'd use BDD for pretty much everything, but add other forms of tests for some things. 2. The key is what I said, create specifications from the perspective of an external "user" of the system you are testing. If that "user" is some code, that is fine, it is the outside perspective that is the important point. BDD works fine for VERY technical things. There are only 2 problems, 1) BDD is not the same as tools, so the ideas work everywhere, the tools may not be correct. I probably wouldn't use Gherkin for testing embedded devices. 2) You have to be even more disciplined as a team when dealing what technical things. The language of the problem domain, the language you should express your specs in, is now a technical one, so you must be laser focused on keeping "what" separate from "how". It is now much easier to slip into bad habits and start writing specs from the perspective of you the producers, rather than from a consumer of your system. Always abstract the interactions in your specs. Never write a spec that describes anything about "how" your system works.
Agreed. The example is bad and misleading. Ideally you would assert on the direct output, and you would use mock objects to observe indirect output (see the Test Doubles guide of xUnit Patterns). Every other use of a mock object most likely means you are probably doing it wrong. Mocking is normally a code smell in unit tests.
Many thanks for this video. It's a bit surprising that, after so many years since the introduction of BDD, most organisations seem not to be able to follow the guidelines you are talking about. I wonder if the 'intent over implementation details' guideline your mention at ua-cam.com/video/zYj70EsD7uI/v-deo.html requires certain mindset that comes with experience - maybe it's simply hard to apply if one never experienced what good looks like...
I am starting to think that you kinda have to do things wrong yourself to see that there is value in the, sometimes, extra thinking to avoid doing things wrong. You have to see the cost of the downside to understand the value of the investment in thinking.
I have been trying to stop my team from testing implementation details and start testing the result or behaviour. No one cares if you opened a database connection, no one cares that you ran this or that method, all we care about is the result correct for a given input
why not add a cup of birtish tea, a confy chair so you can present your slides in a more relaxed way, and also: you could make an awsome podcast with special invites! amazing content thanks
BDD = steering development by focusing on externally visible outcomes (behaviours) rather than implementation detail. We start work by creating an “executable specification” for the behaviour we want.
This must be the 30th time I watch this video because I absolutely lose at the “clickety clickety” part 🤣 It’s awesome It comes across like you’re kind of a Simon Cowell of software development: “yea with those highly coupled acceptance tests, it’s going to be a no from me”
Thanks, I am pleased that you liked it. If you liked that one, you may also like these that also explore BDD in a bit more detail: "Acceptance Testing with Executable Specifications" ua-cam.com/video/knB4jBafR_M/v-deo.html "BDD Better Executable Specifications" ua-cam.com/video/5CXSEINRojM/v-deo.html and "How to Write Acceptance Tests" ua-cam.com/video/JDD5EEJgpHU/v-deo.html
Working in team, this seems good to learn about the code. Working alone, I guess the code would be too splitted, which can make it kind of tiresome to travel up and down, when debugging or communicating the too many f()s. I read a 1k-pages book of a programmer the used to write this way. Too many pages reading each step. Looks beautiful, but tiresome.
Still so many QA engineers take it as given that BDD should be used in the QA chain. BDD means BD*Development* - so we are not on the QA testing level. Cucumber is one of the worst examples here where so many QA guys think they have to take over this silly "language/syntax" just to get along with the assumption that POs or PMs would write test cases for their product. They will NOT. Period. Over my last 20+ years of QA I never met a single PM/PO who even know what Cucumber is. Developers should concentrate on their unit tests. QA guys should concentrate on their domain. My advice, QA fellows: write your own code for any automated tests, any programming language you prefer, no matter. Be as autonomous as possible. Forget Cucumber & co.
Yes, this was one of my first videos, and I have learned a lot abut sound production since then. I hope you try, and enjoy, one of my recent videos - maybe TDD v BDD ua-cam.com/video/Bq_oz7nCNUA/v-deo.html if you are interested in this topic. Thanks for the feedback.
As a big fan of logic, I must say the way you structure and communicate your thinking is extraordinary. I even notice certain terms and definitions that stay consistent across the videos without ever feeling it’s for the sake of technical buzzwordery. A conversation with you must be a fascinating experience!
Thank you! I think that I have a mostly rational, mostly consistent approach to software development. I think that helps with explaining my ideas, because my view, at least inside my head, is consistent. I believe that I have "strong opinions, lightly held". I am easy to convince that I am wrong, if you have evidence or a stronger theory than mine, but I try to not change my mind based on emotion, rhetoric, or sometimes abuse :)
I think you may overstate my conversational skills, I am good on a few nerdy topics and rubbish at chat :)
Your content is similar to what I was taught at the UQAM & ETS in Montreal during my Computer Science master degree. It's really high quality content man.
“You should consciously avoid any knowledge of implementation in the test case” - wise words :-)
Thanks,
“The only true wisdom is in knowing you know nothing.”
― Socrates
:)
I once worked with a Senior QA in a small test team that that was adamant about adding implementation details in test cases, for software that could change from release to release.
Let's just say, we didn't get along :P
this channel is the prime example of we, people, are more interested in entertaining content rather than knowledge.
It was beneficial to gain an understanding of the importance of providing a detailed software specification that outlines the functionality of the software, rather than focusing on the methodology used to achieve these functions. 👍🏻👍🏻
Subscribing! Just watched three videos. There's always lots of content of simple "follow what I do" coding tutorials, but these kinds of knowledge-based, approachable mini-lectures actually help you become a better programmer in practice.
Thank you! My aim is to raise issues and discuss ideas, so Welcome aboard!
Thank you for this video! As a relative newcomer to the industry (in 2014), I am fascinated to see the parallel between in this video (2:42) "BDD meant test-driven development done right" and in Ian Cooper's talk "TDD, Where Did It All Go Wrong" (also available on UA-cam) where at 22:00 (in that video, not this one) Ian Cooper shows that Kent Beck's original formulation of TDD was always about behavior to begin with! Thank you again, I really appreciate being able to piece together some of the history of how we got the terms and practices that we have today.
in aerospace, requirements validation based vehicle mission scope is required. everything else flows down from the requirements during development. subsystems are then functionally broken down and contracted out.
useful info.
thanks for sharing.
I get so much value from these videos! Thank you so much! 🙏🏻
Thanks for the video and enjoyed the presentation. As a 25+ year tester this creates about a million questions for me......
You may like this one too, on "When BDD goes Wrong": ua-cam.com/video/YAZr3LsCzn0/v-deo.html
@@ContinuousDelivery Thanks - I am really keen to understand the non-functional relationship here also. As we know, this can come in many shapes and sizes and is not just the various flavours of performance testing.
@@pauld1987 My preference is to try and capture NFRs as FRs. A NFR is just something a user cares about in disguise. So try and get to the idea that the user cares about. For example, when building an exchange we had stories like "A Trader submits a trade and the order-history service fails, but when the order-history service recovers, the order is still correctly recorded".
@@ContinuousDelivery I wouldnt argue against the point you make. I guess the devil is in the detail and the key point here is who the users are. For example, would a business user care where I fail over from and to? They would care about things like RTO and RPO I agree. Been quite a while since I have had an interesting testing conversation.. your videos have certainly got the juices flowing again.. thank you
He is essentially just describing a way to fulfill the DO-178 objectives without all of the mandatory rigor. DO-178 requires multiple layers of requirements written in plain language from which all code and tests must be written. I think following DO-178 would help a lot of software teams even if they don’t fulfill all objectives (specifically the independence objectives).
Thank you again for sharing your understandings in such a clear manner.
Thinking about the WHAT not the HOW just completely changed my understanding of TDD/BDD. 🤯
Excellent explanation with 100% clarity on BDD. Thank you so much !
Concise and clear - thanks! Interesting to hear the origins and original intent of BDD. Subscribed and shared with my colleagues.
As I say in the video, I think that the three Amigos, and Natural language thing is great, but knowing the history, at least for me, also broadens the applicability.
Hello Robert, I am an aspiring Scrum master and in time I may be required to coach my teams and that may include my PO and this concept of BDD may come handy.
I would need a favour.
Can you please share with me a screenshot or of a user story case that fully captures the BBD ?
I use "should" in my test cases too, but I tend to have more context. They're usually of the form: methodXYZ_shouldDoThis_WhenThatIsTheCase(). When I start testing a new method, I start a "template" with:
@Test
public void test() throws Exception {
// Given
// When
// Then
}
And I start filling in details. Once that test is done, I repeat with another one.
Lately I've been writing a lot of characterization/approval tests. I sometimes find behaviors that feel wrong to me. Since I'm usually not an expert in the the code, I don't want to change the behavior. But I use the test to confirm the questionable behavior. I'll change the name of the test slightly by appending SHOULD_IT to the test name such as: methodXYZ_shouldDoThis_WhenThatIsTheCase_SHOULD_IT().
I'll consult with the team and/or create a ticket. The SHOULD_IT tests helps to reproduce the questionable behavior. If it is the desired behavior, then remove "SHOULD_IT" from the test name and we're done. If it is incorrect behavior, then copy the SHOULD_IT test case, change it to confirm the correct behavior, which should fail when executed. Update the code so that the new test passes and the SHOULD_IT test case fails. Then remove the whole SHOULD_IT test case.
My method template starts with:
@Test
public void should() throws Exception
{
}
:)
Spec /= Test rather Client-Supplier relation specification (rules based on logical assertions)
Scenario /= Test Case rather Client accesses Supplier
Given / When / Then --> Precondition / Process / Post-condition = {P}C{Q}
The Design-by-Contract assertions are the "tests" we are looking for.
Your explanations are helpful, and revealing. In the case of BDD, I am more solidly believing that BDD = Design by Contract.
The most interesting "behaviors" are always between Clients and Suppliers (both human and machine). This relationship is always governed by assertions that define the Rights and Obligations of Clients and Suppliers. Software defects are the experience of a Right or Obligation that is unmet (or violated assertion).
A defect in the Client happens because it does not meet its Obligations to the Rights of the Supplier when called (preconditions).
A defect in the Supplier happens because the Supplier does not meet its Obligations to the Rights of the Client-that is-to produce precisely what the Client needs and when it needs it (post-conditions).
Only the Client-Supplier relationship matters. As long as the Suppliers meet the needs of the Clients, then a system is without defect. Only when the rules (Right and Obligations) of this relationship are not met or followed do we have an interesting defect.
We are left with but a few observations:
1. We might have an unspecified Right or Obligation. If we miss such a specification, we open ourselves to undetected defects.
2. We might have an incorrect specification of a Right or Obligation. This is a defect that will allow improper behavior and outcomes.
3. We have complete, timely Client-Supplier rules, which can detect failures in the Client (broken precondition rule) or Supplier (broken post-condition rule) directly.
Therefore, Design by Contact is the fulfillment of BDD and is the specification language you are looking for.
don't unspecified and incorrect specifications, both allow improper outcomes that are undetected?
His idea of removing things like providing where the response comes to is new, to me at least. This is a really good and thoughtful way to approach it. Makes the business fit test more durable outside the particular implementation.
This video was incredibly enlightening.
Thank you so much for this content :)
Glad it was helpful!
Dave your channel is cool breez for many. It is awesome.
Just bought the Pipeline book and it is very well written.
Glad you enjoy it!
Very Informative video. Every word you spoke felt like you were speaking with your experience.
Keep up. :)
BDD is Ubiqutous language !
That's it...
Thanks for the core point
@11:00 More durable? Perhaps, but certainly not reusable across multiple scenarios. While I do prefer more abstracted scenario write ups, in practice the steps of those scenarios often prove completely unusable across multiple scenarios. There is a place for specific, granular steps - especially in a browser testing.
I am afradi that I disagree. I think that I explain what I mean by the separation more clearly in this video ua-cam.com/video/JDD5EEJgpHU/v-deo.html
I think it important to separate "what" from "how", of course the testing has to evaluate at the level of "how" the system works, but the test cases don't. Abstracting them so that they capture "what the system should do" from a user's perspective gives us a lot more freedom for the system to change without breaking the test cases. It may break the tests, but that will be fixed in the, shared, plumbing rather that "test-case by test-case".
@Continuous Delivery I'm afraid reality has a habit of intruding on the idea you're aiming for. For example, my application has a transaction page with a table on it. The step to check that table is "I expect the transaction table to show these values:" with a Cucumber data table. But our application is used by multiple brands, each with their own layouts to the transaction table. As a result the step definition now has to clarify which transaction table.
Still, the real problem with your approach is the step definitions can never be used outside the scenario they appear in. This makes tools such as Jira's BDD feature file builder useless.
@@michaelmorris4515 I am afraid that none of your assumptions here are true. First, this is not a theoretical approach, I, and many others, have used this on big complex real-world systems. One of my clients used this approach to test medical devices in hospitals. Another uses it to test scientific instruments. I built one of the world's highest-performance financial exchanges using this approach, and I found out this week, that the tests are still working and providing value 13 years later.
I think that your example focuses on the technicalities rather than the behaviour. "I expect the transaction table to show these values" sounds to me like you are leaking implementation detail into your test cases, and that is why they are fragile. What is it that the user really wants? Do they really care about "transaction tables" when they walked up to the computer to do a job, were they thinking "what I need to do is make sure that the transaction table shows these entries"? I doubt it.
I can't give you a real example, because I don't know what your app does, but I'd try an capture the intent that the user had. So forgive me for making something up, but lets say that in your case a "transaction" represents selling something, and your "transaction table" represents a list of things, or services, sold. Then I can think of a few scenarios that matter to a user. "I want to be able to buy something and see that I have bought it" (it ends up in the "transaction table"). "I'd like to be able to buy a few things and see a list of the things that I bought" (they all end up in the transaction table). and so on.
@@ContinuousDelivery What am I missing here? I could rewrite the step to simply be "I expect the transaction table is correct" and not mention the values in the feature file at all - but the step can't be freely reused at that point (well it can, but not without revising the implementation code each time).
What I'm perceiving that I don't like is the inability to reuse the code at all.
@@michaelmorris4515 My approach is to build a Domain Specific Language (DSL) that allows you to capture the behavioural intent of your users. It is perfectly acceptable to capture parameters at that level, as long as they make sense to the user. For example, if I was writing a test for buying books on Amazon, I could imagine adding the facility in my DSL to specify the name of the book, the author of the book and how many copies I wanted. My DSL would probably default these things if I didn't supply them, but I Could be as precises as I like if I need to test detail. But, the language is ONLY the language of the user, not of the system. The technical detail of 'how' the system works is in lower-layer of the test infrastructure, I call these "protocol drivers" which take ideas like "buyABook" and translate them into "put this value in this field, put that value in that field and press this button". That translation is generic for the concept of "buyingBooks" and so is shared with every test that "buysBooks".
I can't thank you enough for this video.
Thanks for these videos Dave, really useful stuff, have always seen the value in tests but, unit testing everything used to drive me mad. Since I moved to BDD my code has improved quite a bit, I am still a newb at it though so these videos really help out. Cheers
Thanks, I am pleased that you have found them helpful.
Just a little correction: in Python, you can actually drop the 'test_' part at the beginning of the test function/method name, you just to make a little configuration of the test framework you used.
Here is an exemple with pytest, in a pytest.ini file at the root dir:
[pytest]
python_functions = should_*
Great video as always! It is that developers focus too much on tools rather than actually understanding the ideas themselves and their benefits,
a great intro talk to BDD
Very good presentation; I thankfully listen to many of Dave's talks.
But the ones about BDD don't take away the doubts that I feel around BDD.
(1) Why be so strict on not describing the 'how'?
It might be very valuable to the users to know and be able to visualise beforehand how they will be interacting with the system.
For a GUI it might be essential that it has clickable, not voice-operated controls.
For other users, in other circumstances, it might be the other way round.
Sometimes, a 'how' requirement screams to be made explicit, and then it is just practical to mention it, instead of working around it in abstract phrases.
(2) Examples are simple, trivial, not realistic scenarios
The math example about adding two numbers is about the behaviour of a calculator.
Why apply BDD and a special template (Gherkin), when we already have math?
It adds another level that needs to be described and understood (and thus can be misunderstood).
It's a huge leap to take between a calculator and modern GUIs!
Most examples I find on the WWW are at this level; they tend to make simple scenarios more complex.
(3) In the video, 2 examples in Gherkin style are given that are readable, but then, surprise, they appear to be anti-patterns.
I agree; these are exactly the kind of examples that keep me from using BDD.
But where are the good examples?
The talk ends with an example about starting a car, which is a 'good' example.
It's a bit ironic that this example is less readable for humans; it is written in pseudo-code.
Also, it is almost self-referencing: 'if the car starts, the engine starts'
This is the level of formal communication that we need to interact with computers, but for humans it is not informative.
(4) I adhere to the approach to have 'executable specs' instead of 'test cases'
But they can never fully replace good old requirements, where the system behaviour is described with all the subtleties that natural language offers.
Different phases of software development ask for different levels of specification on the natural - formal dimension.
(5) The ubiquitous language
For some large companies that maintain long-lived complex products, it might be worth the effort, but for many smaller industries, it is a huge investment to develop and maintain a domain-specific language.
(6) BDD has the promise to bridge the gap between business stakeholders and software developers
BDD offers a template (Gherkin in this case) in which the requirements should be moulded.
Business stakeholders use natural language, diagrams and arm movements to describe what they expect the system to do.
It is a huge effort to mould their high-level ideas into the boundaries of the BDD style. Is it worth it to go through this?
for #1. When you create an API, you make a specification for your service/modules/functions/classes. Once you establish a contract with the user, you are not allowed to change it for free. So if you make "how" a part of your API, you will not be able to change it without ruining everything. Moreover, it would require for you to also test the "how".
However, there ARE cases when there are side effects that the user has to be aware of. For example, when you are implementing an API for a *stateful* service: that is, an invocation of one function will affect its/other functions' next invocation.
Excellent Excellent points, and I though it was only one who had doubts 1 and 4.The points made in the video sound very cool in theory, but in practice they can only serve as high level guidelines which would become very very optional e.g. not to include the slightest of How part in the spec design. Also the BDD tests will never be able to replace the good old detailed requirement documents.
All good and well but at the end of the day some automated GUI/E2E test SHOULD test if a textbox is filled in.
There seems to be a tension here.
I don't understand how you can simultaneously say that you would want your BDD specs to abstract away from commitment to any concrete implementation specifics. (Eg. brain interface to a calculator rather than keyboard). AND that BDD is applicable at smaller scale of individual methods or classes.
At some point, down in those finer granularity details of your system, you WILL find commitments to particular implementations. And if you are to TDD or BDD at that level of granularity, the tests will have to talk about those specific commitments. There will have to be references to text-fields and mouse-clicks etc. because these are the things that the programmers are actually working with.
It seems to me that it's likely to be precisely BECAUSE people hear someone like you talk about writing your BDD specs abstracted away from these specifics, that they imagine that BDD isn't relevant to the finer grained details.
For me the big idea here, for high-level functional tests or low-level unit tests, is to separate what the system (or code) does, from how it does it. BDD style unit tests test that the code does what you want it to do, while, as far as possible, not being tied to how it does it. This means that there is a difference between the external representation of our code, and the code itself. One of the reasons that people struggle is that they don't think enough about this difference, every function signature should hide something of what happens behind it. The test tests the use of the code, not its implementation. The outcome, not the mechanism.
@@ContinuousDelivery Sure. I agree with that.
I'm just pointing out that at some level / granularity of the specification, you WILL end up having to say that the outcome is actually some text that appears in a text-box.
And if you put too much emphasis, every time you present BDD, on saying that a spec shouldn't talk about text-boxes, then of course, some people will get the impression that BDD is only for the highest level specifications and not relevant to the methods that actually do have to make text-boxes.
The broader issue, in my experience, is that most people understand a clean separation between what and how in principle. And are willing to try for it. But that in messy reality they discover a lot of "intermediate" layers.
For example, people want the same button and text-box interface to an activity on both Android, iPhone and the desktop Electron version of the app. But not the micro-service API. So you have to detach the action from the UI. But also have to create a layer of common-to-all-button-and-textbox-UIs stuff, that can be reused across the three cases. Which things are the "whats" within that layer?
Or another example is producing some kind of moving average graph of your data. Is that functionality part of the model / business logic / "what" of the app? Or merely part of the view / presentation / contingent "how we display this" of the app? What about the size of the window you are averaging? Is that data just stored temporarily in the view layer? Or is it an essential part of the model?
I think this is where people tend to get into more trouble and need more help. Not so much they can't understand the basic principle of separating "how" from "what". But when you get into these intermediate layers, identifying which things in that layer, are the necessary "whats", vs which things are merely the unnecessary "hows".
(Oh, and BTW : I'm not just trying to be awkward, I'd love to see a video on your thoughts tackling that topic ... separating hows from whats in intermediate layers)
Hello Dave,
First of all thank you for your very educative, very precise and logical videos. They have been immensely helpful.
I am an aspiring Scrum master and in time I may be required to coach my teams that may include my PO and this concept of BDD may come in handy.
I would need a favour.
Can you please share with me a screenshot or two of a user story case that fully captures the BBD? Thank you.
How does Ubiquitous Language differ from, say, Gane and Sarson's Data Dictionary?
It is not only about data.
This is really interesting. This is the way we use to do development testing, although manually, way back in the day. You used client/business language, you did testing based on business user stories. In the past you paid QA teams to run through it. Then you could buy really expensive software that could automate clicks and check for results in the front-end or the underlying database. It seems we are going full-circle here to back to what we did in the 80s and 90s. I have heard of some automated tools that can do this but most are web-based even though a lot of corporate software is still windows-forms style applications. What are some current tools to do this sort of thing?
Great video. Really love the channel
Thank you
Hello, I have a question :) where should be the technical details about user experience given it is not in the behavior specification?
Separate from the requirements. It is a design choice, not a requirement, so it comes later in the process.
Hello Dave. Thank you for your videos. A colleague of mine and I have helped teams in the company we work for adopt BDD as collaborative approach of delivering software. At the beginning we misunderstood it, yet through continuous learning, we have managed to get it right. Still now and then people ask questions which I cannot answer because my experience with BDD does not go behind borders of our organisation. I have thought you would could be of more help. Could you comment on them shortly, please? Thank you.
1. Question: How do you see the scope of applicability of BDD? What are the types of software development problems for which you would not recommend the BDD approach?
2. Trying to generalize the applicability of BDD beyond problems involving human behaviors, one could consider instead of humans some abstract agents (e.g. machines or processes). Instead of user stories we would have then, say, some “process stories”: representations of how the software should work from the point of view of some processes. Wouldn’t this lead to the generic notion of “software feature”? From this perspective it might seem that Feature Driven Development (FDD) is in a way a generalization of BDD (organizing the work according to predefined feature slices). Any comments on this perspective?
1. I think that BDD is a broadly applicable idea. Fundamentally it is about using automated tests as specifications rather than as tests. To do this the prime directive of BDD, IMO, is to ensure that our specs only say "what" the system should do, without saying anything about "how" it does it. This works for nearly all automated tests. There are times when BDD isn't enough, but I think it is always applicable. It is not good as the only way to test very graphical systems, you can't really write a behavioural spec for "does it look nice". You can write a spec for "If I shoot the bad guy he explodes" that is a behaviour, but what it looks like when he explodes is not really. So I'd use BDD for pretty much everything, but add other forms of tests for some things.
2. The key is what I said, create specifications from the perspective of an external "user" of the system you are testing. If that "user" is some code, that is fine, it is the outside perspective that is the important point. BDD works fine for VERY technical things. There are only 2 problems, 1) BDD is not the same as tools, so the ideas work everywhere, the tools may not be correct. I probably wouldn't use Gherkin for testing embedded devices. 2) You have to be even more disciplined as a team when dealing what technical things. The language of the problem domain, the language you should express your specs in, is now a technical one, so you must be laser focused on keeping "what" separate from "how". It is now much easier to slip into bad habits and start writing specs from the perspective of you the producers, rather than from a consumer of your system. Always abstract the interactions in your specs. Never write a spec that describes anything about "how" your system works.
@@ContinuousDelivery Thank you for your answer Dave. I am going to share it with my colleagues.
In the Python unit example, isn't the _Then_ case ```mock_engine.start.assert_called_once``` a prime example of testing implementation details?
Agreed. The example is bad and misleading. Ideally you would assert on the direct output, and you would use mock objects to observe indirect output (see the Test Doubles guide of xUnit Patterns). Every other use of a mock object most likely means you are probably doing it wrong. Mocking is normally a code smell in unit tests.
Thanks for the video =)
Many thanks for this video. It's a bit surprising that, after so many years since the introduction of BDD, most organisations seem not to be able to follow the guidelines you are talking about. I wonder if the 'intent over implementation details' guideline your mention at ua-cam.com/video/zYj70EsD7uI/v-deo.html requires certain mindset that comes with experience - maybe it's simply hard to apply if one never experienced what good looks like...
Yes, "what not how" seems pretty obvious once you have got it wrong a few times :)
I am starting to think that you kinda have to do things wrong yourself to see that there is value in the, sometimes, extra thinking to avoid doing things wrong. You have to see the cost of the downside to understand the value of the investment in thinking.
I have been trying to stop my team from testing implementation details and start testing the result or behaviour. No one cares if you opened a database connection, no one cares that you ran this or that method, all we care about is the result correct for a given input
Yes, precisely!
@@ContinuousDelivery Please do a video on roles and responsibilities of a senior developer. I'd like to hear your view points
@@rajm1976 Did you see this one? ua-cam.com/video/jMpCF0Z623s/v-deo.html
Nicely explained.
Thank you 🙂
Nothing about NLP in the description😁😁
why not add a cup of birtish tea, a confy chair so you can present your slides in a more relaxed way, and also: you could make an awsome podcast with special invites! amazing content thanks
Always up for a cup of tea :) Thanks.
Strictly speaking, Gherkin isn't a language, it's a syntax. There's no Gherkin API, just some reserved words and a supported syntactical structure.
Fair enough!
In a few sentences what is BDD? I ask because I have looked at a lot of sources and I can't see a definitive definition.
BDD = steering development by focusing on externally visible outcomes (behaviours) rather than implementation detail. We start work by creating an “executable specification” for the behaviour we want.
@@ContinuousDelivery Thank you.
This must be the 30th time I watch this video because I absolutely lose at the “clickety clickety” part 🤣 It’s awesome
It comes across like you’re kind of a Simon Cowell of software development: “yea with those highly coupled acceptance tests, it’s going to be a no from me”
🤣
Awesome Video
Thanks, for watching. I am thinking of doing a fair bit more on BDD
Very helpful!
Thanks, I plan to do more on BDD in future, so stay tuned :)
Great video. thanks :)
Thanks, I am pleased that you liked it.
If you liked that one, you may also like these that also explore BDD in a bit more detail:
"Acceptance Testing with Executable Specifications" ua-cam.com/video/knB4jBafR_M/v-deo.html
"BDD Better Executable Specifications" ua-cam.com/video/5CXSEINRojM/v-deo.html
and
"How to Write Acceptance Tests" ua-cam.com/video/JDD5EEJgpHU/v-deo.html
@@ContinuousDelivery thank you
You are welcome :)
Working in team, this seems good to learn about the code. Working alone, I guess the code would be too splitted, which can make it kind of tiresome to travel up and down, when debugging or communicating the too many f()s.
I read a 1k-pages book of a programmer the used to write this way. Too many pages reading each step. Looks beautiful, but tiresome.
Toddler developer here, Just did Java fundamentals, some OOP concepts and here I am watching TDD, BDD. WTF is wrong with me ??!!
Still so many QA engineers take it as given that BDD should be used in the QA chain.
BDD means BD*Development* - so we are not on the QA testing level.
Cucumber is one of the worst examples here where so many QA guys think they have to take over this silly "language/syntax" just to get along with the assumption that POs or PMs would write test cases for their product. They will NOT. Period. Over my last 20+ years of QA I never met a single PM/PO who even know what Cucumber is. Developers should concentrate on their unit tests. QA guys should concentrate on their domain.
My advice, QA fellows: write your own code for any automated tests, any programming language you prefer, no matter. Be as autonomous as possible. Forget Cucumber & co.
perfect
Thank you 🙏
Hello
I'm sure this is a good video, but there's just a LOT of static microphone noise going on. It's not very pleasant to listen to.
Yes, this was one of my first videos, and I have learned a lot abut sound production since then. I hope you try, and enjoy, one of my recent videos - maybe TDD v BDD ua-cam.com/video/Bq_oz7nCNUA/v-deo.html if you are interested in this topic.
Thanks for the feedback.
You are confusing SHOULD and MUST. Requirements are MUSTs. SHOULD is advice.
Ok
*Bitcoin Diving Down*
Talks a lot of fluff