Stop Simulating BDD, Write Tests

I keep receiving one question. Well, actually two: “What makes Codeception different comparing to Behat“. And the next one: “Why can’t I write Codeception tests in human language like Behat does”. I think the time has come to answer this questions.

As you know, the Behat project is not only a tool but a methodology. The Behavior Driven Development concept requires that the User Stories to be written, and then executed as a test. Behat is a right tool for teams who strictly follow the BDD methodology. But what if you don’t? For what in the world you need tests written in plain English?

If you write a story like this, using Behat and MinkContext for example,

Given I am on "/product/1"
When I click "Purchase"
And I fill in "name" with "Michael"
And I fill in "credit" with "321123123"
And I click "Order"
Then I should see "The product was purchased!"

you actually write test, and not a story. Because the feature you describe won’t change If you rename field “credit” to “credit card”, but the test will fail. You may change the text “The product was purchased” to “iPhone5 was purchased. Thank you!” and that will make test fail too. So you will rewrite the user story every time you change every detail that may affect passing test. In this case the manager who wrote this spec will debug and update user story to make the test pass.

And so your manager becomes tester. His role is now not only to deliver specifications to team but also to make this specifications written as tests and make them pass.

But what If you want one additional step to the scenario?

Given I am "/product/1"
When I click "Purchase"
Then I should see "1 product" in the bin

For this step see "1 product" in the bin you should write a custom step. If only your manager knew PHP for it! Ideally to become manager + tester + PHP guru. But no, manager needs to ask your developer to write the necessary step. Thus, already two people are writing the test. And one of them knows PHP. And when you will hire a QA will you require a knowledge of PHP? Well, yep. Thus because you don’t want to disturb your PHP devs and assign them additional tasks.

When your product growths your tests becomes more and more complex. And you need to keep them updated for each change. When you use the plain English text, you don’t have a control over tests. When your form don’t have a label you need to write custom step instead of I fill in, because it’s actually wrong to use CSS or XPath selectors inside a Behat feature. It’s not plain English text anymore with CSS. You need PHP developers to create additional steps.

Why Codeception is better testing tool? Well, just because it is a testing tool at first. Spend a day, to teach your manager/tester/qa PHP DSL, install a Netbeans or PHPStorm (or any other IDE), and start writing tests. Even when tests looks similarly to Behat features they give more power and control over the process.

<?php
$I = new WebGuy($scenario);
$I->amOnPage('/product/1');
$I->click('Purchase');
$I->fillField('name', 'Michael');
$I->fillField('credit', '321123123');
$I->click('Order');
$I->see('The product was purchased!');

This test in the process of evolution can be transformed into something more complex. You can use CSS and XPath everywhere, so when you change the “Order” button with an image, you can use it’s CSS and quickly update a test to make it pass. Also, you can move repeatable elements into variables and classes and make a test that looks like this:

<?php
$I = new WebGuy($scenario);
$I->amOnPage(ProductPage::URL);
$I->click('#purchase');
$I->fillField('name', $user_name);
$I->fillField('credit', $user_credit);
$I->click('img a#order');
$I->see(ProductPage::$successMessage);

Just the same scenario. Well it’s a bit harder to read, as we added CSS selectors, but as you see this test pretty much more flexible. It can be easily refactored and improved without writing any custom method to helper.

Don’t fall into a marketing trap. You will find yourself writing tests two times: as a feature in plain English and in code with PHP.

It’s better to have one person who can take a full control over test automation then, delegate some tasks to developers, some tasks to managers. If you are developer and you work on your own project. Why PHP is not suitable for you? Why do you want to write code 2 times: once in feature with Gherkin and second time in Context. Is it KISS? Is it DRY? I don’t think so.

Will Codeception support plain text test scenarios? Really, I thought on that. But I don’t want to put the limits. Using plain text makes impossible to use variables, loops, grabbers, and thus use PageObject pattern. Which is very important for solid and concrete test automation platform.

Behat is great BDD tool and it’s author @everzet also states: don’t use it for functional testing. Choose the right tool for job.

Disclaimer: I’m not very good in theory, I’m very practical guy. I won’t refer you to any published books or methodologies. But If you want an authority… Well, David Heinemeier Hansson (the creator of Rails) wrote in his blog: 1”Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests”. And Behat is Cucumber for PHP. DHH is known for his dislike to popular BDD tools like RSpec or Cucumber. But he is very pragmatic and very scrupulous in testing.