Are you the publisher? Claim or contact us about this channel


Embed this content in your HTML

Search

Report adult content:

click to rate:

Account: (login)

More Channels


Showcase


Channel Catalog


Articles on this Page

(showing articles 1 to 50 of 50)
(showing articles 1 to 50 of 50)

Channel Description:

Codeception: BDD-style testing in PHP
    0 0
  • 06/23/13--22:03: Codeception 1.6.3 is out
  • Update: Yii2 module was included in release. See details below.

    It was a one-month release sprint and probably this release took much longer then we expected. 1.6.3 release introduces lots of features, internal unit tests, and internal refactorings. Please take a cup of coffee because we prepared a long story of features you can try in 1.6.3.

    The major internal change you should be aware of is upgrade to Symfony Components 2.3. This may introduce some instability, or conflicts (if you use previous Symfony version), but the good news are that: Symfony 2.3 is LTS, so we are going to stay on this version for a long time.

    One-Runner

    This feature was expected by Symfony2 users, as well as other developers who uses several independent applications in a project (as Symfony bundles are). From now on you are not limited to execute tests from only one directory. In Codeception 1.6.3 you can create a meta-config and make it to point to other codeception test configs:

    include:
      - frontend
      - admin
      - api/rest
    paths:
      log: log
    

    In this codeception.yml we include tests configs from frontend, admin, and app/rest directories. By executing tests from it, we will launch all test suites from this applications.

    More about it...

    Symfony2 users now can create a test suite for each bundle they have and then execute all tests in meta config.

    Namespaces

    If you decided to use a multiple-config option, the first issue you get is conflict of Helper classes and Guy classes. It's natural to use namespaces to separate those classes of multiple applications. From now on, you can assign a namespace for your test classes.

    Facebook Module

    Thanks to Tiger-seo we got a module that might be very handy for all developers that work with Facebook API.

    <?php
    $I = new ApiGuy($scenario);
    $I->am('Guest');
    $I->wantToTest('check-in to a place be published on the Facebook using API');
    $I->haveFacebookTestUserAccount();
    $accessToken = $I->grabFacebookTestUserAccessToken();
    $I->haveHttpHeader('Auth', 'FacebookToken ' . $accessToken);
    $I->amGoingTo('send request to the backend, so that it will publish on user\'s wall on Facebook');
    $I->sendPOST('/api/v1/some-api-endpoint');
    $I->seePostOnFacebookWithAttachedPlace('167724369950862');
    ?>
    

    Don't leave your Facebook applications untested! With this module you can log in as test user and perform interactions using REST API or HTTP calls.

    Facebook Module Referenace

    Helper Customizations

    Before 1.6.3 Helpers was loaded with simple include_once call. I.e., there was no other option then to load them from tests/_helpers directory. In 1.6.3 we replaced this call with autoloader, and also we added a global _bootstrap.php file in which you can prepend any code before helpers are loaded.

    More about autoloading Helpers.

    Also in 1.6.3 you can customiza which methods of module or helper should be imported into Guy class. In other words, you can enable or disable module methods. If your helper is extended from one of standard modules, you can disable usage of inherited methods and implement them on your own. It is especially useful if you want to rename parent class methods:

    <?php
    class SecondDbHelper extends Db {
        public static $includeInheritedActions = false;
    
        public function seeInSecondDb($table, $data = array())
        {
            $this->seeInDatabase($table, $data);
        }
    }    
    ?>
    

    More on helper customization.

    Stubs with Mock matchers.

    Codeception's utility Stub class got lots of improvements by svsool. Mock matching was also added into it, thus you may use shorter mock syntax in your unit tests:

    <?php
    use \Codeception\Util\Stub;
    // example 1
    $mock = Stub::makeEmpty('SomeClass', array(
        'trackInvocationCount' => 
         Stub::once(function() {return true})),
         $this);
    
    $mock->trackInvocationCount();
    
    // example 2
    $mock = Stub::makeEmpty('SomeClass', array('methodNeverCalled',  Stub::never(), $this);
    ?>
    

    Complete reference is coming soon.

    Execute Call

    In case you need to execute custom code in a test, you can now use the execute method for that:

    <?php 
    $I->execute(function() {
      DB::updateCounters();
    });
    ?>
    

    execute method is declared in Guy class itself, thus it can be used anywhere without attaching a module. As you know, Codeception tests files are included 2 times: one for parsing and for execution. Calls inside execute method will be triggered only on test execution.

    Thanks korotovsky for implementation.

    Yii2 module

    Yii Framework

    Yii framework hits its second major version. Yii 2.0 was developed with testing in mind. Despite the fact that Yii 2.0 is still in its early days, we got a working Yii2 module for it. Unlike most of framework modules of Codepception, this one was created by Yii core developer qiangxue.

    We are excited of collaboration with Yii Framework core team. And we wish them Yii 2.0 project to hit stable release soon.

    Various Ehancements and Fixes.

    • --no-colors runner option by igorsantos07.
    • fix to double text in generate:scenarios.
    • ZF1 sessions and auth helpers fixed by brutuscat
    • seeInDatabase can check that table is empty with: seeInDatabase('empty_table') by igorsantos07.
    • ZombieJS updates by synchrone.

    additionally we had lots of fixes to Db module and its drivers. Sorry, we can't list all of them here.

    As it was said we had internal refactorings, to make code stable and clearer. Soon we plan to publish a complete blog on Codeception internals.

    Update

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    Release sponsored by 2AmigOS

    2AmigOS


    0 0

    Another release that despite the minor version change brings major improvements. Meet 1.6.4, which adds lots of new ways to customize your test automation platform and improve your tests. And yes, before reading this post take a cup of coffee. We prepared lots of features for you and this long post.

    PageObjects

    Long awaited feature of adding PageObjects into the core. Codeception looks pretty different from other testing frameworks in Java or Ruby. So it was hard to understand in which way the PageObject should be implemented.

    Actually you can think of a PageObject that is just a storage of UI locators (UI Map).

    <?php
    class ArticlesPage {
    
        const URL = '/articles';
    
        static $articleList = '#list';
        static $newArticleButton = '#toolbar a.new';
    
        static function row($id)
        {
            return $articleList ." .article-$id";
        }
    }
    ?>
    

    In a test you can use such PageObject this way:

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('find article #1 in a list and edit it');
    $I->amOnPage(ArticlesPage::URL);
    $I->seeElement(ArticlesPage::$articleList);
    $I->click('Edit', ArticlesPage::row(1));
    $I->see('Editing Article #1');
    ?>
    

    But how to make PageObjects actually to define page interaction too? Can we improve that? Sure! In a test they may look like:

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('find article #1 in a list and edit it');
    ArticlesPage::of($I)
        ->visit()
        ->openArticleForEditing(1);
    $I->see('Editing Article #1');
    ?>
    

    We have moved some logic into the PageObject class, and so we can reuse its methods in other tests. Read more on generating PageObjects on newly updated Guides page.

    StepObject

    Alternatively, interaction logic can be kept in StepObject classes. In which we recommend to define actions that may require passing through several pages. Also it may be useful if you want to define actions based on user roles.

    <?php
    $I = new WebGuy\AdminSteps($scenario);
    $I->am('admin');
    $I->wantTo('create a new admin and check his account');
    $I->logIntoAdminArea();
    $I->createUser('davert','123456', 'admin');
    $I->logout();
    $I->login('davert','123456');
    $I->seeInCurrentUrl('/admin');
    ?>
    

    AdminSteps class inherits from WebGuy class, thus you have common actions from modules, as well as newly defined customized actions like createUser, login, logout in your tests.

    StepObjects are now in Guides too.

    Groups && Extensions

    Now it is possible to include 3rd party code into Codeception. Current options are pretty limited, you can extend Codeception only by listening to its internal events. Why do you need that? Not sure. But check out our Notifier extension that we prepared to demonstrate the power of extensions. Also you can develop your own alternative output formatter.

    Group Classes are special extensions, that listen to events from a tests of a specific group. Thus, they are very if some tests require common environment setup. Group classes are also good for loading fixtures.

    Here is the sample group class:

    <?php
    class AdminGroup extends \Codeception\Platform\Group {
    
        static $group = 'admin';
    
        public function _before(\Codeception\Event\Test $e)
        {
            $this->writeln("Preparing for [admin] test...");
        }
    
        public function _after(\Codeception\Event\Test $e)
        {
            $this->writeln("Finishing [admin] test...");
        }
    }
    ?>
    

    Read more about Groups and Extensions.

    Conditional Asserts

    Pretty simple, yet useful feature when you want your test not to be stopped on failure.

    <?php
    $I->canSee('Hello World');
    $I->see('Hello World');
    ?>
    

    This two assertions do just the same, but if canSee fails to match 'Hello World' text on a page, it doesn't stop the test. Still failed assertion will be displayed in final report.

    Guides section about that.

    Assertion Failure Messages Improved

    For most Framework and Mink modules we improved the error messages that happen on failures. No more mystic messages like failed asserting that 0 greater then 0. Better exceptions with better error reports.

    Comments Simplified

    <?php
    $I['comments can be easily added to a test'];
    $I['and displayed in output when executed'];
    $I['and added to HTML reports'];
    $I['pretty cool when you follow BDD or ATDD'];
    $I['describe everything in comments and then automate them'];
    ?>
    

    Minor Fixes and Improvements

    • codecoverage for multiple runner is now stored into right dir, thanks piccagliani.
    • header actions were added to REST module by brutuscat.
    • added environment management to Symfony2 module by SimonEast.

    Update

    It is very important to execute "build" after the update. Also this update has lots of changes, if you have an issues with them, please report them to Github. If you have issues updating, fall back to previous version for now. We are planning to introduce more humane stability politics soon. Wait for announcements.

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0

    WordPress has dozens of plugins. And probably you have developed your own. How would you know your plugin works for everyone? Does it conflicts with others? Does it work after your last change?

    Oh yes, releasing the plugin means to make it reliable. This why we highly recommend to use automated testing in your development. Its not that really hard as you expect. Quite the contrary: automated testing saves your time for clicking and filling forms on a page.

    In this post we will examine how can you test a sample WordPress plugin with Codeception framework. Our patient is User Submitted Posts plugin by Jeffrey Starr.

    This plugin allows regular users to submit a posts to your WordPress blog.

    We assume a plugin is already installed and activated.

    Plugins Activated

    And we created "Submit a Post" page with a plugin snippet included on it. Through this page we expect user stories to be submitted.

    Submit Post

    This plugin has lots of settings that include post fields, available categories, security issues, etc. What we want to do is try to switch different options and try to submit a post. Depending on option switched we will see some changes in the form, moderation or publishing process.

    Automation of testing allows us to write a script one time and replay it whenever a change is introduced. Will the plugin work when WordPress hits new release? Will the plugin work when we introduce a new option? Without automated testing we would spend hours to check the same stuff over and over. Lets spend a few hours writing a proper tests, and save days on manual testing.

    Install Codeception

    Codeception can be easily installed. It requires PHP 5.3 and higher. You can download it here or grab from site with wget:

    wget http://codeception.com/codecept.phar
    

    Downloaded codecept.phar file should be placed into your WordPress directory. This file should be executed with PHP from console

    Console

    To install it run the bootstrap command:

    php codecept.phar bootstrap
    

    This creates a new directory tests with different folders in it.

    Tests

    There are 3 different folders (called suites) to store tests: acceptance, functional, and unit. For our purposes we will need only the acceptance suite. Acceptance tests will replay our actions in browser.

    To speed up the tests we are not using real browser here, but we use its emulator, a so-called PHPBrowser, based on curl calls.

    First Test

    Let's create a test named "SubmitPost". We will need generate:cept command for that.

    php codecept.phar generate:cept acceptance SubmitPost
    

    A newly generated test will look like this:

     
    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('perform actions and see result');
    
    ?>
    

    This PHP code is written in the way you would describe your actions while you are testing a plugin. What we will actually try to check? Let's define a scenario:

    • enter a site
    • go to "submit posts" page
    • fill all fields
    • submit a post
    • check a message for valid submission is shown
    • admin should see the post
    • and publish it
    • post should be seen on site

    Lets write the first step into a test.

     
    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    
    ?>
    

    We moved to the front page of a site:

    WordPress

    To move to the "submit posts" page (our step #2) we will need to click on "Submit a Post" menu item:

     
    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    
    ?>
    

    That's right, everything is as simple as you would tell it to a mate. As we are on page submission page we got a form and a few fields to fill in.

    Submit

    We are going to publish the review for the "Game of Drones" book into related category. Follow the code:

     
    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    $I->fillField('Your Name', 'Michael');
    $I->fillField('Your URL','http://drone-rules.com');
    $I->fillField('Post Title', 'Game of Drones Review');
    $I->fillField('Post Tags', 'review book rob-starkraft');
    $I->selectOption('Post Category', 'Game of Drones');
    $I->fillField('Post Content', 'This story is epic and characters are amazing.');
    $I->click('Submit Post');
    ?>
    

    After a post is submitted a message Success! Thank you for your submission. is shown.

    Sent

    Our test wouldn't be a real test without at least one verification. The result of our current actions is this "Success" message. If we don't see it on a screen, we assume that test failed.

     
    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    $I->fillField('Your Name', 'Michael');
    $I->fillField('Your URL','http://drone-rules.com');
    $I->fillField('Post Title', 'Game of Drones Review');
    $I->fillField('Post Tags', 'review book rob-starkraft');
    $I->selectOption('Post Category', 'Game of Drones');
    $I->fillField('Post Content', 'This story is epic and characters are amazing.');
    $I->click('Submit Post');
    $I->see('Success! Thank you for your submission.');
    ?>
    

    Execute Test

    As we have one assertion with the see command in the end, it's a good idea to try this test in action. As it was said, we will execute this test with the PHPBrowser, i.e. browser emulator. We should point it to the right URL to access our site. For such parameters a config file acceptance.suite.yml exists.

    Config

    We need to start a web server and specify proper local url of wordpress site with a plugin we are testing. We are ready to execute this test with run command.

    php codecept.phar run
    

    And ups...

    TestFailed

    Looks like everything were ok, before we tried to select option Post Category. We saw in previous screen, there was such field in a list, but why WebGuy couldn't find it on a page? It was there.

    Well, here is the answer. WebGuy tries to fetch a form element by its label, whenever a label tag has attribute for that points to the id of a field , we can select this field by label.

    SelectField

    But as you see, unlike the previous Post Tags field, the Post Category select tag doesn't have an id. Thus, the label do not point to it, as we would expect.

    How do we resolve this situation? If we can't match the field by its label, we can probably match it by CSS, pretty similar to how we would do that in jQuery:

     
    <?php
    $I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
    ?>
    

    Let's execute the test once again... And we see it has passed.

    Passed

    Great, still we only verified the "Success" message was shown. We didn't check that admin can actually see the user submitted post. This is where we start the next part of this tutorial.

    The final test code we got today is:

     
    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    $I->fillField('Your Name', 'Michael');
    $I->fillField('Your URL','http://drone-rules.com');
    $I->fillField('Post Title', 'Game of Drones Review');
    $I->fillField('Post Tags', 'review book rob-starkraft');
    $I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
    $I->fillField('Post Content', 'This story is epic and characters are amazing.');
    $I->click('Submit Post');
    $I->see('Success! Thank you for your submission.');
    ?>
    

    As you see it is pretty straight forward. And there is nothing hard in writing such tests. But execution of this test took much less then we would reproduce this steps manually, in a browser. Also we can replay this test in any time. Isn't that a good reason to start testing today?

    In Next Series:

    • We will finish the test to verify that admin can publish user submitted post.
    • We will rework the test make it even more compact and readable.
    • We will learn how to deal with data in Codeception.

    Before we proceed, try to reproduce the following steps locally and prepare the testing environment. You can download this demo project or clone it from GitHub. And try to write and run some tests on your own.


    0 0

    “Nothing is True, Everything is Permitted” William S. Burroughs

    We already announced AspectMock, the mocking framework that may dramatically change the way you do testing in PHP. In this video this Jeffrey Way shows how AspectMock is different from others. In this post we will demonstrate its powers too, and we will try to break some stereotypes about PHP testing.

    To get the code tested, you should always keep in mind how you would write a test for it. We know unit testing requires some good practices to follow and bad practices to avoid.

    For example, you should not use singletons. They are bad. Why? Code that use singletons can't be tested.

    But what if we could mock singletons:

    <?php
    function testSingleton()
    {
        $class = MySingleton::getInstance();
        $this->assertInstanceOf('MySingleton', $class);
        test::double('MySingleton', ['getInstance' => new DOMDocument]);
        $this->assertInstanceOf('DOMDocument', $class);
    }
    ?>
    

    And with AspectMock we really do it - the test is passing. Then should we still consider a singleton to be a bad practice?

    Beyond Good and Evil

    Classes and methods in PHP are declared statically and can't be changed in runtime. This can be treated as language limitation. Dependency Injection pattern can be used as a workaround for this limitation and widely used for testing. AspectMock breaks the limitation. The same can probably be achieved with Runkit extension. But AspectMock doesn't require you to install additional extensions, and uses only PHP methods to do its job.

    "Testability" should not be used as argument deciding what design pattern is right to use and what is not. When you develop with PHP you should always rely on common sense only. Production code should be efficient, fast, readable, and maintainable. The tests should not introduce redundant abstractions to the production code.

    Real World Experience With Yii2

    Let's get hands on AspectMock. We will use a demo application from the upcoming Yii2 framework. Despite having dependency injection container, Yii2 does not use it for models. It relies on static calls to global Yii class.

    Take a look into LoginForm model of advanced application from the Yii2 repo.

    Here is the source code:

    <?php
    namespace common\models;
    
    use Yii;
    use yii\base\Model;
    
    class LoginForm extends Model
    {
        public $username;
        public $password;
        public $rememberMe = true;
    
        public function rules()
        {
            return array(
                // username and password are both required
                array('username, password', 'required'),
                // password is validated by validatePassword()
                array('password', 'validatePassword'),
                // rememberMe must be a boolean value
                array('rememberMe', 'boolean'),
            );
        }
    
        public function validatePassword()
        {
            $user = User::findByUsername($this->username);
            if (!$user || !$user->validatePassword($this->password)) {
                $this->addError('password', 'Incorrect username or password.');
            }
        }
    
        public function login()
        {
            if ($this->validate()) {
                $user = User::findByUsername($this->username);
                Yii::$app->user->login($user, $this->rememberMe ? 3600*24*30 : 0);
                return true;
            } else {
                return false;
            }
        }
    }
    ?>
    

    As you see, it can't be tested in classical unit testing. The only option we have here is to write an integration test for this class. But with AspectMock we can easily get this class tested with 100% code coverage.

    Let's test successful and unsuccessful login scenarios in LoginForm.

    LoginForm relies on User class. That's why to write a test, we will mock some of its methods. We will create a mock with test::double call. In a second argument we are passing the methods that are goint to be replaced and the values they should return.

    <?php
        public function setUp()
        {
            test::double('common\models\User', [
                'findByUsername' => new User,
                'getId' => 1,
            ]);
    
        }
    ?>    
    

    With this User::findByUsername() will always return an empty User instance. And user id will always be 1. For user to log in we need that $user->validatePassword() returned true. We will mock that call in a test.

    <?php
    public function testCanLoginWhenValid()
    {
        $user = test::double('common\models\User', ['validatePassword' => true]);
    
        $model = new LoginForm();
        $model->username = 'davert';
        $model->password = '123456';
    
        $this->assertTrue($model->login());
        $user->verifyInvoked('findByUsername',['davert']);
        $user->verifyInvoked('validatePassword',['123456']);
    }
    ?>    
    

    Additionally we did a check that validatePassword method was called, and user was found by findByUsername call. In production environment, this methods would use the database.

    The same way we can check that user can't log in with invalid password:

    <?php
    public function testCantLoginWhenInvalid()
    {
        $user = test::double('common\models\User', ['validatePassword' => false]);
    
        $model = new LoginForm();
        $model->username = 'davert';
        $model->password = '123456';
    
        $this->assertFalse($model->login());
        $user->verifyInvoked('findByUsername',['davert']);
        $user->verifyInvoked('validatePassword',['123456']);
    }
    ?>    
    

    And in the end we will also check that user can't be logged in without a password.

    <?php
    public function testCantLoginWithoutPassword()
    {
        test::double('common\models\User', ['validatePassword' => true]);
        $model = new LoginForm();
        $model->username = 'davert';
        $this->assertFalse($model->login());
        $model->password = '123456';
        $this->assertTrue($model->login());
    }    
    ?>
    

    If we execute this tests with Codeception we will see all them pass successfully:

    passed

    If you want to see this with your own eyes, clone this application from Github and run Codeception tests:

    php vendor/bin/codecept run
    

    Pay attention to tests/_bootstrap.php file where AspectMock Kernel is initialized. Yii autoloader was loaded through AspectKernel as well. That is important to point AspectMock to a custom autoloader if you do not rely on Composer's autoloader entirely.

    How it Works

    There are no magical meadows and mighty unicorns in a hat. Still AspectMock uses something really powerful to break the rules. You may have heard of Aspect Oriented Programming. Go AOP framework, developed by @lisachenko does awesome job to bring the AOP to PHP world. It intercepts all method calls and allows to put your own advices for them. The AspectMock is just an advice on top of Go Aop.

    Go Aop scnans all libraries and enhances include and require statements with PHP filters. Go adds a parent proxy class to any loaded PHP class on the fly. So If we get back to Yii2 example, User::findByUsername call will invoke that method on a proxy class.

    Conclusions

    AspectMock still considered to be an experimental project. But it has a wide potential. It is very simple and easy to use. It has very tiny api easy to remember and understand. That's why tests developed with AspectMock are very clean and readable.

    AspectMock is not a testing tool for the bad code. The good code is efficient code. WordPress is much popular then any PHP framework, because of its efficiency. Magento does not have unit tests (only integration), but is the most popular ecommerce platform. We can't say how many there are unit tests in Facebook, but we can bet, it started without unit tests. Code should do its job. Code should be readable and maintanable. Overusing dependency injection does not make the code more efficient in any sense. By the way, in Ruby dependency injection is not widely used, but as you may know ruby developers are very passionate about testing.

    AspectMock is not a tool for newbies who just didn't manage to learn the good practices. It is advanced tool, that require you to set dependencies explicitly in a test. That may require deep knowledge on internals of framework you use.

    You can try it on your own project. If you have code parts that can't be unit tested in classical manner, then AspectMock can do a job for you.


    0 0

    In previous part of this tutorial we installed Codeception and got a simple test for User Submitted Posts plugin. We tested that user can send a post and see a message that the post was successfully sent. Nothing more.

    Let's remind the test SubmitPostCept we done previously:

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    $I->fillField('Your Name', 'Michael');
    $I->fillField('Your URL','http://drone-rules.com');
    $I->fillField('Post Title', 'Game of Drones Review');
    $I->fillField('Post Tags', 'review book rob-starkraft');
    $I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
    $I->fillField('Post Content', 'This story is epic and characters are amazing.');
    $I->click('Submit Post');
    $I->see('Success! Thank you for your submission.');
    ?>
    

    But well, we didn't verify that the admin has received this post. And it can be published after a moderation. Thus, we will require few more steps to make a test complete. We will need to login to WordPress admin dashboard.

    Dashboard

    Test commands which allow us to do that are pretty obvious. To keep the code listing shorter we won't show the code from previous lesson. But you should understand that we just append new commands to the previous steps.

    <?php
    $I->amOnPage('/wp-login.php');
    $I->fillField('Username', 'admin');
    $I->fillField('Password','admin');
    $I->click('Log In');
    $I->see('Dashboard');
    ?>
    

    Then we go to Posts section to get all the post listed. We expect to see the "Game of Drones" post in the list.

    Posts

    Let's also check that it was not published by default and it is in Pending state. Game of Drones Review - Pending should be found inside the table html tag, right? We can even specify the CSS class .posts for that table.

    <?php
    $I->amOnPage('/wp-login.php');
    $I->fillField('Username', 'admin');
    $I->fillField('Password','admin');
    $I->click('Log In');
    $I->see('Dashboard');
    $I->click('Posts');
    $I->see('Game of Drones Review','table.posts');
    ?>
    

    That's right, the see command we use for assertions (and the test will fail if assertions fail) has a second parameter which allow us to narrow the results from the whole page, to a particular area on a page, which we can define by CSS:

    Posts CSS

    What's left? We need to review and publish this post, right? The result we expect... Well, we will trust the notifications once again, and message Post published should be enough for us.

    Published

    And that's how we will do that in test.

    <?php
    $I->amOnPage('/wp-login.php');
    $I->fillField('Username', 'admin');
    $I->fillField('Password','admin');
    $I->click('Log In');
    $I->see('Dashboard');
    $I->click('Posts');
    $I->see('Game of Drones Review','table.posts');
    $I->click('Game of Drones Review','table.posts');
    $I->click('Publish');
    $I->see('Post published');
    ?>
    

    And that would be a good point to execute tests once again. Probably you remember, that you need execute codecept.phar with run parameter from console.

    php codecept.phar run
    

    What we are seeing now? Oh no. Just another fail.

    Fail

    What could go wrong? Hopefully Codeception gives us a suggestion to look for the complete HTML code in _log directory. This directory, actually tests/_log, was meant to store all data related to test execution. We can see there two log files, and the file which we actually need: SubmitPostCept.php.page.fail.html.

    Log

    This file, named after our test name, stores the HTML that was on page, before the fail. Let's open it in a browser to get a clue what might go wrong here.

    Publish

    It looks like we are still on the Edit Post page we were before. And the post status is still Pending Review. It looks like click('Publish') command didn't do its job.

    Take a look on screenshot. as you can see, word Publish occur several times on a page. We can assume that we were clicking wrong one. What should we do in this case? We can specify the exact that blue Publish button with CSS.

    Codeception can use CSS instead of names for clicking the elements. The button have id=publish attribute, thus, it can be found by #publish selector. That what we would probably do if we were using jQuery.

    <?php
    $I->amOnPage('/wp-login.php');
    $I->fillField('Username', 'admin');
    $I->fillField('Password','admin');
    $I->click('Log In');
    $I->see('Dashboard');
    $I->click('Posts');
    $I->see('Game of Drones Review','table.posts');
    $I->click('Game of Drones Review');
    $I->see('Edit Post');
    $I->click('#publish'); // id of "Publish" button
    $I->see('Post published');
    ?>
    

    And that fixes the test. You can check that by executing test once again. Let's not trust the notification test and check that a post really is on site. We can click View Post link right near the Post updated message.

    Page

    What we will check that there is site motto on a page (thus we know, we see the blog theme), and that post title is shown in .entry-title class. And yep, we see this story is "epic and amazing". Let's face the final and complete code of our test:

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    $I->fillField('Your Name', 'Michael');
    $I->fillField('Your URL','http://drone-rules.com');
    $I->fillField('Post Title', 'Game of Drones Review');
    $I->fillField('Post Tags', 'review book rob-starkraft');
    $I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
    $I->fillField('Post Content', 'This story is epic and characters are amazing.');
    $I->click('Submit Post');
    $I->see('Success! Thank you for your submission.');
    
    $I->amOnPage('/wp-login.php');
    $I->fillField('Username', 'admin');
    $I->fillField('Password','admin');
    $I->click('Log In');
    $I->see('Dashboard');
    $I->click('Posts');
    $I->see('Game of Drones Review','table.posts');
    $I->click('Game of Drones Review');
    $I->see('Edit Post');
    $I->click('#publish');
    $I->see('Post published');
    $I->click('View Post');
    $I->see('Just another WordPress site');
    $I->see('Game of Drones Review','.entry-title');
    $I->see('This story is epic and characters are amazing.');
    ?>
    

    Well, we did a good job. But the test is like too long. And we can't understand what was going on here. We can add comments to the code, and Codeception has some valuable helpers to add extra text informations with expect and amGoingTo commands.

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('submitted a post by user and publish it as admin');
    
    $I->amGoingTo('submit a post as a regular user');
    $I->amOnPage('/');
    $I->click('Submit a Post');
    $I->fillField('Your Name', 'Michael');
    $I->fillField('Your URL','http://drone-rules.com');
    $I->fillField('Post Title', 'Game of Drones Review');
    $I->fillField('Post Tags', 'review book rob-starkraft');
    $I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
    $I->fillField('Post Content', 'This story is epic and characters are amazing.');
    $I->click('Submit Post');
    $I->see('Success! Thank you for your submission.');
    
    $I->amGoingTo('log in as admin');
    $I->amOnPage('/wp-login.php');
    $I->fillField('Username', 'admin');
    $I->fillField('Password','admin');
    $I->click('Log In');
    $I->see('Dashboard');
    
    $I->expect('submitted post was added to a list');
    $I->click('Posts');
    $I->see('Game of Drones Review','table.posts');
    $I->click('Game of Drones Review');
    
    $I->amGoingTo('publish this post');
    $I->see('Edit Post');
    $I->click('#publish');
    $I->see('Post published');
    
    $I->expect('post is available on blog');
    $I->click('View Post');
    $I->see('Just another WordPress site');
    $I->see('Game of Drones Review','.entry-title');
    $I->see('This story is epic and characters are amazing.');
    ?>
    

    We divided our test scenario into logical parts. At least we have left some notice about what is going on and what we were going to achieve. If we execute test with --steps option, we will get a output with all passed steps listed:

    php codecept.phar run --steps
    

    Final

    Our comments were added to list of passed steps, thus we can easily understand what was going on. And that's quite enough for today. Our test is pretty mature, and covers not only plugin functionality, but WordPress core functions too. Whenever this plugin gets updated, this test should pass to ensure we did everything right.

    But the only thing left. After we executed tests several times, we got this picture on Posts screen.

    Previous Posts

    We have attack of clones here. Yep, each test created its own post and published it. Probably, it is not a good idea to pollute the blog with dozens of posts with similar names. We use post title and content in a test, so probably we can't be sure, what post we are dealing with: current one or the post from a previous test.

    That will lead us to the idea of data cleanup. Probably we should delete post after it was published, to revert all our changes. Alternatively we can install WordPress Reset Plugin to reset WordPress to its initial state. In both cases we will need to append some steps to our test to get data cleaned before the next test.


    Let's make this your home task. To not start with scratch, you can download code of this tutorial or clone it on Github.

    In next lesson we will try to refactor this test and get a few more of them. Don't worry, they will be much shorter than this one. See you soon!


    0 0

    This is a minor release, mostly done to fix some bugs, you have encountered. Please, submit your Pull Requests for the bugs critical of yours. Most of pull requests are accepted, but if you will start a proposal, we can recommend you the best way to implement the fix.

    Selenium2 Compatiblity

    Selenium2 server v.2.34 was released recently and to use it you need to update Codeception. If you use phar version, you should replace your old codecept.phar with new one.

    Debug Levels

    At least one useful feature we prepared for you. Debug output in PhpBrowser and REST modules was extended with additional information that will be printed in debug mode:

    debug

    Tests would be much easier to debug when you see reponse headers, status codes and client cookies. Don't forget to add --debug to run your PhpBrowser acceptance tests, option to see that.

    Title Actions

    Two basic yet useful actions were added to all web interaction modules.

    <?php
    $I->seeInTitle('My Blog | My Post #1');
    $I->dontSeeInTitle('Her Blog');
    ?>
    

    Should be useful, right?

    Single Test

    Finally you can now execute a single test from Cest or Test testcases.

    php codecept.phar run tests/unit/UserModelTest.php:testSave
    

    In this case we will execute only testSave test out of UserModelTest TestCase. The same works for Cests. You may write only the beginning of test name, to execute it.

    Bugfixes

    • fix to correct displaying of non-latin characters in html-report by shofel
    • --xml output for Codeception\TestCase\Test fixed
    • fixed unserialize error during code coverage. Anyway, if you ever seen this, you didn't setup coverage correctly.
    • Interactive console console command does not boot with error stacktrace.
    • Clearing only tables and not views in Db->cleanup()
    • PDO $dbh is now passed to Db module corretcly #414

    Release Plan

    Also we are planning to get more stable releases, and follow the Semantic Versioning. This means that current stable branch is 1.6. If you submit patches and bugfixes, you should propose them into 1.6 branch. Experimental features should go to master.

    release branch status
    Stable 1.6 Build Status Latest Stable
    Development master Build Status Dependencies Status

    Update

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0
  • 08/29/13--15:03: Codeception 1.6.6: Sequences
  • A minor release with one major announcement. In 1.6.6 most bugfixes were included from pull requests. Thanks for everyone who did the contributions.

    Sequence

    Also a very tiny new module was included. It's Sequence which was created to generate unique data for your tests. Sequnces become pretty handy if you don't do database cleanup before each tests. They guarantee you can get unique names and values for each test.

    <?php
    $I = new WebGuy\PostSteps($scenario);
    $I->wantTo('create a post');
    $I->createPost('Post'.sq(1), 'Lorem Ipsum');
    $I->see('Post created sucessfully');
    $I->see('Post'.sq(1), '#posts');
    ?>
    

    No matter how much times you execute this test, each time you see a new post is created with different name.

    Bugfixes

    • Remote codecoverage now works with Selenium2 module. Please update c3.php file to use it.
    • IoC container access in Laravel4 module by @allmyitjason.
    • PostgreSQL driver fixes by @mohangk and @korotovsky .
    • don't rollback for inactive transaction in Dbh module by @sillylogger
    • fix to guy classes generation with namespaces by @vinu.
    • SQLite improvements by @piccagliani

    Update

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    Living on the Edge: WebDriver

    In July a group of Facebook developers set the goal to write the complete new Selenium Webdriver bindings. They decided to do it finally the right way, with the same WebDriver interface it is used in other languages like Java and C#. Ironically, Selenium2 module of Codeception uses the old webdriver bindings from Facebook.They were very hard in use, and had lots of issues. Most common issues were solved in Element34 fork, which was then forked by Instaclick to bring namespaces and PSR-0, which was then used in Mink's Selenium2Driver, and Mink was used in Codeception.

    Pretty tricky, right?

    Currently there are 3 WebDriver bindings in PHP.

    • Selenium2TestCase of PHPUnit which is the most old, the most complete and the most OOP webdriver implementation. But if you have worked with its api, you understand how robust it is to learn and use.
    • Element34 fork based on initial facebook/webdriver bindings, but with hooks to solve common pitfalls.
    • Selenium2Driver of Mink which incorporates Element34 bindings and Syn.js library to perform most of interactions via JavaScript bypassing WebDriver API.

    Ok. Now we have new facebook webdriver bindings. They are in active development and they lack proper documentation. But the good part of it, that even without documentation you will easily learn how to work with them. Any question on StackOverflow with examples in Java or C# will work in PHP just the same way.

    In Codeception master we created a new WebDriver module which uses new webdriver bindings. This module will be included into first 1.7 release, but it won't be recommended for regular use before the stable version of php-webdriver is released.

    To try the new WebDriver you should switch to dev-master in composer.json or use the pre-prelease phar package.

    WebDriver module does not implement everyhting the Selenium2 module has. There is no right clicks, drag and drops, and more. But there are few handy improvements:

    • submitForm action as in PhpBrowser.
    • waitForElement action to wait for element to appear on page.
    • selectOption and checkOption now both work with radio buttons.
    • seeElementInDOM to check the invisible elements.
    • waitForElementChange to wait that element changed
    • implicit waits with wait config prameter (default is 5 secs).
    • maximizeWindow specially for @aditya- :).

    In all other ways its pretty compatible with Selenium2 module. Try it on your own risk or wait for stable versions.


    0 0
  • 09/13/13--15:03: Understanding AspectMock
  • As you may know, AspectMock is non-ordinary mocking framework that can override any method of any class in your application. This is practically useful If you want to unit test code, which was not aimed to be testable from start. Also AspectMock gives you power to write efficient code at first, and not affect production code with testing design.

    Test Design

    Even AspectMock proposes a flexibility in testing, it doesn't drive you into bad application design. If you use classes globally (without injecting them) or you use static properties, methods, or singletones, it's all right while they are defined as your internal API. Such API methods should be well documented, especially for cases, where they should be used, and where not.

    If we use ActiveRecord pattern, we can assume that all models are inherited from ActiveRecord class. The only point in which our models is accessing database is save method of that class. Thus, we need only to block its call, If we don't want the database to be hit.

    <?php
    test::double('ActiveRecord', ['save' => false]);
    $user = new User(['name' => 'davert']);
    $user->save(); // false
    ?>
    

    Sure, integration testing using database gives us more reliable results. And no one ignores that fact. But unit tests allows to cover more cases, without implementing and loading fixtures. They are much faster too.

    Features and Drawbacks

    AspectMock may sounds cool for you, but you feel that there should be pitfalls. Let's be honest, and list all of them here.

    • The most common issue is to get AspectMock installed. We won't list different configuration options here, they are well documented in Github readme. But the idea is pretty simple: you should include directories with files expected to be mocked. If you don't rely completely on autoloader from composer, you should include your autoloaders too.
    • AspectMock will slow down execution in about 20%. That's because all methods of all classes are intercepted.

    You may be curious If AspectMock affect the stack traces? The answer is no. AspectMock (starting from 0.4) does not change the line order in mocked classes, thus you get truth worthy information in stack trace. Sure, AspectMock changes those files, a bit, but more about that in next section.

    Can I debug my tests when using AspectMock? And here are the good news: Sure, you can! In Debug mode you will see your classes, with no mock including in them.

    To summarize: AspectMock has no side effects on unit testing process. Its magic is properly hidden to not affect the development.

    Dark Magic Inside

    Before implementing AspectMock into your project you might want to know, how it works in details. AspectMock is powered by Go Aop Framework.

    Go AOP Framework uses php stream wrappers with filter to parse PHP files before requiring them. That may even happen in runtime. Thus, by analyzing file, we can find all its methods, and inject mocking code into it. To do so, all requiresshould include a filter. This will look like:

    <?php
    require 'myfile.php';
    ?>
    

    will be replaced with

    <?php
    require 'php://read=go.source.transforming.loader/resource=myfile.php';
    ?>
    

    That will make PHP file to be parsed before loading, and changed on the fly. Go AOP is pretty smart to cache already parsed files.

    Now time comes for AspectMock. For every method of every class, AspectMock inserts one line into very beginning of method definition. This sample class

    <?php
    class User {
        
        function setName($name)
        {
            $this->name = $name;
        }    
    
    }
    ?>
    

    will be replaced with:

    <?php
    class User {
        
        function setName($name)
        { if (($__am_res = __amock_before($this, __CLASS__, __FUNCTION__, array($name), false)) !== __AM_CONTINUE__) return $__am_res; 
            $this->name = $name;
        }    
    
    }
    ?>
    

    As you see only one line is added. If a stub was registered for this method, its result will be returned, and method itself won't be invoked.

    If you will enter into class in debug mode, you won't see the line injected by AspectMock. But you will notice its there, even not show, in step by step debug.

    And that's probably all the dark magic you should be aware of. Probably it's not too tricky and you will get along with it.

    Conclusion

    Unit tests are important part of testing pyramid. They are fast and they are flexible. You should not ignore them just because it may be hard for you to implement them. Its not that hard anymore. With AspectMock you can get a good code coverage with less efforts for any kind of modern php application.


    0 0
  • 09/14/13--15:03: Codeception 1.6.7
  • This release is minor, yet fixes lots of bugs you might have encountered. If you didn't encounter them, well, then, lucky you. Still it's a good idea to upgrade. Maybe you will encounter some bugs after that :)

    Ok, let's turn off the irony mode, and post a regular list of changes.

    • fix to 80 port issue, if you had problems connecting to server via PhpBrowser, then this must be it. Thanks to @tiger-seo.
    • fix in REST module when using application/json header and passing parameters.
    • seeJsonResponseContains of REST can now search on all nesting levels.
    • fix to Sequence module.
    • Step class code improved by guilhermeFranco
    • Using suite with defined namespace was improved by @Borales.
    • Generators fixes by @piccagliani and davert.

    Update

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0

    One of the most important requirements for tests is maintainability. The tests can live in a project for a months or even years. One day it may happen that some old tests start failing and the team got no idea about what the test does.

    Does this test checks something important? Maybe specification has changed and the test should be rewritten? The team who worked on tests that early days wrote them only to make them pass. Team is not sure what was the purpose of test.

    In such situations it would have been nice if a developer who wrote the test at least has left some comments. But not. Usually no one documents tests. Test passes and developer is completely satisfied with that fact.

    Proper test structure and readability is the only way to maintainable tests. Tests must not turn to a legacy code. Is there a way to write better tests?

    The rule, dictated by BDD, is quite simple: write tests for specifications. Do not test just methods, test the behavior. As you know, there are plenty of BDD frameworks that replace classical unit testing with specification testing. You may have heard of RSpec in Ruby, Jasmine, Mocha and others in JavaScript.

    If you ever did testing in JavaScript you know how popular mentioned BDD frameworks are. Why can't we have something similar in PHP? We got PHPSpec which is nice, but looks much different from mentioned frameworks. What if we want something more usual? Something like Jasmine in PHP?

    Even if we had such BDD framework none will ever adopt it as we have PHPUnit for all kind of testing in PHP. We won't switch PHPUnit in favor of some geeky BDD tool. But actually to write BDD-styled tests, inspired by Jasmine we don't need to do dramatic changes.

    We can use Specify, a simple trait inserted into your PHPUnit's TestCase that allows you to store several specifications in a test and write them in BDD way.

    At first we will write down some specifications in a body of typical PHPUnit's test:

    <?php
    // this is just a PHPUnit's testcase
    class PostTest extends PHPUnit_Framework_TestCase {
    
        use Codeception\Specify;
    
        // just a regular test declaration
        public function testPublication()
        {
            $this->specify('post can be published');
            $this->specify('post should contain a title');
            $this->specify('post should contain a body');
            $this->specify('author of post should not be banned');      
        }
    }
    ?>
    

    Pretty sweet, we started with describing things before the test. But can't we do the same with comments? specify method is much better then comments as it introduces code blocks into PHPUnit.

    To see it in action, let's write the tests.

    <?php
    // this is just a PHPUnit's testcase
    class PostTest extends PHPUnit_Framework_TestCase {
    
        use Codeception\Specify;
    
        // just a regular test declaration
        public function testPublication()
        {
            $this->post = new Post;
            $this->post->setAuthor(new User());
    
            $this->specify('post can be published', function() {
                $this->post->setTitle('Testing is Fun!');
                $this->post->setBody('Thats for sure');
                $this->assertTrue($this->post->publish());
            });
    
            $this->specify('post should contain a title', function() {
                $this->assertFalse($this->post->publish());
                $this->assertArrayHasKey('title', $this->post->errors());        
            });
    
            $this->specify('post should contain a body', function() {
                $this->assertFalse($this->post->publish());
                $this->assertArrayHasKey('body', $this->post->errors());     
            });
    
            $this->specify('author of post should not be banned', function() {         
                $this->post->getAuthor()->setIsBanned(true);
    
                $this->post->setTitle('Testing is Fun!');
                $this->post->setBody('Thats for sure');           
    
                $this->assertFalse($this->post->publish());
                $this->assertArrayHasKey('author', $this->post->errors());
            });      
        }
    }
    ?>
    

    This code blocks will be executed inside the same test. But it's important to notice that each code block is isolated, thus, when an assertion inside a block fails, the test won't stop the execution. That is how the specify is different from comments.

    Now we've got a list of specification and code examples for each case. If one day our site will allow micro-blogging, we can easily find post should contain a body specification and remove it. That's pretty flexible, thus maintainable.

    Please note, that all the specification are grouped by context. In plain PHPUnit you would create each code block as a separate method. This way it's pretty hard to all the tests related to one specific feature, in our case - publishing.

    Ok, we got nice specifications. But can we also replace classical asserts with some more BDD stuff? Sure. We have another tiny package Verify which is also inspired by Jasmine. Assert keyword is replaced either with expect (as Jasmine does) or verify. This asserts change the order of assetion to improve readability.

    Let's rewrite our test with Verify so you could feel the difference.

    <?php
    // this is just a PHPUnit's testcase
    class PostTest extends PHPUnit_Framework_TestCase {
    
        use Codeception\Specify;
    
        // just a regular test declaration
        public function testPublication()
        {
            $this->post = new Post;
            $this->post->setAuthor(new User());
    
            $this->specify('post can be published', function() {
                $this->post->setTitle('Testing is Fun!');
                $this->post->setBody('Thats for sure');
                expect_that($this->post->publish());
            });
    
            $this->specify('post should contain a title', function() {
                expect_not($this->post->publish());
                expect($this->post->errors())->hasKey('title');       
            });
    
            $this->specify('post should contain a body', function() {
                expect_not($this->post->publish());
                expect($this->post->errors())->hasKey('body');        
            });
    
            $this->specify('author of post should not be banned', function() {         
                $this->post->getAuthor()->setIsBanned(true);
    
                $this->post->setTitle('Testing is Fun!');
                $this->post->setBody('Thats for sure');           
    
                expect_not($this->post->publish());
                expect($this->post->errors())->hasKey('author');
            });      
        }
    }
    ?>
    

    Basically it's a deal of habit. The expect(XX)->toBe(YY) style is more natural for reading as you read from left to right. But If you got used to assertXXX syntax (where the code is read from right), you can skip Verify library.


    With just a few tiny libraries we converted a classical flat PHPUnit test into separate code blocks driven by specification. Test looks very similar to what we saw in Jasmine, just as we intended.

    And the piece of advice to your team: introduce a rule "no assertion without specification" to make tests readable and easy to maintain.

    You can install Specify and Verify via Composer:

    php composer.phar require "codeception/specify:*" --dev
    php composer.phar require "codeception/verify:*" --dev
    

    To introduce specify codeblocks, just add use Codeception\Specify into any TestCase file. BDD styled assertions with expect keywords are installed automatically.

    Hint: Isolated codeblocks are especially useful for testing exceptions.


    0 0
  • 10/07/13--15:03: Codeception 1.6.8
  • Yet another minor release before the 1.7 comes.

    In 1.7 you will see new Selenium WebDriver module, better output formatter moved to Symfony Components, and other useful features. It will come shortly during next week(s). But for now lets list bugfixes contributed by our community members. Thank you all!

    Bugfixes

    • ZF2 module fixes by Marcelo Araújo.
    • Cleanup for custom enum types in PostgreSQL module by korotovsky.
    • Disabled fields are sent by submitForm method no more. Thx to TrustNik.
    • seeResponseIsXml added to REST module by FnTm. Expect more REST/Xml features in 1.7
    • WebDriver module usage fixed.
    • Fixed unit testing results display when used with CodeGuy object
    • seeElement and dontSeeElement work correctly now.
    • Conditional Asserts work as expected now.
    • seeCookie can be used with different domain set. Thx vkn.
    • fixed submitForm for forms that don't have submit buttons in PhpBrowser by vkn.

    Translations

    It's good for you to know, that Russian and Portuguese translations of Codeception Guides were started. If you know either of this langueges (or even both, he-he) please, help us get translations done! By now several chapters are already translated, but you know, there are still lot of work,

    Update

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0
  • 10/11/13--15:03: Codeception 1.7: WebDriver
  • This is new Codeception with awaited WebDriver module in it. WebDriver module is new incarnation of Selenium implementation. As it was mentioned in previous post this WebDriver module is based on facebook/php-webdriver bindings. The most valuable thing in new Selenium bindings that they can be used just the same way Selenium is used in Java. It's very important project for PHP community, and we say "thank you" to all the Facebook behind it. One day PHP will be used in acceptance testing as widely as Java or Ruby is used.

    WebDriver

    WebDriver module is pretty new, yet you may want to switch to it from Selenium2. It uses just the same interface, so migration should come smoothly. If you have issues, then no hurry, stay with Selenium2 for a while. Let's list a few features that WebDriver module has:

    Implicit waits

    <?php
    $I->waitForText('foo', 30); // secs
    $I->waitForElement('#agree_button', 30);
    $I->waitForElementChange('#menu', function(\WebDriverElement $el) {
        return $el->isDisplayed();
    }, 100);
    $I->waitForJS("return $.active == 0;", 60);
    ?>
    

    Better Keyboard Manipoluation

    <?php
    // <input id="page" value="old" />
    $I->pressKey('#page','a'); // => olda
    $I->pressKey('#page',array('ctrl','a'),'new'); //=> new
    $I->pressKey('#page',array('shift','111'),'1','x'); //=> old!!!1x
    $I->pressKey('descendant-or-self::*[@id='page']','u'); //=> old!!!1xu
    $I->pressKey('#name', array('ctrl', 'a'), WebDriverKeys::DELETE); //=>''
    ?>
    

    submitForm

    A submit form method was implemented in WebDriver.

    <?php
    $I->submitForm('#login', array('login' => 'davert', 'password' => '123456'));
    ?>
    

    Common Selenium API

    Thus If you got any Selenium questions any answers on StackOverflow will help you. PHP implementation is so close to Java that you can use any answer in PHP.

    You can invoke WebDriver methods directly with executeInSelenium

    <?php
    $I->executeInSelenium(function(\WebDriver $webdriver) {
      $webdriver->get('http://google.com');
    });
    ?>
    

    Symfony Output

    You will notice a better output formatting in your console. That's because we migrated to Symfony Console output. As you may know, Symfony console has 3 levels of verbosity, that can set via -v option. Codeception now support them. If you want to get all available information about test, run with -vvv option. The --debug option is now equivalent for running -vv. That's right, you can get even more information with vvv. The output will be improved during the development of 1.7 branch. We hope to get completely different output depending on level of verbosity set.

    Cest Annotations

    Cests are now much smarter then they were before. If you were using StepObject you might wondered how can you pass step object class into it. That was not really obvious, as by default you get Guy class defined in config. But now you can use guy annotation to specify which guy class to use.

    <?php
    /**
     * @guy WebGuy\AdminSteps
     */
    class AdminCest {
    
        function banUser(WebGuy\AdminSteps $I)
        {
            // ...
        }
    
    }
    ?>
    

    Alternatively you can use guy annotation for the method itself.

    Also you can now use before and after annotations to define whic methods of Cest class should be executed before the current one. Thus you can move similar actions into protected methods and invoke them via annotations.

    <?php
    class ModeratorCest {
    
        protected function login(WebGuy $I)
        {
            $I->amOnPage('/login');
            // ...
        }
    
        /**
        * @before login
        */
        function banUser(WebGuy $I)
        {
            $I->see('Logout');
            // ...
        }
    
    }
    ?>
    

    Just by using annotations you can control the invokations of methods of the Cest class. Sure, you should define your support methods with protected, so they won't be executed as tests themselves. Another thing worth to mention, that callbacks defined in after annotation will be called even the main test has failed, thus it makes them useful for clean ups.

    We still maintain and bugfix 1.6 branch and there will be 1.6 bugfix releases. The old release 1.6.9 (yep, 1.6.9 was released to with minor bugfixes) can be downloaded from http://codeception.com/releases/1.6.9/codecept.phar.

    Removed Dependencies

    Optional dependencies were removed from the core, so if you use Composer version and one of the following packages:

        "behat/mink-selenium-driver": "1.1.*",
        "facebook/php-sdk": "3.*",
        "behat/mink-zombie-driver": "1.1.*",
        "videlalvaro/php-amqplib": "*"
    

    you should include them manually in your composer.json. If you use phar version - nothing is changed for you.

    Update

    redownload your codeception.phar for update:

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0
  • 10/17/13--15:03: 1.7: Bugfix release
  • The release of 1.7 added new WebDriver module as well as rewritten Output component. Some if changes where major and was not tested for all crucial cases. If you feel comfortable with 1.6 you can stay on 1.6 branch. But If you want to get more features with some instabilities - connect to 1.7

    Bugfixes in 1.7.1

    • error and failures are now displayed correctly with improved stack traces.
    • fix for module before/after hooks in Codeception\TestCase\Test.
    • select option in WebDriver throws readable message
    • wait in WebDriver throws exception when receives 1000 seconds.

    Bugfixes in 1.6.10 (also in 1.7)

    • Fix the problem when the method getPort() return 443 80 or null by thbourlove.
    • Notifies about CURL not installed.

    Update

    redownload your codeception.phar for update:

    1.7.1

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    1.6.10

    wget http://codeception.com/releases/1.6.10/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0
  • 11/01/13--15:03: Codeception 1.7.2
  • So here is November and a new release with a minor changes is here. The development on 1.8 branch is started and you will get some of a long awaited features soon as well as Phalcon. Btw, you can already try Phalcon module for functional testing (yes, its in master branch, and not inclued in phar yet).

    Bugfixes:

    • Updated to be able to use seeOptionIsSelected and dontSeeOptionIsSelected by label in framework modules by piccagliani
    • Update to appendField function to handle checkbox value or label by allmyitjason
    • Removed default delay WebDriver allmyitjason
    • better error description whenever config file could not be found by tiger-seo.
    • Added appendField function into WebDriver by allmyitjason
    • switched to use getInternalResponse in frameworks modules. Fixes "PHP Fatal error: Call to undefined method **\Response::getStatus() " in Symfony2, Laravel4.
    • Fixed PhpBrowser module persisting HTTP authentication between tests by elazar
    • Hardcoded 'localhost' removed for Yii1 by kop
    • CodeCoverage works for WebDriver module as well by allmyitjason
    • Improved output formatting for unit tests by davert

    Some of this fixes were also added into 1.6.

    Update

    redownload your codeception.phar for update:

    1.7.2

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    1.6.11

    wget http://codeception.com/releases/1.6.10/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0

    In this post we will explore some basics of user acceptance testing with Selenium. We will do this with classical unit testing framework PHPUnit, web browser Firefox, and with new php-webdriver library recently developed by Facebook.

    Selenium allows us to record user actions that we do inside a browser and then automate them. PHPUnit will be used to do various assertions and check them for fallacy. And php-webdriver is used to connect PHP with Selenium, in order to do browser manipulation in PHP.

    Probably you know, that PHPUnit itself can do Selenium manipulations via PHPUnit. There is PHPUnit_Selenium extension you may use. We will use php-webdriver instead because this implementation is modern and its API is more clean and looks much the same way as original Java Selenium API. That's why it is easier to learn, and much powerful, then the PHPUnit's one. For example, it allows you use send native touch events, which is important in era of mobile-ready web applications.

    Grab the tools

    Let's install all the required tools using Composer. For this we will need to have composer.json file created:

    {
        "require-dev": {
            "phpunit/phpunit": "*",
            "facebook/webdriver": "*"
        }
    }
    

    We won't develop any application, thus we are ok, with require-dev section only. Let's run

    php composer.phar install
    

    and grab the latest versions of both libraries. We also will need Selenium server executable as well. You need Java installed in order to run the Selenium server. You can launch it by running this:

    java -jar selenium-server-standalone-2.37.0.jar
    

    And when the tools are prepared, let's write some tests.

    PHPUnit Test

    So let's try to test something in the web. Let's start with something simple and well known, like Github. So, let's create a file and call it GitHubTest.

    <?php
    class GitHubTests extends PHPUnit_Framework_TestCase {
    }
    ?>
    

    As any PHPUnit test it should extend PHPUnit_Framework_TestCase (as it was mentioned, we are not using PHPUnit_Extensions_Selenium2TestCase here). For every test we will need to launch a browser, or in other words, we are starting a Selenium session. This is done by creating RemoteWebDriver instance:

    <?php
    class GitHubTests extends PHPUnit_Framework_TestCase {
    
        /**
         * @var \RemoteWebDriver
         */
        protected $webDriver;
    
        public function setUp()
        {
            $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
            $this->webDriver = new \RemoteWebDriver('http://localhost:4444/wd/hub', $capabilities);
        }
    
    }
    ?>
    

    This initialization part is taken from php-webdriver README file. We don't have any test yet, so let's write something a very basic. Something like: "If I open http://github.com, page title should contain GitHub".

    <?php
    class GitHubTest extends PHPUnit_Framework_TestCase {
    
        /**
         * @var \RemoteWebDriver
         */
        protected $webDriver;
    
        public function setUp()
        {
            $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
            $this->webDriver = new \RemoteWebDriver('http://localhost:4444/wd/hub', $capabilities);
        }
    
        protected $url = 'https://github.com';
    
        public function testGitHubHome()
        {
            $this->webDriver->get($this->url);
            // checking that page title contains word 'GitHub'
            $this->assertContains('GitHub', $this->webDriver->getTitle());
        }    
    
    }
    ?>
    

    Now we execute our first test with phpunit

    php vendor/bin/phpunit GitHubTest.php
    

    and in a few seconds we should see a Firefox window with Github Page in it

    Opening web page with PHPUnit and Selenium

    please notice the WebDriver text in the status bar, this tells you that this browser window is controlled by WebDriver.

    In a console we will see this output:

    PHPUnit 3.7.28 by Sebastian Bergmann.
    
    
    .
    
    Time: 19.44 seconds, Memory: 1.75Mb
    
    OK (1 test, 1 assertion)
    

    We will see that test has finished, but the browser window stays opened. That is because we did not implement a tearDown method, that should be used to close the webdriver session:

    <?php
    class GitHubTest extends PHPUnit_Framework_TestCase {
    
        /**
         * @var \RemoteWebDriver
         */
        protected $webDriver;
    
        public function setUp()
        {
            $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
            $this->webDriver = new \RemoteWebDriver('http://localhost:4444/wd/hub', $capabilities);
        }
    
        public function tearDown()
        {
            $this->webDriver->close();
        }
    }
    ?>
    

    Advanced Test

    We didn't touch any of page elements in a previous test. We just opened the page and checked its title. But the power of webdriver reveals when you want to click elements, fill forms, drag and drop elements, etc. That's why we will write a test that demonstrates some of this features.

    But how can control the browser? Should we move the mouse in order to click on element? Well, not exactly. WebDriver allows us to locate element on page by its ID, class name, element name, CSS, or XPath. Let's list all possible locator types, taken from WebDriverBy class:

    • WebDriverBy::className() - searches for element by its CSS class.
    • WebDriverBy::cssSelector() - searches for element by its CSS selector (like jQuery).
    • WebDriverBy::id() - searches for element by its id.
    • WebDriverBy::linkText() - searches for a link whose visible text equals to the value provided.
    • WebDriverBy::partialLinkText() - same as above, but link partly contain the value.
    • WebDriverBy::tagName() - search for element by its tag name.
    • WebDriverBy::xpath() - search for element by xpath. The most complex, yet, most powerful way for element location.

    To find element we should use webDriver->findElement method, with locator specified with WebDriverBy.

    After the matched element is found we can click on it. Like this:

    <?php
    class GitHubTest extends PHPUnit_Framework_TestCase {
    
        /**
         * @var \RemoteWebDriver
         */
        protected $webDriver;
    
        public function setUp()
        {
            $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
            $this->webDriver = new \RemoteWebDriver('http://localhost:4444/wd/hub', $capabilities);
        }
    
        public function tearDown()
        {
            $this->webDriver->close();
        }
    
        public function testSearch()
        {
            $this->webDriver->get($this->url);
            // find search field by its id
            $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
            $search->click();
        }    
    }
    ?>
    

    Locating Web Element in Firefox

    We are clicking the GitHub global search field, located in top menu bar, matched by its id. By the way, how did we get the element's id? That's a good question. Searching for element locators is the most important task in acceptance testing. For every test we need to get the elements that are involved in it. Let's show some simple tricks that will definitely help you in writing complex acceptance tests.

    Locating Elements: Tips & Tricks

    The first thing we can do is to pause the test execution. While browser window is still open, we can use it to find the locator. To pause the test execution lets write this helper method inside a test class:

    <?php
        protected function waitForUserInput()
        {
            if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
        }
    ?>
    

    If we use it somewhere in our tests, PHPUnit will wait for Enter key pressed in console before going on.

    <?php
        public function testSearch()
        {
            $this->webDriver->get($this->url);
            $this->waitForUserInput(); // paused       
        }    
    ?>
    

    Now when the browser window is opened we are free to search for required element with no hurry. We are using Firfox Developer Tools for that. With the Element Inspector within we can point to element and get its unique CSS locator.

    Using WebDriver with PHPUnit to test GitHub

    That is how we got search field id: #js-command-bar-field. Doing the sample steps, let's continue writing our test and find php-webdriver repository on GitHub.

    <?php
    class GitHubTest extends PHPUnit_Framework_TestCase {
    
        protected $url = 'http://github.com';
        /**
         * @var \RemoteWebDriver
         */
        protected $webDriver;
    
        public function setUp()
        {
            $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
            $this->webDriver = new \RemoteWebDriver('http://localhost:4444/wd/hub', $capabilities);
        }
    
        public function tearDown()
        {
            $this->webDriver->close();
        }
    
        public function testSearch()
        {
            $this->webDriver->get($this->url);
            
            // find search field by its id
            $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
            $search->click();
            
            // typing into field
            $this->webDriver->getKeyboard()->sendKeys('php-webdriver');
            
            // pressing "Enter"
            $this->webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
            
            $firstResult = $this->webDriver->findElement(
                // some CSS selectors can be very long:
                WebDriverBy::cssSelector('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)')
            );
            
            $firstResult->click();
            
            // we expect that facebook/php-webdriver was the first result
            $this->assertContains("php-webdriver",$this->webDriver->getTitle());
            
            // checking current url
            $this->assertEquals(
                'http://github.org/facebook/php-webdriver', 
                $this->webDriver->getCurrentURL()
            );
        }
    
    
        protected function waitForUserInput()
        {
            if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
        }
    
    }
    ?>
    

    If we run this test we will see that it is failing on the last step:

    PHPUnit Test is Failing on Assertion

    That's because we forgot that GitHub uses https by default, and GitHub is a company and not a non-profit organization (as we used to think of it, he-he). Though let's change the expected url to 'https://github.com/facebook/php-webdriver' and see the test is passing.

    Element Not Found

    Probably we will also want to check if element is located on a page. If we were using Selenium2TestCase of PHPUnit, we would have several nice assertion that we can use just for that. In case of php-webdriver library we will need to implement them on our own. But that's pretty easy. Php-Webdriver throws various exceptions, which we can handle and transform into PHPUnit's assertions:

    <?php
        protected function assertElementNotFound($by)
        {
            try {
                $this->webDriver->findElement($by);
                // element not found
            } catch (\NoSuchElementWebDriverError $e) {
                // just increment the assertions counter
                $this->assertTrue(true);
                // element not found, as we expected
                return;
            }
            $this->fail("Element not found");
            
        }
    
    ?>
    

    You can create similar assertion just in the same manner.

    We will use newly created assertElementNotFound method to check that there is no user avatar on "facebook/php-webdriver" page.

    <?php
    class GitHubTest extends PHPUnit_Framework_TestCase {
    
        protected $url = 'http://github.com';
        /**
         * @var \RemoteWebDriver
         */
        protected $webDriver;
    
        public function setUp()
        {
            $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
            $this->webDriver = new \RemoteWebDriver('http://localhost:4444/wd/hub', $capabilities);
        }
    
        public function tearDown()
        {
            $this->webDriver->close();
        }
    
        public function testGitHubHome()
        {
            $this->webDriver->get($this->url);
            // checking that page title contains word 'GitHub'
            $this->assertContains('GitHub', $this->webDriver->getTitle());
        }
    
        public function testSearch()
        {
            $this->webDriver->get($this->url);
    
            // find search field by its id
            $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
            $search->click();
    
            // typing into field
            $this->webDriver->getKeyboard()->sendKeys('php-webdriver');
    
            // pressing "Enter"
            $this->webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
    
            $firstResult = $this->webDriver->findElement(
                // some CSS selectors can be very long:
                WebDriverBy::cssSelector('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)')
            );
    
            $firstResult->click();
    
            // we expect that facebook/php-webdriver was the first result
            $this->assertContains("php-webdriver",$this->webDriver->getTitle());
    
            $this->assertEquals('https://github.com/facebook/php-webdriver', $this->webDriver->getCurrentURL());
    
            $this->assertElementNotFound(WebDriverBy::className('avatar'));
    
        }
    
        protected function waitForUserInput()
        {
            if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
        }
    
        protected function assertElementNotFound($by)
        {
            try {
                $this->webDriver->findElement($by);
            } catch (\NoSuchElementWebDriverError $e) {
                $this->assertTrue(true);
                return;
            }
            $this->fail("Unexpectedly element was found");
            
        }
    }
    ?>
    

    Refactoring

    To clean up some things we will separate test methods and support methods. It is a good idea to move custom assertions into trait: WebDriverAssertions. And the pause switcher waitForUserInput can be moved into WebDriverDevelop trait. We can enable this trait in a test class, when we develop a test, and turn it off once we finished.

    The complete demo project, after this basic refactoring, you can find on GitHub.

    And what about Codeception?

    So you did notice that this is Codeception blog. But we didn't use Codeception framework at all in this tutorial. Sure, we need to mention, that Codeception includes php-webdriver library and WebDriver module out of the box starting from version 1.7. In Codeception you can perform all the web manipulations in a much simpler manner using the WebGuy APIs. If you use Codeception you don't need to implement your own WebDriver assertions nor write boilerplate code.

    Conclusions

    No matter you are using Codeception or not it is a good idea to understand how to perform browser based acceptance testing using just the php-webdriver by itself. Php-webdriver library provides very clean and flexible APIs you will enjoy working with.

    PHP is a language popular for web development, but not for web testing. Test automation engineers prefer Java and Ruby over it. And there is a serious reason for that. There is no such thing like "official Selenium bindings" for PHP, i.e. there is no Selenium client library for PHP created by Selenium team. Developers of php-webdriver get very close to the official Selenium client APIs. That's why you should use php-webdriver - it really feels and works like native and official Selenium libraries. That is especially important if you have an experience writing acceptance tests in Java or Ruby. Moving to PHP is not that hard, when all the APIs are the same.


    0 0

    This release brings lots of changes. Finally we got working DataProviders (the issue was opened for about a year), and @depends tag. But the details below. Let's start with the most important new features:

    Phalcon Framework Support

    Phalcon

    Phalcon is the fastest PHP framework in the world. And that's not just a marketing slogan. Phalcon was developed as C extension, so it is already precompiled and loads into memory only once. Phalcon is modern framework with DI container, ORM (inspired by Doctrine), and templating engine (inspired by Twig). It is one of the most innovative projects in PHP world, so you should at least check it out.

    If you already work with Phalcon, here is a good news for you. Codeception got Phalcon1 module, that allows you to write functional tests with minimum setup. Besides standard framework interface this module provides actions for session and database access. It also wraps all tests into nested transactions, and rollbacks them in the end. Thus, all your database tests run superfast.

    Is there any Phalcon project tested with Codeception? Yes, it is the official Phalcon's forum.

    This module was developed by cujo and improved by davert. We hope you like it.

    Environments

    This is something you might not expect in the form it was produced, but, probably, that was long awaited. This is to run tests multiple times over different environments. The most common issue is running acceptance tests over different browsers: firefox, phantomjs, chrome. Now you can create 3 different configurations and you can get tests will be repeated 3 times: in firefox, in phantomjs, and in chrome. The second usecase is run tests over different databases.

    Feature is pretty straightforward in use. You define the name of environment below the env key, and then you redefine any of configuration values you need.

    ``` yaml
    class_name: WebGuy
    modules:
        enabled:
            - WebDriver
            - WebHelper
        config:
            WebDriver:
                url: 'http://127.0.0.1:8000/'
                browser: 'firefox'
    
    env:
        phantom:
             modules:
                config:
                    WebDriver:
                        browser: 'phantomjs'
    

    Advanced Usage chapter was updated.

    DataProviders

    You can use PHPUnit dataproviders in Codeception\TestCase\Test files. Yep. Finally.

    Probably DataProviders are not really readable, as you need always to refer into data sets, which may be defined in the different part of a testcase. You can consider using examples of Codeception\Specify library, as for alternative for dataproviders.

    Is there a way you can use data providers in scenario driven test? Not exactly, but you can emulate them with loops and conditional asserts:

    <?php
    foreach ($posts as $post) {
        $I->canSee($post->title,'.post h2');
    }
    ?>
    

    Depends

    Declare depending tests in Cest and Test files. Works just as the original @depends of PHPUnit. In Cests you can combine this with @before annotation. More information in Advanced Usage.

    Debug

    Debug output was refactored, and moved out to Codeception\Util\Debug class. This class can be used globally, i.e in tests, helpers, - wherever you want. To print debug information you should call:

    <?php
    use Codeception\Util\Debug;
    Debug::debug("This is working");
    ?>
    

    This change dramatically improves debug output. You can also pause execution with pause static method of this class. Useful for debugging and tests development, implemented in WebDriver module as pauseExecution action.

    Bugfixes and Minor Changes

    • WebDriver module got pauseExecution method which pauses running test in debug mode.
    • Generated PageObject file URL const was changed to static variable.
    • waitForElementChange() callback return value was not being used by wheelsandcogs | also in 1.7
    • bugfix for making screenshots with WebDriver by Michael Wang. | also in 1.7
    • Doctrine2: handle NULL value in seeInRepository param array by imanzuk | also in 1.7
    • REST: fix adding parameters to url to non-GET HTTP methods by sheershoff
    • REST: Added sendOPTIONS() and sendHEAD() requests for CORS testing by elazar
    • Symfony2: fixed usage of profiler
    • Strict declaratin error fixes for framework constraints | also in 1.6, 1.7

    Update

    Warning. Module Unit was deprecated in 1.6 and was removed in 1.8. Please disable it in unit.suite.yml if you were using it. If you see this error:

    Codeception PHP Testing Framework v1.8.0
    Powered by PHPUnit 3.7.28 by Sebastian Bergmann.
    
    
    
      [Codeception\Exception\Configuration]
      Unit could not be found and loaded
    

    Just disable Unit module. Thanks

    If you prefer stability over features you can stay on 1.7 or 1.6 releases. We've got them updated too.

    redownload your codeception.phar for update:

    1.8.0

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    1.7.3

    wget http://codeception.com/releases/1.7.3/codecept.phar -O codecept.phar
    

    1.6.12

    wget http://codeception.com/releases/1.6.12/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    What to expect from 2.0

    It is almost 2 years of Codeception, and we are planning to release 2.0 version as a next major to celebrate that. It is a major change, thus we can add few BC breaks. We are planning to:

    • move to PHP 5.4. Not really necessary, yet it is getting hard to support 5.3.
    • remove Mink entirely in favor of WebDriver for browser-testing and Goutte for headless testing.
    • remove 2-times test execution (finally!).
    • ??? (proposed by you)

    0 0

    Last time we discovered API of new WebDriver PHP bindings. We wrote a basic test in PHPUnit that uses Selenium and Firefox browser to find php-webdriver library on Github. Today we will reimplement the same test with Codeception.

    Installation

    Depending on your preferences you can install Codeception either by downloading codecept.phar archive from the site, or alternatively by using composer.

    {
        "require-dev": {
            "codeception/codeception": "*",
        }
    }
    

    With composer you will need to execute:

    php composer.phar install
    

    In previous tutorial we did installation using Composer, so in current examples we will be using it as well.

    Sure, we also need Selenium server executable as well. You need Java installed in order to run the Selenium server. You can launch it by running this:

    java -jar selenium-server-standalone-2.37.0.jar
    

    When all installation steps are done, we can continue with creating Codeception bootstrap.

    Bootstrap

    Unlike the phpunit Codeception requires a small bootstrap step. Codeception helps you to organize tests into 3 categories: acceptance, functional, and unit tests. To create all tests and support directories, you will need to run the bootstrap command.

    vendor/bin/codecept bootstrap
    

    Selenium tests are acceptance tests. So let's create a skeleton for the basic acceptance test:

    vendor/bin/codecept generate:cept acceptance GitHub
    

    This will create new file in tests/acceptance/GitHubCept.php. But we need some additional configuration to be done before proceeding. Instead of creating webdriver session in tests manually, we delegate this to Codeception. Codeception will take care for creating session before each test, and closing it after. That's why Selenium configuration should be written into tests/acceptance.suite.yml.

    class_name: WebGuy
    modules:
        enabled:
            - WebDriver
            - WebHelper
        config:
            WebDriver:
                url: 'http://github.com'
                browser: 'firefox'
    

    Each time you change configuration in Codeception you should run the build command.

    vendor/bin/codecept build
    

    Writing a Test

    Let's start with something very basic. We will open the Github page and we will make sure that GitHub word is within the page title.

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('see GitHub word in title ');
    $I->amOnPage('/');
    $I->seeInTitle('GitHub');
    ?>
    

    We are using the wantTo command just to give a test clean description. amOnPage command opens browser on the github home page, and all the commands that start with see are assertions you can use. There are lots of commands in WebGuy class you can use writing the test. All them are taking from corresponging modules, in our case it is WebDriver module. If you use IDE you can check them all with autocompletion.

    Codeception Autocomplete

    But let's execute a test with run command:

    vendor/bin/codecept run
    

    And you will see this output:

    Codeception PHP Testing Framework v1.8.0
    Powered by PHPUnit 3.7.28 by Sebastian Bergmann.
    
    Functional Tests (0) ------------------------
    ---------------------------------------------
    
    Acceptance Tests (1) -------------------------------------------
    Trying to see github word in title  (GitHubCept.php)       Ok
    ----------------------------------------------------------------
    
    Unit Tests (0) ------------------------------
    ---------------------------------------------
    
    
    Time: 21.62 seconds, Memory: 5.00Mb
    
    OK (1 test, 1 assertion)
    

    You may have noticed that Codeception is itself powered by PHPUnit. Basically you can execute native PHPUnit tests inside Codeception, but the main idea of Codeception is scenario driven tests written from an eye of a tester. Each test should descibe user's actions in simple words: I see, I click, etc. Let's see how using just this simple terms with no OOP complexity we can write a bit more complex test.

    Our test should open github in a browser and use the search form to get to "facebook/php-webdriver" library page. Before writing the test we did some research of GitHub page to find selectors we can use to match elements on page. This was described in previous tutorial.

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('find facebook/php-webdriver on GitHub');
    $I->amOnPage('/');
    $I->fillField('#js-command-bar-field','php-webdriver');
    $I->pressKey('#js-command-bar-field', WebDriverKeys::ENTER);
    $I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
    $I->seeInTitle("php-webdriver");
    $I->seeCurrentUrlEquals('/facebook/php-webdriver');
    ?>
    

    You can execute this test and see it passes. Unlike the classical PHPUnit test, this code does not require comments. It is very clean and easy to understand and edit by anyone a team with basic PHP/HTML/CSS skills.

    Tips and Tricks

    In this section we will share some ideas, that you can use in your tests. Sure that would not be the same tips we have in PHPUnit. But in PHPUnit test we wrote a helper function to pause the execution. This helped us to search for required locators. In Codeception (starting from 1.8) we can use $I->pauseExecution(); method for this. The test will stop the scenario in that place and wait for Enter to be pressed. Worth to mention, that pauseExecution works only in debug mode (sure, you don't want to freeze when running tests on CI). So you should pass --debug option when using it.

    But even Codeception provides us with dozen of predefined commands, we might want to access webdriver session by your own. For this we have a executeInSelenium command:

    <?
    $I->executeInSelenium(function(RemoteWebDriver $webDriver) {
        $webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
    });
    ?>
    

    But that doesn't make our test more readable. What can we do to simplify it a bit? Well, let's replace this double search for '#js-command-bar-field' input field. We can use submitForm command that works for forms on any kind, and you can pass array of field values into it. Current form id is top_search_form, so we can rewrite our test like:

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('find facebook/php-webdriver on GitHub');
    $I->amOnPage('/');
    $I->submitForm('#top_search_form', array('q' => 'php-webdriver'));
    $I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
    $I->seeInTitle("php-webdriver");
    $I->seeCurrentUrlEquals('/facebook/php-webdriver');
    ?>
    

    When running WebDriver tests with Codeception they may look a bit slower then the same tests run with PHPUnit. That's because Codeception has default delay of 5 seconds for each test action on page. This delay can be set to 0 in order to run test faster. This is configured in acceptance.suite.yml

    class_name: WebGuy
    modules:
        enabled:
            - WebDriver
            - WebHelper
        config:
            WebDriver:
                url: 'http://github.com'
                browser: 'firefox'
                wait: 0
    

    This should execute the same test 30% faster then before. But if your page use JavaScript and Ajax, you should increase the wait time or use various waitFor commands of WebDriver.

    Conclusion

    At first sight setting up Codeception Selenium test may look a bit harder then with PHPUnit. At first sight it's hard to see real benefits in it. But the key idea of Codeception is to separate the test code and support code. So you should write only tests, and most of helper methods was already written for you. Thus, the test is kept clean and readable. It is easy to change and easy to manage. If you want to switch a browser, you should not edit a test, but change the configuration. You want to execute test in 2 browsers one by one? No problems, just change a configuration.

    Codeception is very flexible framework that you can use to write your Selenium tests. After all it's really simple and fun.

    Source file of this tutorial are available on GitHub.


    0 0

    Codeception 1.8.1 is out. Bugfixes and small useful features in it. The most interesting improvement was done by frqnck. Phar version now has self-update command which acts the same way as it is in composer. From now on you can easily upgrade codeception.phar file.

    Small yet important change in WebDriver module. Default wait parameter is set to 0. This was done because Selenium implicit waits didn't work as expected - this parameter slowed down test execution. Browser waited for element even if it was already on a page. Please notice this on upgrading.

    Changes

    • upgraded to php-webdriver 0.3
    • added general sendAjaxRequest() method to all framework/phpbrowser modules to send ajax requests of any types by gchaincl.
    • fixed URI construction in Yii1 module by kop(also in 1.7)
    • Fixed Yii2 statusCode by cebe
    • fixed: Placeholder\Registry::unsetRegistry() should only be used with < 2.2.0 by Bittarman
    • waitForElementChange() was fixed in WebDriver module
    • fixed seeLink and dontSeeLink methods in framework modules by enumag.
    • added seeHttpHeaderOnce to REST module for checking if headers appear only once.
    • setUpBeforeClass/tearDownAfterClass method will work as they are expected in PHPUnit
    • Debug::debug can output any variable in console.
    • fixed: "WebDriver makeScreenshot doesn't create directories" by joksnet
    • fixed grabValueFrom method in framework modules (also in 1.7)
    • fixed: "Disable remote coverage not work" by tiger-seo (also in 1.6, 1.7)
    • self-update command added to phar by frqnck

    Dependencies were updated, thus Symfony components were updated to 2.4. No changes to composer.json was made, so this release is compatible with Symfony 2.3 as well.

    Update

    redownload your codeception.phar for update:

    1.8.1

    wget http://codeception.com/codecept.phar -O codecept.phar
    

    1.7.4

    wget http://codeception.com/releases/1.7.4/codecept.phar -O codecept.phar
    

    for composer version

    $ php composer.phar update
    

    0 0

    So how do you check that your applications sends email correctly? It looks like dealing with emails is always a challenge. How would you verify that email message is formatted and delivered correctly, without actually sending them to your clients? That's the first question. And the second question is: how can we automate testing of emails?

    For both questions we have an answer. There are two awesome services that is developed to help developers in dealing with email hell. It is Mailtrap and Mailcatcher. Both services run SMTP server that does not deliver emails, but store them locally. They both have a web interface in which you can review all the outgoing emails. The difference between this services are: mailtrap runs as a web service, and mailcatcher is a ruby gem that can be installed locally.

    mailcatcher

    Its up to you which one to use. Definitely they will simplify your life while developing a web application. Do they have something to offer for testing? Sure! We can access all handled emails via REST API and verify our assertions.

    In this post we will marry Mailcatcher with PHPUnit testing frameworks. We've chosen Mailcatcher for not to rely on 3rd-party web service and have all the tests run locally. We will write methods for both PHPUnit and Codeception in order to provide different solutions and compare them.

    Before we start we need to make sure that Mailcatcher is installed and running. When done you can access its web interface on 1080 port and use 1025 port for fake SMTP server. Configure your web application to use exactly that port in your application when running in test environment.

    Testing emails in PHPUnit

    Mailcatcher has a really simple REST API that is used for email access. Here is a quote from their official site.

    A fairly RESTful URL schema means you can download a list of messages in JSON from /messages, each message's metadata with /messages/:id.json, and then the pertinent parts with /messages/:id.html and /messages/:id.plain for the default HTML and plain text version, /messages/:id/:cid for individual attachments by CID, or the whole message with /messages/:id.source.

    What was not mentioned that you can also clear all emails by sending DELETE request to /messages. The most complete documentation on API is its code. Even you don't know Ruby it is really fair.

    Thus, we will need to send GET and DELETE requests and parse json response. To send them we will use Guzzle framework. PHPUnit and Guzzle can be easily installed via Composer:

    {
        "require-dev": {
            "phpunit/phpunit": "*",
            "guzzle/guzzle": "~3.7"
        }
    }
    

    Let's create EmailTestCase file and place MailCatcher API calls into it.

    <?php
    class EmailTestCase extends PHPUnit_Framework_TestCase {
    
        /**
         * @var \Guzzle\Http\Client
         */
        private $mailcatcher;
    
        public function setUp()
        {
            $this->mailcatcher = new \Guzzle\Http\Client('http://127.0.0.1:1080');
    
            // clean emails between tests
            $this->cleanMessages();
        }
    
        // api calls
        public function cleanMessages()
        {
            $this->mailcatcher->delete('/messages')->send();
        }
    
        public function getLastMessage()
        {
            $messages = $this->getMessages();
            if (empty($messages)) {
                $this->fail("No messages received");
            }
            // messages are in descending order
            return reset($messages);
        }
    
        public function getMessages()
        {
            $jsonResponse = $this->mailcatcher->get('/messages')->send();
            return json_decode($jsonResponse->getBody());
        }
    ?>    
    

    That's enough for fetching list of all delivered emails. All the emails will be cleaned between tests, so test will be executed in isolation. Let's implement some assertion methods to check sender, recipient, subject and body of the email.

    <?php
        // assertions
        public function assertEmailIsSent($description = '')
        {
            $this->assertNotEmpty($this->getMessages(), $description);
        }
        
        public function assertEmailSubjectContains($needle, $email, $description = '')
        {
            $this->assertContains($needle, $email->subject, $description);
        }
    
        public function assertEmailSubjectEquals($expected, $email, $description = '')
        {
            $this->assertContains($expected, $email->subject, $description);
        }
    
        public function assertEmailHtmlContains($needle, $email, $description = '')
        {
            $response = $this->mailcatcher->get("/messages/{$email->id}.html")->send();
            $this->assertContains($needle, (string)$response->getBody(), $description);
        }
    
        public function assertEmailTextContains($needle, $email, $description = '')
        {
            $response = $this->mailcatcher->get("/messages/{$email->id}.plain")->send();
            $this->assertContains($needle, (string)$response->getBody(), $description);
        }
    
        public function assertEmailSenderEquals($expected, $email, $description = '')
        {
            $response = $this->mailcatcher->get("/messages/{$email->id}.json")->send();
            $email = json_decode($response->getBody());
            $this->assertEquals($expected, $email->sender, $description);
        }
    
        public function assertEmailRecipientsContain($needle, $email, $description = '')
        {
            $response = $this->mailcatcher->get("/messages/{$email->id}.json")->send();
            $email = json_decode($response->getBody());
            $this->assertContains($needle, $email->recipients, $description);
        }
    ?>
    

    The complete code listing is published as gist.

    Example

    How a test using this may this EmailTestCase class may look like?

    <?php      
        function testNotificationIsSent()
        {
            // ... trigger notifications
    
            $email = $this->getLastMessage();
            $this->assertEmailSenderEquals('<bugira@bugira.com>', $email);
            $this->assertEmailRecipientsContain('<davert@ukr.net>', $email);
            $this->assertEmailSubjectEquals('[Bugira] Ticket #2 has been closed', $email);
            $this->assertEmailSubjectContains('Ticket #2', $email);
            $this->assertEmailHtmlContains('#2 integer pede justo lacinia eget tincidunt', $email);
        }
    ?>    
    

    We got a really simple class for testing emails from your application. Ok, that's not a unit testing. For unit testing you should use mocking framework to fake the delivery in your PHP code. But you can use this class in acceptance tests (with Selenium) or integration tests. It is much simpler to test emails this way, then to dig into internals of your email sending library and defining mocks. The drawback here is usage of standalone daemon, and reconfiguring your application to use its SMTP server.


    It looks like this post is long enough to be published. We will continue email testing next time with Codeception framework. We will develop EmailHelper class for scenario-driven tests of Codeception.

    EmailTestCase Source


    0 0
  • 01/30/14--14:03: Codeception 1.8.2: Bugfixes
  • Time passed since the previous release. Since 1.8.1 we got a nice list of significant bugfixes and we are ready to publish new stable release. No new features added, actually, but they are coming in new 2.0 branch which is actively developed in this days.

    Changes:

    • [REST] match similar elements in JSON arrays fixed in #837 by blacknoir.
    • generate:pageobject now takes -c option correctly #809
    • [REST] Fixed setting Content-Type header #827
    • [REST] Headers are uppercased according to BrowserKit standard.
    • [Db] records inserted with haveInDatabase now are cleaned in _after event by dirk-helbert #761.
    • [Laravel] Fixed usage of Redirect::back in tests
    • Fixed collecting CodeCoverage using WebDriver/Selenium2 modules.
    • [REST] Fixed "Call to undefined method Symfony\Component\HttpFoundation\Request::getCookies() in codeception/codeception/src/Codeception/Module/REST.php line 352" by casconed #814
    • Fixed: tests run twice if you use .dist.yml config by tomtomsen #582
    • Environments: Test classes was loaded only once in multi-environment mode. Fixed by ayastreb and his nice tokenizer solition #812
    • Excluding running abstract classes in addCest by filipgorny #792
    • [PhpBrowser] Fixed setting cookies from headers
    • [Framework] Form data on page was not cleaned after form submit. So when sending the same form twice, cached data was submitted. This is now fixed.
    • and others...

    Also we've got a Mockery module for using powerful Mockery framework with Codeception. It is not included in Codeception itself but can be installed via Composer. (Thanks to Jáchym Toušek)

    Thanks to all contributors, thanks for getting bug fixed and reported. We are trying to be better with each release.

    Update

    redownload your codeception.phar for update:

    1.8.2

    php codecept.phar self-update
    

    for composer version

    $ php composer.phar update codeception/codeception
    

    What's Next

    We are moving to Codeception 2.0. It's first alpha is expected on next week. Stay tuned and get ready for new features!

    While developing Codeception 2.0 we released a lightweight task runner Robo. We needed it to run routine tasks for Codeception - building phar archives, merging releases, etc. You may use it as lightweight alternative for Phing or PHP alternative for shell scripts. It's not very documented, it doesn't include list of all required tasks, but it will evolve. If you have ideas on improvement, or you want to add more tasks into it - please send Pull Requests.

    And thanks to @pfaocle for the feedback on using Robo!


    0 0

    Here goes another minor release with some fixes and improvements. Codeception 1.8 now supports Laravel 4.1 and Yii2 and is tested for this frameworks on Travis. Also Laravel and Yii modules got some nice new actions for database interactions.

    Laravel, Yii2, and Phalcon frameworks implement ActiveRecord pattern. That's why all database actions in this modules look and work in a very similar manner.

    Laravel 4.1

    <?php
    $user_id = $I->haveRecord('users', array('name' => 'Davert'));
    $I->seeRecord('users', array('name' => 'davert'));
    $I->dontSeeRecord('users', array('name' => 'davert'));
    $user = $I->grabRecord('users', array('name' => 'davert'));
    ?>

    This methods will work for Laravel 4 as well, but Laravel 4.1 supports nested transactions and allows us to wrap functional test into one database transaction. This is really useful, as we can rollback any database changes we do in functional tests. Tests also run really fast, as nothing is written in database. A new cleanup config option was introduced to Laravel4 module, and by default it is turned on.

    Now it is really simple to use database in your functional tests. Don't hesitate and try *Record methods in action!

    Also nice seeSessionErrorMessage was added by elijan to perform validation error assertions.

    Yii2

    Yii2 is in very active development, and its official basic application is tested with Codeception, and uses Specify and Verify libraries. Yii2 module is tested on Travis as in official Yii2 repo and in Codeception repo as well.

    <?php
    $user_id = $I->haveRecord('app\model\users', array('name' => 'Davert'));
    $I->seeRecord('app\model\users', array('name' => 'davert'));
    $I->dontSeeRecord('app\model\users', array('name' => 'davert'));
    $user = $I->grabRecord('app\model\users', array('name' => 'davert'));
    ?>

    ActiveRecord methods work in very similar manner. We expect that Yii2 will have nested transactions support before the release.

    Thanks to Ragazzo for Yii2 module contributions and bughunting.

    Bugfixes

    • CodeCoverage was improved. Remote codecoverage with WebDriver or PhpBrowser opens page, sends authorization cookie before the test.
    • WebDriver cookies (and sessions) are destroyed after the test. If you have troubles when session is not restared in WebDriver - toggle restart option.

    Update

    redownload your codeception.phar for update:

    1.8.3

    php codecept.phar self-update

    for composer version

    $ php composer.phar update codeception/codeception

    P.S. Yeah, yeah, Codeception 2.0 is on its way.


    0 0
  • 02/17/14--17:03: Codeception 2.0 alpha
  • Finally we are ready to show you Codeception 2.0. We tried not to break everything (as it was supposed for major version change), but keep things work as they are, but maybe in a different way. Let's review the most important changes:

    Codeception: Not Just For Guys

    Before 2.0 there were only Guys in it. That is not fair! We wanted to make Codeception a tool for everyone: for guys, girls, developers, test engineers, ninjas and even wookiees... (nope, wookiees should wait for 3.0).

    That's why you can now choose the desired actor from a list during the bootstrap process.

    Before proceed you can choose default actor:
    
    $I = new [ACTOR]
      Select an actor. Default: Guy  
      [0] Guy
      [1] Girl
      [2] Person
      [3] Engineer
      [4] Ninja
      [5] Dev
    

    so you all your test actors (that's how we call guy classes now) will be in form like TestGirl, WebNinja, CodeDev, etc. Pretty flexible.

    Codeception: Not Alone

    Before Codeception 2.0 guys (or should we say actors now) were left to themselves. You know, it is so sad to see that there is only $I in the test, forever alone, like on a desert island, or in space... But in 2.0 you can invite some friends into your tests. Let's say...

    <?php
    $I = new WebGuy($scenario);
    $nick = $I->haveFriend('nick');
    ?>

    So we can write multi-session tests that can be executed in two browser windows. You may try to run this test on github.com to see how it works:

    <?php
    $I = new WebGuy($scenario);
    $I->wantTo('surf Github with Nick');
    $I->amOnPage('/');
    $I->submitForm('#top_search_form', array('q' => 'php-webdriver'));
    $nick = $I->haveFriend('nick');
    $nick->does(function(WebGuy $I) {
        $I->amOnPage('/Codeception/Codeception');
        $I->click('Issues');
        $I->canSeeInTitle('Issues');
    });
    $I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
    $I->seeInTitle("php-webdriver");
    $nick->does(function(WebGuy $I) {
        $I->click('Milestones');
        $I->canSeeInTitle('Milestones');
    });
    $I->seeCurrentUrlEquals('/facebook/php-webdriver');
    $I->click('Issues');
    $I->canSeeInTitle('Issues');
    ?>

    As you see, everything in does closure is executed in separate session. This way you can test user-to-user interactions on your site. For example Alice writes a private message to Bob and Bob responds back. Similarly you can have multiple REST sessions in a test.

    Such scenario cases can be implemented. That's what friends are for.

    Notable Changes

    As it was announced earlier, the main goals for Codeception 2.0 was internal refactoring.

    • Mink (and its drivers) was removed completely. Instead you can use WebDriver module to do Selenium testing, and PhpBrowser (which uses Goutte) for browser-emulation. PhpBrowser module is now more compatible with frameworks modules, they use the same methods and acts in the same manner. If you were using Selenium or Selenium2 module you should switch to WebDriver, for PHPBrowser everything should (crossing fingers) work smoothly.
    • 2-phases test execution with tricky magic including usage of Maybe class was removed. Tests are now executed one time, like any regular PHP file. So you can use any PHP code in your tests, and appearance of Maybe object would not confuse you anymore.
    <?php
    $card = $I->grabTextFrom('#credit_card')
    var_dump($card); // was showing `Maybe` instead of real value
    ?>
    • Codeception 2.0 require PHP 5.4 and higher. Time changes, PHP 5.3 is getting harder and harder to support, thus we decided to move to 5.4 and keep our code base up to date. And yes, we wanted to use short array syntax. We are tired to keeping write all those nasty array() stuff.
    • Actor classes (initial Guy classes) are now rebuilt automatically. Thus, you want get exception when change suite configuration, or add methods to helper. Rebuilds are not required anymore.
    • Added Assert module that can be used to write common asserts in your tests. You can now use seeEquals, seeContains and other actions inside your Cepts or Cests.
    • Experimental: added Silex module. We need your Feedback on using it.

    Minor Internal Changes

    Refactoring, Refactoring, Refactoring. We use PSR-2 now. We rewrote CodeCoverage. We have better directory structure... More new files. What else? Oh, lets admit it, these are not the changes you would actually notice. But internals are now more clean and easy to understand (Except the parts which heavily rely on PHPUnit).

    Upgrading

    We'd like to ask you to try Codeception 2.0 on your projects. Before the release is ready we need to collect feedback and fix all encountered issues. You know where GitHub issues are.

    Download:

    wget http://codeception.com/releases/2.0.0-alpha/codecept.phar

    Via Composer:

    composer require --dev "codeception/codeception:2.0.0-alpha"

    Development of 2.0 now happens in master branch, so keep track on changes and send your pull requests.

    Some Upgrading Notes

    • Run build command
    • Replace Selenium2 to WebDriver module
    • Check you don't use PHPBrowser->session property anywhere (it was Mink part)
    • CodeCoverage with c3 will require new version of c3.

    What's next?

    We need your feedback, and meanwhile we will work on updating documentation parts. 1.8.x will be maintained, but new features will be added to 2.x branch.


    0 0
  • 03/17/14--18:03: Codeception 2.0 beta
  • It took about a month to get to the beta release. This release brings lots of changes even a few BC breaks and clearly shows what Codeception 2.0 is planning to be. Today we will announce more technical changes, and not conceptual ones. As it was stated previously: Codeception 2.0 will bring lots of small improvements, will be much easier to extend, and understand. Changes for beta release were not planned initially, but were added to help Codeception evolve during the 2.0 branch development. Let's list briefly all major changes and see how to deal with them. Documentation for 2.0 branch can be found on GitHub and will be published on RC release.

    Changes

    • Upgraded to PHPUnit 4.0
    • Upgraded to facebook/webdriver 0.4
    • RunFailed extension added. To rerun tests on fail include Codeception\Platform\RunFailed extension, and call codecept run -g failed.
    • Logger disabled by default and moved to Extension. You will need to install Monolog by yourself, and enable Codeception\Platform\Logger as extension.
    • Methods _before/_after of Cest can use $I object.
    • Destination for xml/html/reports can be customized. For instance
    codecept run --xml myreport.xml
    codecept run --xml /home/davert/report.xml
    codecept run -g database --xml database.xml
    ```
    • --coverage + --xml or --html won't produce xml or html codecoverage output. Use new options coverage-xml and coverage-html instead. They were added so you could specify custom destination for codecoverage reports as well.
    • you can get current environment in a test by accessing $this->env (even in Cept)
    • shortcut functions added: codecept_debug to print custom output in debug mode, codecept_root_dir, codecept_log_dir, codecept_data_dir for retrieving Codeception paths.
    • extensions can change global config before processing.

    Breaking Changes

    In Codeception 1.x bootstrap files were used differently for different types of tests. They were introduced to pass variables (and fixtures) into Cept scenarios. They were almost useless in Cests, and absolutely useless for Test files. They are not removed at all, but now they are loaded only once before the suite. If you were using bootstrap files for setting fixtures, you still can use them, but you will need to require them manually.

    <?php
    require __DIR__ . '/_bootstrap.php'
    
    $I = new WebGuy($scenario);
    ?>

    The same way you can load bootstrap files into Cests. But we recommend to create _fixtures.php file, and use bootstrap file for suite initialization.

    Dynamic Groups

    One significant internal feature is dynamic groups. You can save test names into file and run them inside a group. That's how the RunFailed extension works: it saves names of failed tests into file tests/_log/failed, then execute codecept run -g failed to run these tests.

    groups:
      # add 2 tests to db group
      db: [tests/unit/PersistTest.php, tests/unit/DataTest.php]
    
      # add list of tests to slow group
      slow: tests/_log/slow

    For example, you can create the list of the most slow tests, and run them inside their own group. Pretty interesting and powerful feature that also can be used to run tests in parallel.

    Parallel Test Execution

    Not yet! Codeception does not support parallel test execution, nor will be provide it out of the box. That's because there is no solution that will fit for everyone. You may want to run parallel tests locally in threads via pthreads extension, or run them on different hosts via SSH, AMQP, etc. But we already have everything to implement parallel testing.

    Here is the algorithm:

    • split all tests into groups (with dynamic groups)
    • run each groups separately
    • merge results

    And we will write guides on implementing this algorithm, as well as we plan to release a sample tool that will run tests in parallel.

    Try it

    As usual, we need your feedback to get 2.0 released as stable.

    Download:

    wget http://codeception.com/releases/2.0.0-beta/codecept.phar

    Via Composer:

    composer require --dev "codeception/codeception:2.0.0-beta"

    P.S. Our sponsors 2Amigos updated their site and logo. Check it out!


    0 0

    Bugfix release on stable 1.8 branch. While you are waiting for 2.0 RC and final version, you can update your current Codeception to get the newest patches and updates. Here they are:

    • [WebDriver] Correctly return to main page when switchToIFrame() is called by n8whnp.
    • [WebDriver] fixed screenshot capture by FnTm.
    • [Frameworks] More details in debug output by Ragazzo.
    • Fixed problem with encoding in $I->wantTo() by vgoodvin.
    • Add clone and unset capabilites to Maybe by brutuscat.
    • [WebDriver] Fixes to submitForm when typing password by pcairns.
    • [Phalcon1] haveRecord return id even when id is not public by xavier-rodet.
    • Modules to be loaded from global context by Ragazzo. Now you can pass long name of module into module configuration:
    class_name: WebGuy
    modules:
        enabled: [\My\Custom\Module, WebHelper]

    Really helpful feature if you have multiple Codeception inits in one project and you want to have shared module.

    All those features were included in 2.0-beta. Thanks to all contributors!

    Update

    redownload your codeception.phar for update:

    1.8.4

    php codecept.phar self-update

    for composer version

    $ php composer.phar update codeception/codeception

    0 0
  • 04/04/14--18:03: Codeception 1.8.5
  • New improvements and bugfixes came during the last month into the 1.8 branch. We summarized them all and prepared a new release.

    Ecosystem

    But before we proceed you should check out the new Addons section on this site. We had only extensions listed there, but now you can find new modules and even applications there. And let's name at least two projects you should definitely be aware of:

    Thank you for all extensions and modules developers. If you developed a module, helper, or extension, share your work with others by adding your project to addons page (click Edit link in the bottom).

    Bugfixes

    • [WebDriver] facebook/webdriver version was set strictly to 0.3. And this won't be changed in future for 1.8 branch. We accidentaly commited ~0.3 into the 1.8.4 which made composer to install 0.4 version of facebook/webdriver. There was API break in 0.4, so WebDriver module didn't work properly.
    • [Phalcon] Phalcon 1.3 compatibility added #944
    • [Laravel] Service providers are now correcly loaded by ShawnMcCool.
    • [Laravel] findRecord fixed by andkom.
    • [ZF2] Fixed handling of query parameters by piccagliani.
    • [Frameworks] Fixed problem with submitting form with input[type=image] by piccagliani.

    Features

    • [Webdriver][Frameworks] input field can now be located by its name.
    • [Webdriver][Frameworks] element can be clicked by name too.

    Update

    redownload your codeception.phar for update:

    1.8.5

    php codecept.phar self-update

    for composer version

    $ php composer.phar update codeception/codeception

    0 0

    Codeception 2.0 is almost ready for final release. Thanks for everyone who already running alpha/beta version of Codeception and provides us with valuable feedback. If everything goes well the final release will be published in a week. The delay between releases is dictated by the need to allow parallel tests execution. As it was said before: Codeception won't provide parallel test running out of the box, yet we will add a new chapter into guides where you will learn how to implement parallelism model that matches needs of your project.

    Improvements

    Strict Locators

    In order to provide better element locations you can now specify locators explicitly like this: ['css' => 'input.name']. This will work for PHPBrowser, Frameworks modules, and WebDriver.

    <?php
    $I->fillField(['css' => 'input.name'], 'jon');
    $I->click(['link' => 'Submit']);
    $I->seeElement(['xpath' => './/div[@data-name="jon"]']);
    ?>

    Allowed locator mechanisms are:

    • id
    • name
    • css
    • xpath
    • link
    • class

    You can use strict locators in your Page Object classes as well.

    See Element with Attributes

    To improve checking element with expected attributes seeElement now takes second parameter:

    <?php
    $I->amOnPage('/form/field');
    $I->seeElement('input[name=login]');
    $I->seeElement('input', ['name' => 'login']); // same as above
    ?>

    dontSeeElement works in corresponding manner.

    grabAttributeFrom method added

    To fetch value of attribute of element:

    <?php
    $I->amOnPage('/search');
    $attr = $I->grabAttributeFrom('form', 'method');
    $attr == 'GET'
    ?>

    Also makeScreenshot method of WebDriver was simplified. Filename of saved screenshot does not include a test name.

    Try it

    Download:

    wget http://codeception.com/releases/2.0.0-RC/codecept.phar

    Via Composer:

    composer require --dev "codeception/codeception:2.0.0-RC"

    0 0

    Unfortunately this is not a final release of Codeception 2.0 you may have expected. Still we had to get another release candidate in order to implement one breaking and important change: upgrade to Guzzle version 4.

    As you know PHPBroswer of Codeception used Mink, Goutte, and Guzzle to emulate visiting pages, filling forms, and other user-like web interaction. Recently Guzzle got a new major release, so we had to upgrade Codeception as well. We also removed Mink and Goutte and Codeception uses Guzzle directly. This will allow us faster to hunt down and fix the issues which may appear in future.

    Guzzle changed their package name and namespace in latest release. If you were using Guzzle 3 and you want to upgrade to Codeception 2 you won't get any conflicts in dependencies. But if you were using Guzzle\Http\Client class in your tests or helpers you will to change it to GuzzleHttp\Client.

    PHPBrowser additional configuration

    Guzzle4 got very flexible configuration. You may omit CURL_ constants and use request parameters to set headers, auth, cookies, and SSL verification. Codeception now accepts all request parameters of Guzzle in module configuration part:

    modules:
       enabled: [PhpBrowser]
       config:
          PhpBrowser:
             url: 'http://localhost'
             auth: ['admin', '123345]
             headers:
                'User-Agent': 'Codeception'
                Accept: 'application/json'
             curl:
                 CURLOPT_RETURNTRANSFER: true

    And yes, curl configuration section is left for more advanced options and for backwards compatibility. BTW, SSL settings should be also defined via new configuration using verify, cert, ssl_key request options.

    Try it

    As usual we need your feedback. Guzzle upgrade is very dramatic change and we recommend to try it on your test suites. If you have issues report them and we will try to fix them before the final release, which is schenduled for next week.

    Download:

    wget http://codeception.com/releases/2.0.0-RC2/codecept.phar

    Via Composer:

    composer require --dev "codeception/codeception:2.0.0-RC2"

    0 0
  • 06/05/14--18:03: Codeception 2.0 Final
  • Codeception 2.0 is out. It was a long journey to get here but it was worth it. Main purpose of making Codeception 2.0 was not to break things up. Instead we focused on making Codeception simpler in use and reduce that awkward WTF moments you were experiencing rebuilding Guy classes and discovering dark magic of Maybe object.

    That's it. Codeception does not have 2-phases of execution and runs your tests as native PHP scenarios. It also does not require you to rebuild actor classes on configuration change - it is done automatically. And at last we don't have Guys anymore. They were renamed in more neutral Tester objects. But it is up to you how will you call your actors: Testers, Guys, or even ninjas, those names can be configured!

    Naming Conventions

    We decided to change some naming conventions in Codeception.

    • _log directory was renamed to _output. It was logical decision, as Codeception does not print any logs by default, but constantly uses _log directory for printing debug information and reports.
    • _helpers was renamed to _support. Now it is more obvious that this directory may contain not only helpers but any kind of support code required for tests.

    As it was said, Guys were renamed to Testers. Actor classes and helpers are named by suite. For instance, acceptance test will start with this line:

    <?php
    $I = new AcceptanceTester($scenario);
    // and uses AcceptanceHelper
    ?>

    In unit tests we simplified access to actor classes by just using $this->tester property (actor title). Looks a bit more readable.

    <?php
    function testSavingUser()
    {
        $user = new User();
        $user->setName('Miles Davis');
        $user->save();
        $this->tester->seeInDatabase('users',array('name' => 'Miles', 'surname' => 'Davis'));
    }
    ?>

    If you are upgrading from 1.x you can still use codeGuy as it was left for compatibility.

    Listed naming changes are recommended defaults. You don't need to follow them if you plan to upgrade from 1.x.

    Changes

    Let's briefly name some important changes added in Codeception 2.0

    • PHP >= 5.4
    • Upgraded to PHPUnit 4.x
    • Upgraded to Guzzle 4.x
    • Removed Mink (and Selenium, Selenium2, ZombieJS modules)
    • Removed Goutte
    • Logger moved to extension and disabled by default
    • Naming conventions changed
    • One-time execution for all tests
    • Autorebuild of actor classes
    • MultiSession testing with Friends
    • Strict Locators introduced
    • Fail fast option -f added to run command
    • Coverage options changed behavior in run command
    • Bootstrap test are loaded suite, not before each test
    • Parallel Execution (!!!)

    Guides

    Guides were reviewed and updated for 2.0 release. Unit Testing guide was completely rewritten, and new chapter Parallel Execution was added. Please note that parallel running tests is advanced feature and will require some time for you for set everything up correctly. But If your tests run longer then 30 minutes, you will get a real benefit out of it.

    The Most Important Bug Fixed

    The Most Important Bug was reported by many of our users:

    We decided to fix it for 2.0 release. Now the date in footer is displayed correctly according to PHP rules (using <?=).

    Upgrading

    Codeception 2.0 was publishing pre-releases since February. Lots of developers already gave it a try and reported their issues. We hope the upgrade will go smoothly for you. Basically what you need is to:

    • run build command
    • remove any internal usage of Mink or Goutte
    • remove any usage of $scenario->prepare() as it always return false now
    • optionally remove $scenario->running() calls and $I->execute calls. They are not required anymore.
    • take a look into PhpBrowser module which is uses Guzzle4 directly. You may have issues with using it (as it was completely rewritten). Please report those issues to GitHub.
    • Selenium2 module should be replaced with WebDriver (as you already didn't do so).
    • if you use variables in _bootstrap.php, rename it to _fixtures.php and load it manually into your tests.

    If you have issues during upgrade send issues to GitHub and we will make upgrade more smoother.

    wget http://codeception.com/codecept.phar

    Via Composer:

    composer require --dev "codeception/codeception:2.0.0"

    P.S. Codeception 1.8.6 was released as well. List of its changes will go tomorrow.


    0 0

    First bugfix release from the 2.0 version. This release introduces very important fix for running Codeception with PHP 5.4.x. Due to bug in PHP you would see strange errors while trying to execute Cept files. We can't say which exect PHP 5.4 versions were affected, but if you had issues running Codeception 2.0 earlier, please upgrade.

    The next notable change is see method of WebDriver module. Now it checks for a visible text on a page. If you need to check if text is present inside HTML source of a page you can use newly added seeInPageSource method for that. Thanks artyfarty for this bugfix.

    Update

    redownload your codeception.phar for update:

    php codecept.phar self-update

    for composer version

    $ php composer.phar update codeception/codeception

    Changelog Page

    From now on there will be no blogposts for minor releases. There is new changelog page which displays changes in latest releases. You should refer to it in order to follow the updates. Important changes, features, bugfixes will be mentioned there.


    0 0
  • 06/27/14--18:03: Unit Testing With Database
  • He-hey! You just opened this page to say that there is no such thing as unit testing with database. Unit tests are supposed to test isolated pieces, and sure unit tests should not touch the database. And we can agree with that. But it happened that "unit test" term is far more popular then "integration test". Unit test have one very strong requirement. Isolation. Complete isolation from other code units and services. In real world we can spoil our lives trying to achieve complete isolation. And even when with pain and sweat we finally got out models isolated from database, we receive a very strange class of tests. They do not provide a valuable feedback. Model test that does not connect to real database is useless, as it provide false positive results, and the code behind may crash connecting to real storage. So what do we want from unit test after all? To be written in complete isolation (what for?), or provide real feedback?

    So let's not to concentrate on terms. Let's concentrate on testing.

    Testing Faster With Transactions

    What drawbacks we receive if unit tests are connecting to database? Sure it will be dramatic slowdown. We will need to connect to database, insert data, and perform a cleanup afterwards. The most simple way of cleaning things up is to repopulate database completely. Yet, this is the slowest way. But there is a much better way of doing things. Especially for unit (and functional) tests. We can use transactions.

    That's right. We can cover a test into transaction, and rollback it after the test is finished. This is the fastest and the most stable way of working with database in unit tests. The only issue with it is that not all databases support nested transactions. For instance, in MySQL you can't begin transaction and begin next transaction as there can be only one at one point. Yet, those nested transactions can be emulated. And here are good news: if you use one of popular PHP Frameworks: Laravel, Yii2, Phalcon, or Doctrine ORM, your framework already can do that! Thus, in unit test you can begin transaction in setup, and finish it in teardown of a test. Sure, you should use the framework's database layer for that.

    In Codeception we already use this feature for speeding up functional tests. But it was not that obvious that you can use the same practice for unit tests.

    Let's connect our framework to the unit.suite.yml. In current example we will use Laravel, yet very similar steps can be done for Yii2, Phalcon, and Symfony+Doctrine.

    # Codeception Test Suite Configuration
    
    # suite for unit (internal) tests.
    class_name: UnitTester
    modules:
        enabled: [Laravel4, UnitHelper, Asserts]

    That's all. This line of Laravel module will start the transaction before each test in suite.

    Using ActiveRecord Helpers

    Laravel, Phalcon, and Yii are frameworks that use ActiveRecord pattern for working with database. Codeception Framework modules have built-in helpers, which share the standard interface to work with database records.

    Let's write a sample Laravel test which demonstrates usage of Codeception helpers. In this example we will test activate method of User model. It activates user when provided activation key matches predefined one.

    <?php
    class UserTest extends \Codeception\TestCase\Test
    {
        /**
         * @var UnitTester
         */
        protected $tester;
    
        protected $user_id;
    
        function _before()
        {
            // preparing a user, inserting user record to database
            $this->user_id = $this->tester->haveRecord('users', [
                'username' => 'John',
                'email' => 'john@snow.com',
                'activation_key' => '123456',
                'is_active' => 0
    
            ]);
        }
    
        function testUserCanBeActivatedWithValidKey()
        {
            // lookup for user with Eloquent API
            $user = User::find($this->user_id);
            // executing action
            $isActivated = $user->activate('123456'));
            // performing assertion
            $this->assertTrue($isActivated);
            // checking that data was actually saved into database
            $this->tester->seeRecord('users', [
                'id' => $this->user_id,
                'is_active' => 1
             ]);
        }
    
        function testUserNotActivatedWithInvalidKey()
        {
            $user = User::find($this->user_id);
            $this->assertFalse($user->activate('00000'));
            $this->tester->dontSeeRecord('users', [
                'id' => $this->user_id,
                'is_active' => 1
             ]);
        }
    }
    ?>

    As it was mentioned, similar test can be written for other mentioned frameworks. Depending on framework API ActiveRecord helper methods

    • grabRecord
    • seeRecord
    • haveRecord
    • doneSeeRecord

    may take table name (Laravel) or model name (Phalcon, Yii2).

    Who is that Tester?

    $this->tester object used here is instance of Actor class and contain all methods of modules and helpers used in unit suite. It would be a good idea to put any shared code into helpers, and reuse across test cases by accessing $this->tester. This object is injected into any testcase that extends Codeception\TestCase\Test.

    Conclusion

    Sure, we could implement similar test using just Eloquent API without using Codeception helpers. Still, they are quite useful. Especially assertion methods like seeRecord which queries database for a record with given attributes.


    0 0
  • 10/11/14--18:03: Codeception Updates
  • Codeception is improving from day to day with the help of our contributors. Recently we had new 2.0.6 release with lots of bugfixes and new features. Unfortunately some not obvious regressions were introduced, and we had to release 2.0.7 as fast as we could.

    In order to maintain a predictable release cycle we plan to release a new bugfix version each month and deliver a new major version each 6 months. We tend to follow this plan for future, yet we are not ready to follow it strictly.

    Today we have a few more announcements to sum up all recent work on Codeception and its support projects.

    CodeCoverage

    Latest updates to c3 brought full compatibility with Codeception 2.0. Also c3 can be installed and updated via Composer.

    "require-dev": {
        "codeception/c3": "2.*"
    },
    "scripts": {
        "post-install-cmd": [
            "Codeception\\c3\\Installer::copyC3ToRoot"
        ],
        "post-update-cmd": [
            "Codeception\\c3\\Installer::copyC3ToRoot"
        ]
    }

    This allows to follow updates of code coverage collector (c3) easily. Also please note, that c3 issues were moved to main Codeception repo.

    AspectMock

    AspectMock was recently updated to version 0.5.0. Latest update is powered by Go Aop framework version 0.5. In new AspectMock fixed passing parameters by reference, and added ability to mock PHP functions.

    AspectMock allows to replace PHP function in desired namespace with user-defined closure or a return single return value. For instance, you can stub file operations by mocking file_get_contents or file_put_contents. You can also mock time functions, in order to make tests stable to time changes.

    <?php
    namespace App\Cache;
    
    $func = test::func('App\Cache', 'file_put_contents');
    $cache = new FileCache;
    $cache->store('hello'); // calls file_put_contents
    $func->verifyInvoked(); // verifies file_put_content was called
    test::cleanup(); // restore original function
    ?>

    Same can be done for time functions:

    <?php
    namespace demo;
    test::func('demo', 'time', 'now');
    $this->assertEquals('now', time());
    ?>

    You can install latest AspectMock via Composer:

    composer require "codeception/aspect-mock:~0.5"

    Codeception 2.1

    Next major release of Codeception will include new features, breaking changes, with ability to keep backwards compatibility. Here are the most interesting features you should wait for:

    • All support classes like actors, steps, pages, etc, will be moved to tests/_support directory.
    • Support classes will use PSR-4 standard.
    • Dependency Injection. All support and test classes can be included one into another: Pages can be injected into Helpers, Helpers can be injected into Cest classes, etc.
    • Module Conflicts. Lots of issues are happening when users try ot include modules with same apis into one suite. Like PhpBrowser + Laravel4 + WebDriver. Those module is possible to be used together, but possibly that was not really intended to be used this way. Call to $I->amOnPage may be executed by any provided modules depending on the order of their declaration. This leads to unpredictable errors, and is the source of confusion. In 2.1 Codeception will allow to define module conflicts, and throw exceptions if modules are not expected to be used together.
    • ...and more

    Global Installation

    One of recently discussed topics for Codeception, was global installation of Codeception. From now on you can install Codeception globally using Composer:

    composer global require 'codeception/codeception=2.0.*'

    This will be recommended installation method since Codeception 2.1.

    RoboCI

    This project that desires its own post in this blog. It is open-source reproduction of Travis CI using Docker and Travis Cookbooks. RoboCI can be use to execute Travis CI tests locally or on remote server. It does not require VirtualBox nor Vagrant installed. It was executed with Codeception internal tests, with Selenium + Firefox, MySQL, PostgreSQL, MongoDb, and other services. You can read more about RoboCI and try it in your projects. In future we plan to provide consulting services on using RoboCI with Codeception or any other PHP testing framework. Stay in touch for updates!


    0 0

    One of the greatest things of testing Ruby on Rails applications was usage of Factories, managed by FactoryGirl. It changed the way the test data was managed in test. Instead of defining fixtures one by one, in a single .yml configuration, it allowed to generate required models per test. This concept is very convenient, however it is not widespread in PHP world. Probably, this was because, there is no single ORM in PHP, like ActiveRecord is for Ruby. We have Doctrine, Eloquent, and each major framework has its own ORM layer, so that we it is pretty hard to deal with models of those frameworks in one manner.

    But it looks like factories finally came to PHP! The League of Extraordinary Packages produced a great package, self-claimed as factory_girl for PHP. Meet Factory Muffin. It allows simple generation of database records for integration and functional testing. Let's see how it can be used with Codeception.

    Setup in Codeception

    At first lets add league/factory-muffin": "~2.0 to composer.json:

    "require-dev": {
        "codeception/codeception": "~2.0",
        "league/factory-muffin": "~2.0"
    }

    Then we execute composer install.

    Loading composer repositories with package information
    Updating dependencies (including require-dev)
      - Installing fzaninotto/faker (v1.4.0)
        Downloading: 100%         
    
      - Installing league/factory-muffin (v2.1.1)
        Downloading: 100%         
    
    Writing lock file
    Generating autoload files
    Generating optimized class loader

    As you may noticed, FactoryMuffin uses awesome Faker library for generating random data for models.

    For using factories inside tests we will create FactoryHelper:

    php vendor/bin/codecept g:helper Factory
    Helper /home/davert/project/tests/_support/FactoryHelper.php created

    We will define factories in the _initialize method of FactoryHelper. Let's say we have two models, Post and User in application. This is how we specify rules for generating new objects of these models.

    <?php
    class FactoryHelper extends \Codeception\Module
    {
        /**
         * @var \League\FactoryMuffin\Factory
         */
        protected $factory;
        
        public function _initialize()
        {
            $this->factory = new \League\FactoryMuffin\Factory;
            $this->factory->define('Post', array(
                'title'    => 'sentence|5', // title with random 5 words
                'body'   => 'text', // random text
            ));
    
            $this->factory->define('User', array(
                'email' => 'unique:email', // random unique email
            ));
        }
    }
    ?>

    This is how generation of Post and User is defined. From the box, Muffin is designed to work with ActiveRecord models, and with Eloquent in particular. So you can use it in Yii, Phalcon, yet we will demonstrate its usage in Laravel application. FactoryMuffin can also be customized to work with Doctrine as well.

    Using Factories in Laravel

    To use FactoryMuffin in Laravel functional tests we need to create additional methods in FactoryHelper for generating and saving records: havePosts and haveUsers methods. They will populate database with number of records specified. We will need to clean up those records at the end of a test.

    <?php
    class FactoryHelper extends \Codeception\Module
    {
        /**
         * @var \League\FactoryMuffin\Factory
         */
        protected $factory;
        
        public function _initialize()
        {
            $this->factory = new \League\FactoryMuffin\Factory;
            $this->factory->define('Post', array(
                'title'    => 'sentence|5', // title with random 5 words
                'body'   => 'text', // random text
            ));
    
            $this->factory->define('User', array(
                'email' => 'unique:email', // random unique email
            ));
        }
    
        public function havePosts($num)
        {
            $this->factory->seed($num, 'Post');
        }
    
        public function haveUsers($num)
        {
            $this->factory->seed($num, 'Post');
        }
    
        public function _after(\Codeception\TestCase $test)
        {
            // actually this is not needed
            // if you use cleanup: true option 
            // in Laravel4 module
            $this->factory->deleteSaved();
        }
    }
    ?>

    By including FactoryHelper to the functional suite config, we can use it inside the actor ($I) object.

    This allows us to test features like pagination. For instance, we can check that only 20 posts are listed on a page:

    <?php
    $I = new FunctionalTester($scenario);
    $I->wantTo('check that only 20 posts are listed');
    $I->havePosts(40);
    $I->amOnPage('/posts');
    $I->seeNumberOfElements('.post', 20);
    ?>

    Source code of this example is on GitHub.

    Factories in Unit Tests

    Factories can also be used in unit testing. Let's say user can create posts, and in order to optimize queries we are saving number of user posts in num_posts column of users table. We are going to test that this column is updated each time a new post by user is created.

    We will need one more method added into FactoryHelper class:

    <?php
    public function produce($model, $attributes = array())
    {
        return $this->factory->create($model, $attributes);
    }
    ?>

    After we include FactoryHelper into unit suite we can use its methods in tests:

    <?php
    class UserTest extends \Codeception\TestCase\Test
    {
        /**
         * @var UnitTester
         */
        protected $tester;
    
        function testCounterCache()
        {
            $user = $this->tester->produce('User');
            $this->assertEquals(0, $user->num_posts);
    
            $this->tester->produce('Post', ['user_id' => $user->id]);
            $this->assertEquals(1, User::find($user->id)->num_posts);
    
            $this->tester->produce('Post', ['user_id' => $user->id]);
            $this->assertEquals(2, User::find($user->id)->num_posts);
        }
    }
    ?>

    Conclusion

    As you see, factories make your tests clean and expressive. Forget about managing test data manually, forget about fixtures. Use FactoryMuffin.

    Update FactoryHelper for Symfony2 and Doctrine modules


    0 0

    Selenium PhantomJS

    This post summarizes some recent work that was done for creating portable testing environments. The idea was to reduce dependencies required by acceptance testing. As you know, testing of web pages require Selenium or PhanromJS servers, which allow to interact with web page, and execute user scripts on page. However, installing those dependencies, can be a pain. Especially Selenium. If you are not running it inside desktop environment you probably need to have Firefox or Chromium, and they can't be run without virtual framebuffer, as they have some display server to run on. Setting up those packages can take liters of sweat and bunch of nerves.

    That's why we decided to pack Selenium and PhantomJS into portable Docker containers. Did you hear about Docker? If not - you probably should take some time to look at it. Docker allows you to create virtual environments on Linux hosts without real virtualization. Docker creates an isolated environment inside the host machine without recreating operating system.

    Docker containers are lightweight and easy to use. If you have Linux. Probably all your servers use Linux, so you might find docker images of Selenium and PhantomJS pretty useful!

    SeleniumEnv: Selenium, Xvfb, Firefox, and Chromium Inside Container

    OK, lets assume you are using Linux and Docker >=1.2. From now on running Selenium won't take from you anything more then running those commands:

    docker run -i -t -p 4444:4444 davert/selenium-env

    This will start virtual display server (Xvfb) and Selenium inside a container. Please note, you don't need anything of those installed on your host. Not even Java, required to run Selenium. Only Docker is needed and you can run Selenium in one line!

    By running this command you will see this output:

    Running Selenium Env: Selenium Server, and Xvfb with Firefox and Chromium
    Host IP: 172.17.42.1
    [dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
    [dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
    [dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
    [dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
    [dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
    [dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
    [dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!
    02:02:30.641 INFO - Launching a standalone server
    02:02:30.670 INFO - Java: Oracle Corporation 24.60-b09
    02:02:30.670 INFO - OS: Linux 3.16.0-24-generic amd64
    02:02:30.684 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
    02:02:30.735 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: LINUX
    02:02:30.790 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
    02:02:30.791 INFO - Version Jetty/5.1.x
    02:02:30.792 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
    02:02:30.792 INFO - Started HttpContext[/selenium-server,/selenium-server]
    02:02:30.793 INFO - Started HttpContext[/,/]
    02:02:30.806 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@1a966bcb
    02:02:30.807 INFO - Started HttpContext[/wd,/wd]
    02:02:30.809 INFO - Started SocketListener on 0.0.0.0:4444
    02:02:30.809 INFO - Started org.openqa.jetty.jetty.Server@2b6ea258

    By specifying -p 4444:4444 option in docker run we mapped container's port 4444, which is default for Selenium to the same port on our host machine. So, any process can connect to Selenium on its standard port, even Selenium is completely isolated from other processes.

    And here comes a tricky question. If Selenium is completely isolated, how can we test applications running on a local web server? We have a solution for that. If your application is served by nginx or any other web server, you can probably access it by hostname (other than localhost). What we can do is to pass hostname inside a container.

    docker run -i -t -p 4444:4444 -e APP_HOST=myhost davert/selenium-env

    This will bind myhost to your host machine IP. So, container, and Firefox, Chromium, can get access to your local web site from inside container.

    Similar idea is used for application executed on specific port. If you use PHP built-in server, you can start it at specific port, and pass it into container as well. Please note, that to make server accessible from container, it should be started on 0.0.0.0 IP:

    php -S 0.0.0.0:8000
    # in new terminal window
    docker run -i -t -p 4444:4444 -e APP_PORT=8000 davert/selenium-env 
    # in new terminal window
    codecept run acceptance

    PhantomJsEnv: PhantomJS Inside Container

    Instructions are the same for running PhantomJS. It can be started with

    docker run -i -t -p 4444:4444 davert/phantomjs-env

    Everything else is pretty similar. Also you can easily switch from Selenium to PhantomJS by starting or stopping those two containers. Isn't that useful?

    Images and Source Code

    SeleniumEnv and PhantomJsEnv are created from Travis CI Cookbooks and packed into containers. They are free to use and free to modify for custom needs.

    If you need to have Selenium Server from version other than 2.44.0 (which is latest for today), you should update Docker file and build an image from it.

    SeleniumEnv and PhantomJsEnv may simplify testing for all Linux users. They will definitely simplify setting up Continuous Integration servers. You can even recreate a complete testing environment inside a container. Take a look into RoboCI project for this. It allows you to build Travis CI-like service on your own host!

    Use them and enjoy :)

    P.S. Feedback, Pull Requests, and Issues, are welcome )


    0 0

    Jenkins

    A frequent question that appear from time to time is How to set up Codeception with Jenkins. Despite the growth of popularity of various Continuous Integration solutions Jenkins is still the most popular open-source solution on market. It is easy to setup and is easy to customize by applying various 3rd-party plugins. In this post we will take a look on how to setup testing process of PHP project with Jenkins CI and Codeception.

    Create new job in Jenkins

    Preparing Jenkins

    We will start from an empty Jenkins Server executed on localhost. Let's create a new Job for project tested with Codeception and name it PHPCodeceptionProject. Before we proceed to configuring it, lets install required plugins:

    • Git Plugin - for building tests for Git repo
    • Green Balls - to display success results in green.
    • xUnit Plugin, jUnit Plugin - to process and display Codeception XML reports
    • HTML Publisher Plugin - to process Codeception HTML reports
    • AnsiColor - to show colorized console output.

    Jenkins Plugins

    Basic Setup

    Once everything is installed lets get back to configuring our newly created PHPCodeceptionProject. At first we will set up Git to checkout code and build. Depending on your needs you can set up periodical build or trigger build once the change is pushed to GitHub (you will need GitHub plugin for that). You may also set up Jenkins as alternative to Travis CI to build your pull requests with GitHub pull request builder plugin.

    We will enable colors in console using Ansi Color

    Jenkins ANSI color

    Next and the most important part is to define build step. We won't create any sofisticated environment settings like setting up database and services. We assume that all we need to execute tests is to inctall composer dependencies and run codeception tests:

    composer install
    ./vendor/bin/codecept run

    Jenkins Codeception Build Step

    And that's the absolute minimum we need to execute tests in Jenkins. We can save project and execute the job. Jenkins will clone repository from Git, install composer dependencies and run Codeception tests. If tests fail you can review the console output to discover what went wrong.

    Jenkins Console Output

    XML Reports

    But we don't want to analyze console output for each failing build. Especially If Jenkins can collect and display the results inside its web UI. Codeception can export its results using JUnit XML format. To generate XML report on each build we will need to append --xml option to Codeception execution command. Codeception will print result.xml file containing information about test status with steps and stack traces for failing tests. Unfortunately Jenkins can't process such special like Codeception steps. So we will need to configure to print data only which strictly applies to xUnit xml format. This should be done in codeception.yml

    settings:
        strict_xml: true

    Now let's update our build step to generate xml:

    composer install
    ./vendor/bin/codecept run --xml

    and ask Jenkins to collect resulted XML. This can be done as part of Post-build actions. Let's add Publish xUnit test result report action and configure it to use with PHPUnit reports.

    Use PHPUnit xUnit builder for Jenkins

    Now we should specify path to PHPUnit style XML reports. In case of standard Codeception setup we should specify tests/_output/*.xml as a pattern for matching resulted XMLs. Now we save the project and rebuild it.

    Jenkins Result Trend

    Now for all builds we will see results trend graph that shows us percentage of passing and failing tests. We also will see a Latest Test Result link which will lead to to the page where all executed tests and their stats listed in a table.

    HTML Reports

    To get more details on steps executed you can ask Codeception to generate HTML report. Jenkins can display them as well.

    composer install
    ./vendor/bin/codecept run --html

    Now we need HTML Publisher plugin configured to display generated HTML files. It should be added as post-build action similar way we did it for XML reports.

    Jenkins Codeception HTML Setup

    Jenkins should locate report.html located at tests/_output/. Now Jenkins will display HTML reports for each build.

    Jenkins HTML Report Jenkins Codeception HTML Results

    That's pretty much everything you need to get a simple Jenkins setup with Codeception.


    0 0
  • 05/01/15--18:03: Codeception 2.1 Beta
  • We are glad to announce the first release of Codeception 2.1. It was a long time of developing the next major version of our testing framework. We delayed the release many times trying to bring as many cool features as we could. Today we can say that master branch is stabilized and we are prepared to share results of our work with you. Codeception 2.1 tries to do its best in test codebase organizing, improving ux, and customization.

    Take a look what is landed in Codeception 2.1.

    Full list of features is available in changelog. Documentation was updated accordingly.

    You can install Codeception 2.1 via composer by requiring 2.1.0-beta version, or by downloading phar.

    Give new Codeception a try and send us feedback. As always use GitHub issues for that. Thanks for being with us all that time! We hope you will love this release.

    Stable 2.1.0 is expected by the middle-end of May

    Upgrade Notes

    • If you are upgrading from Codeception 2.0 delete actor classes in tests/acceptance, tests/functional, etc, and rebuild actor classes. They will be recreated in tests/_support.
    • Try to execute new tests. If there are unmet dependencies, or module conflicts, you will be notified by exception with an advice how to resolve it.
    • It is recommended to recreate Helpers per suite with g:helper generator, like codecept g:helper Acceptance. New helpers will be store in tests/_support/Helper.
    • Same for PageObjects and StepObjects, they should be moved to _support.

    0 0
  • 06/18/15--18:03: Codeception 2.1 RC
  • Today we are announcing Codeception 2.1 RC release. We are almost ready to release a stable version, however we'd like to receive more feedback before the actual launch. We encourage you to install the latest 2.1 release and try it on your projects. Migrations should be painless if you follow the upgrade instructions. Here are the things you should keep in mind while upgrading:

    • REST, SOAP and Doctrine2 modules rely on another module which should be explicitly set via depends config param. For instance, you should pass Laravel5 module if you do API testing with REST module
    modules:
        enabled: [REST, Laravel5]
        config:
            REST:
                depends: Laravel5
    
    • Actor classes are moved to _support directory. Please delete all previously generated actor classes which reside in suites directories. New AcceptanceTester, FunctionalTester, UnitTester classes are expected to be extended with custom methods.
    • Modules that share similar interfaces like WebDriver, PhpBrowser, and framework modules won't run together. You should avoid enabling more than one of those modules in suite config to avoid confusion. If you enable Laravel5 and WebDriver and execute $I->amOnPage('/') you can't be sure how this command is exected - will it open a browser window using WebDriver protocol, or it will be handled by Laravel framework directly.
    • Cept files should avoid setting their metadata via $scenario methods. Instead of calling $scenario->skip(), $scenario->group('firefox'), etc, it is recommended to set scenario settings with annotations // @skip, // @group firefox. With this change you can now skip tests based on a condition: if (substr(PHP_OS, 0, 3) == 'Win') $scenario->skip().
    • Kohana, Symfony1, Doctrine1 modules were removed and they reside in separate repositories in Codeception organization on GitHub.

    That should be enough to have everything upgraded. We already prepared updated guides (they didn't change from 2.1.0-beta). Please try new Codeception and tell us your opinion in comments or on GitHub.

    Codeception 2.1 can be used with latest Guzzle 6 (as well as previous Guzzle releases). That's the most important change since the beta version. See the changelog for more info. We are preparing site updates and demo apps to reflect new changes. Stay tuned!

    Download 2.1.0-rc phar

    via composer.json:

    "codeception/codeception": "2.1.0-rc1"
    

    0 0
  • 06/29/15--18:03: Codeception 2.1 Is Here
  • We are finally ready to show you the Codeception 2.1. Since RC we stabilized its codebase, and we encourage you to start all your new projects with Codeception 2.1. As well as migrate old ones. Codeception 2.1 is aimed for consistency and provide you the better experience for testing your web applications. This new release makes tests even more simple to read, write, and maintain.

    If you didn't track for the changes in master we will list all the new features here:

    • We added Recorder extension, which is probably the most fancy feature you may try. Using it you can record test execution history by saving a screenshot of each step. This is handy for running tests on CI, debugging tests executed via PhantomJS or showing nice reports to your boss.

    recorder

    • Updated to Guzzle 6. Codeception can now work both with Guzzle v5 and Guzzle v6. PhpBrowser chooses right connector depending on Guzzle version installed.
    • PSR-4: all support classes moved to tests/_support by default. Actors, Helpers, PageObjects, StepObjects, GroupObjects to follow PSR-4 naming style. New AcceptanceTester, FunctionalTester, UnitTester classes are expected to be extended with custom methods. For instance, you can define some common behaviors there. For instance, it is a good idea to place login method into the actor class:
    <?php
    class AcceptanceTester extends \Codeception\Actor
    {
        use _generated\AcceptanceTesterActions;
    
        public function login()
        {
           $this->amOnPage('/login');
           $this->fillField('name', 'jon');
           $this->fillField('password', '123345');
           $this->click('Login');
        }
    }
    ?>
    • Dependency Injection: support classes can be injected into tests. Support classes can be injected into each other too.
    <?php
    class UserCest 
    {
      // inject page objects into Cests
      function updatePassword(\Page\User $page, AcceptanceTester $I)
      {
          $page->openProfile();
          $page->editProfile();
          $I->fillField($page->oldPasswordField, '123456');
          $I->fillField($page->newPasswordField, '654321');
          $I->fillField($page->passwordFieldRepeat, '654321');
          $I->click($page->saveBtn);
          $I->see('Password has been updated');
      }
    
      // inject step object into Cest
      function adminUpdatePassword(\Step\Admin $I)
      {
          $I->enterAdminArea();
          $I->changePassword('654321');
          $I->see('Password has been updated');
      }
    }
    ?>
    • Module config simplified: Modules can be configured in enabled section of suite config. Take a look of this sample declaration of Api suite, there is no config section inside modules.
    modules:
        enabled:
            - WebDriver:
                url: http://codeception.com
                browser: firefox
            - \Helper\Acceptance
    
    • Dependencies: module can explicitly define dependencies and expect their injection. REST, SOAP and Doctrine2 modules rely on another module which should be explicitly set via depends config param.
    class_name: ApiTester
    modules:
        enabled:
            - REST:
                url: https://api.github.com/
                depends: PhpBrowser           
            - \Helper\Api
    

    As you can see, you don't need to specify PhpBrowser in enabled section, you can set it only via depends.

    • Conflicts: module can define conflicts with each other. Modules that share similar interfaces like WebDriver, PhpBrowser, and framework modules won't run together. You should avoid enabling more than one of those modules in suite config to avoid confusion. If you enable Laravel5 and WebDriver and execute $I->amOnPage('/') you can't be sure how this command is exected - will it open a browser window using WebDriver protocol, or it will be handled by Laravel framework directly.

    • Current modules, environment, and test name can be received in scenario.

    <?php
    $scenario->current('name'); // returns current test name
    $scenario->current('modules'); // returns current modules
    $scenario->current('env'); // returns environment
    
    // can be used like this
    if ($scenario->current('env') == 'firefox') {
      // do something for firefox only
    }
    // naming screenshot
    $I->makeScreenshot($scenario->current('name').'_screenshot.png');
    ?>
    • Environment Matrix: You can run tests combining several environments by separating their names by comma:
    codecept run --env dev,phantom --env dev,chrome --env dev,firefox
    

    Environments can now be stored in separate configuration files in tests/_envs dir created with generate:environment command:

    tests/_envs
    |── chrome.yml
    |── phantom.yml
    |── firefox.yml
    |── dev.yml
    
    • Cept files should avoid setting their metadata via $scenario methods. Instead of calling $scenario->skip(), $scenario->group('firefox'), etc, it is recommended to set scenario settings with annotations // @skip, // @group firefox. However, you can use $scenario->skip whenever you need to do it on some condition, like
    <?php
    if (substr(PHP_OS, 0, 3) == 'Win') $scenario->skip()
    ?>
    • Improved HTML reports html report

    • Modules can be partially loaded. If you need only some actions to be included into tester objects. For instance, you want to have REST API tests with Laravel5 module. In this case you probably don't want methods from Laravel module like amOnPage or see to be included into the ApiTester as they interact with HTML pages, which we are not supposed to use. But you still need Laravel ORM methods like seeRecord to verify that changes were saved to database. In this case you can enable only ORM methods of Laravel5 module.

    modules:
        enabled: 
            - Laravel5:
                part: ORM
            - REST:
                part: Json
                depends: Laravel5
    

    As for REST module you can load methods only for API format you are using. You can choose either XML or Json, so only methods for Json or XML will be loaded.

    • Whenever you have steps grouped inside actor class, pageobject, or stepobject, you can the step executed with its substeps in console or HTML report. html report
    • RunFailed extension is enabled by default for all new projects. It means that you can rerun your failed tests by executing codecept run -g failed when this extension is enabled.

    And lots of other notable improvements which you can see in changelog.

    Starting from Codeception 2.1 we recommend using Cest as a default format for all scenario-driven acceptance and functional, or api tests. Guides were rewritten to reflect new improved approaches to testing that you should practice by using Codeception.

    Upgrading

    • It is pretty easy to upgrade from 2.0. The only thing you should start with is to rebuild your actor classes by running codecept build command. The old actor classes (*Tester or *Guy) in suite directories should be deleted manually.
    • REST, SOAP, Doctrine2 module will need to be configured to use a dependent module as it was shown above. You will get a detailed exception with configuration example once you execute tests with those modules.
    • Helpers, PageObjects, StepObjects are expected to follow PSR-4 standard and use namespaces. It is recommended to rename classes to replace suffixes with namespaces, like UserPage => Page\User. However, those changes are not required, so you can keep your support objects without changes.

    If you are using 2.0 and you won't plan to upgrade in nearest future, you can still use releases from 2.0 branch. Minor fixes and improvements will still be accepted there. Also the site will contain documentation for 2.0 and 2.1 versions.

    Try Codeception 2.1 today by installing it via Composer:

    composer require "codeception/codeception:*" --dev
    

    or by downloading it as Phar archive

    And provide a feedback!


    0 0

    Codeception Testing Framework from its roots was a plugin of symfony 1 framework. Today Codeception is powered by Symfony components and can be used to run functional tests for practically any popular PHP framework.

    Why would you someone ever choose Codeception if Symfony already has mature testing infrastructure. Well, here are our reasons. Codeception tests are:

    • fast, as each functional/integration test is wrapped into transaction using Doctrine ORM
    • scenario-driven, it means that tests are linear, described in easy to get PHP DSL
    • can be used for testing complex interactions inside functional tests.
    • easy to write, as Codeception already provides bundled actions and assertions for most popular use cases.
    • combine all testing levels (acceptance, functional, unit) in one tool.

    Today we will write how Codeception can be installed into a Symfony project and fit with its structure: we will put functional and unit tests to corresponding bundles, write acceptance tests for complete application, and use one runner to execute them all at once.

    symfony-demo

    We will use official symfony-demo application in this example. Once you get it installed you should add Codeception testing framework to dev dependencies list in composer.json:

    composer require --dev "codeception/codeception:~2.1"

    As you may know Codeception has bootstrap command which create common file structure for acceptance, functional, and unit tests. However, as we decided we will follow Symfony way and skip creating global functional and unit tests. So we start with bootstrapping empty project without predefined suites.

    php bin/codecept bootstrap --empty

    As you may see, we got new tests directory and codeception.yml file created. Let's have acceptance tests there:

    php bin/codecept g:suite acceptance

    Acceptance tests are expected to test the site from an end-user's perspective. No matter how many unit tests you have in your projects you can't get without acceptance testing. What if you see a blank page even all unit tests passed. How could this happen? Maybe you rendered wrong template, maybe some scripts or styles were not loaded. Those things can't be handled with internal: unit or functional testing, however with acceptance tests you may be confident that UI is available for customers.

    That's why we recommend to have tests with real browser interaction. You can learn more about acceptance testing from our guides.

    But what about unit and functional tests? As we decided we will put them into bundles. Symfony-demo has only AppBundle included, so we will create new Codeception setup in src/AppBundle. Take a note that we want those tests to be placed in their own namespace:

    php bin/codecept bootstrap --empty -c src/AppBundle --namespace AppBundle

    We will also create unit and functional suites there:

    php bin/codecept g:suite functional -c src/AppBundle
    php bin/codecept g:suite unit -c src/AppBundle

    As you noticed we specify path to different Codeception setup with -c or --config option.

    Unit tests of Codeception are not quite different from regular PHPUnit tests. You can even copy your old PHPUnit tests to src/AppBundle/tests/unit and have Codeception run them. It is much more interesting to use Codeception to have functional tests replacing ones extending Symfony\Bundle\FrameworkBundle\Test\WebTestCase class.

    Let's have a test that will check that there is specific number of posts on a page. Symfony-demo app has the similar test included:

    <?php
    namespace AppBundle\Tests\Controller;
    use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
    use AppBundle\Entity\Post;
    
    class BlogControllerTest extends WebTestCase
    {
      public function testIndex()
      {
          $client = static::createClient();
          $crawler = $client->request('GET', '/en/blog/');
          $this->assertCount(
              Post::NUM_ITEMS,
              $crawler->filter('article.post'),
              'The homepage displays the right number of posts.'
          );
      }
    }

    We will rewrite it in Codeception manner. At first we are generating new empty test case for it. We use scenario-driven test format called Cest:

    php bin/codecept g:cest functional BlogCest -c src/AppBundle

    And here goes the test:

    <?php
    namespace AppBundle;
    use AppBundle\Entity\Post;
    
    class BlogCest 
    {
        public function postsOnIndexPage(FunctionalTester $I)
        {
            $I->amOnPage('/en/blog/');
            $I->seeNumberOfElements('article.post', Post::NUM_ITEMS);
        }
    }
    ?>

    As you see, Codeception test is shorter. It is simple, clean, and can be easily extended for more complex interactions. However, we are not ready to run it yet. We need to prepare Codeception to run functional tests inside Symfony context. For this we need to edit src/AppBundle/tests/functional.yml configuration file to enable modules: Symfony2 and Doctrine2 to use:

    class_name: FunctionalTester
    modules:
        enabled:
            - Symfony2:
                app_path: '../../app'
                var_path: '../../app'
            - Doctrine2:
                depends: Symfony2
            - \AcmeBundle\Helper\Functional

    The most important thing here is to provide valid app and var paths for Symfony. Also we are specifying that Doctrine's EntityManager should be taken from Symfony DIC. Let's run functional tests of AppBundle:

    php bin/codecept run functional -c src/AppBundle

    In this case you will see following output:

    Codeception PHP Testing Framework v2.1.2
    Powered by PHPUnit 4.8.2 by Sebastian Bergmann and contributors.
    
    AcmeBundle.functional Tests (1) ------------------------
    Posts on index page (BlogCest::postsOnIndexPage)     Ok
    --------------------------------------------------------
    
    
    Time: 274 ms, Memory: 36.00Mb
    
    OK (1 test, 1 assertion)

    But you can do much more with functional testing. You can insert/assert data with Doctrine by using prepared methods like haveInRepository, seeInRepository of Doctrine2 module. You can perform complex web interactions like filling forms, clicking links, following redirects and much more with methods of Symfony2 module. Those modules are combined together and their methods are available in FunctionalTester class you are supposed to use for writing functional tests. If you are interested to see more complex Codeception tests, we've got them for you.

    Btw, you can use Symfony2 and Doctrine2 module for writing your unit and integration tests as well.


    But how can we run acceptance tests of a project with tests from AppBundle together? We need to edit codeception.yml configuration file in project root to make it. Let's add those lines there:

    include:
        - src/*Bundle

    That's it. For now Codeception will include all installations stored in Bundles on run. If you execute:

    php bin/codecept run

    you will probably see that BlogCest of AppBundle was executed as it was expected to.


    tests

    The most complex thing in starting using Codeception with Symfony is have it configured. Despite Codeception is auto-connecting to Symfony framework and Doctrine you still have to do some changes to follow Symfony structure. Please take a detailed look into our forked version of symfony-demo project which we configured in the manner we described in this post. Please use similar configuration for all your Symfony projects.

    Start using Codeception and discover how complex things can be tested in really simple manner. And once again, even functional and integration tests are really fast, as we start transaction before each test and rollback it afterwards. Write them as many as you need to, do not rely on unit tests only!

    P.S. Symfony2 and Doctrine2 module is seeking for an active maintainer. If you work with Symfony and Codeception please contact us to join Codeception team!


    0 0

    Codeception 2.2 is going to land in nearest weeks. We planned to release it much earlier, but as we are not bound to specific date, we decided to work more on improvements and include the most tasty ones in next release. In this and following blogposts we will review the most significant of upcoming features. So take a look what we prepared for you.

    We don’t plan any major breaking changes in this release. However, we will remove deprecations, as you may know we display deprecation warnings in output, so this break should be at least predictable.

    PHPUnit5 support

    Yes, Codeception is finally will unlock it’s dependency on PHPUnit 4.8 so you could receive the latest stable versions of PHPUnit. As you may know PHPUnit 5.x has much better support of PHP7, so anyone who develop using the latest version of PHP should use it. But wait, it’s not the feature of Codeception 2.2 itself! Codeception 2.1.7 will be released with PHPUnit 5.x support as well.

    Test Dependencies

    Finally @depends annotation works as you might expected it to. The hardest thing about dependencies is that PHPUnit does nothing to organize dependent tests in a right order:

    PHPUnit supports the declaration of explicit dependencies between test methods. Such dependencies do not define the order in which the test methods are to be executed but they allow the returning of an instance of the test fixture by a producer and passing it to the dependent consumers.

    So their usage in Codeception was useless, especially if you were executing tests in random order with shuffle: true configuration. Some test frameworks, like minitest from Ruby run tests in shuffle by default. On the contrary PHPUnit restricts that, test with @depends annotation must be be executed after the test it relies on. It actually requires you to declare dependent tests in the same file and use the exact order of test execution.

    Codeception takes the exact @depends annotation, reorders tests so dependent tests always will executed after the tests they rely on. This will work for Cest and Test formats both. And you can actually declare dependencies for tests in other files. Be sure, they will be taken, executed in the right order, and if a test fails its dependencies will be skipped. This is likely improve acceptance testing. If login test doesn’t pass, you shouldn’t even try to launch all other tests with authorization required.

    Here is the annotation format to use to declare dependencies. It should be {fullClassName}:{testName} or {testName} for the test in the same class.

    @depends login
    @depends UserCest:login
    @depends App\Service\UserTest:create
    

    Unlike PHPUnit, you can’t pass data between dependent tests. Alternatively you can use Helpers or Codeception\Util\Fixtures class to store shared data.

    Params

    This lines are written just in the same time as corresponding pull request is merged:

    Codeception 2.2 will allow you to use parameter in configuration files populated from environment variables, yaml configs or .env files. You can use parametrized vars in suite configs, by wrapping them in % placeholder, and adding one or several parameter providers using the params key in global config:

    params: [env] # get prameters from environment vars
    

    You can use Symfony-style .yml files, Laravel-style .env, or .ini files to receive params.

    This feature would be useful if you want to share database configuration with application and tests. This will especially be useful if you use some sort of credentials passed via environment vars. For instance, if you use cloud testing services, you can set login and password with params and get the real values from environment:

        modules:
           enabled:
              - WebDriver:
                 url: http://mysite.com
                 host: '%SAUCE_USER%:%SAUCE_KEY%@ondemand.saucelabs.com'
                 port: 80
                 browser: chrome
                 capabilities:
                     platform: 'Windows 10'
    

    Conflicts

    This is possibly a breaking feature which was announced in Codeception 2.1 but never worked the way it supposed to be. So let’s talk about it once again. A friendly reminder, if you use config like this:

    modules:
        enabled: [PhpBrowser, Laravel5, WebDriver]
    

    you will have problems in Codeception 2.2, if you didn’t have them earlier. Basically the issue here, that those modules provide pretty the same API but different backends for test executions. So it’s hard to tell from config, will this tests be executed as Laravel functional tests, as acceptance tests in real browser, or HTML-scrapper called PhpBrowser. We have lots of issues, when developers misuse those modules, trying to include everything in one config, and we can’t help them besides the good advice to choose one module of one kind.

    To avoid this confusion we introduced Conflicts API. Module can declare conflict for modules of the same API, and ConfigurationException will be thrown if you try to use them in group.

    But what if you really need to use Laravel5 module along with, let’s say, WebDriver? Don’t worry, partially loaded modules won’t get into conflict. The most common use case is using Laravel ORM methods to create-delete data for acceptance test. In this case you don’t need to load all actions of Laravel5 module but only those related to ORM:

    modules:
        enabled:
            - WebDriver:
                url: http://localhost
                browser: firefox
            - Laravel5:
                part: orm
    

    this way you receive haveRecord, seeRecord, grabRecord, have, haveMultiple methods running though Eloquent but amOnPage, see, etc will be executed through WebDriver.

    If you were mixing WebDriver with PhpBrowser in order to use REST API inside acceptance tests, you can still have it. Don’t enable PhpBrowser but set it as dependent module for REST:

    modules:
        enabled:
            - WebDriver:
                url: http://localhost/
                browser: firefox
            - REST:
                url: http://localhost/api
                depends: PhpBrowser
    

    This way you can use API to create test data for application:

    <?php
    $I = new AcceptanceTester($scenario);
    // let's create a post
    $I->sendPOST('/posts', ['title' => 'Hello, Codeception 2.2', 'body' => 'Almost here']);
    
    // check it is actually there
    $I->amOnPage('/posts');
    $I->see('Hello, Codeception', 'h2');
    

    …and much more to come

    Sure, that was not everything what is coming in 2.2! Stay tuned for new cool announcements, which will be posted soon. We will describe new modules, new test formats, and new features! Stay tuned.


    0 0

    Today we continue to show you new features of Codeception 2.2, planned to release this March. In previous post we’ve got you acquainted with Test Dependencies, Params, and Conflicts. This post will continue this overview, and we will start with some nice modules, which might be useful to you.

    As you know, in Codeception we try to help developers and testers share their experience by providing set of shared pieces, and not to reinvent the wheel. Modules play exactly that role: you just include modules to fit your project, and write your tests, concentrating on its business logic, and not on implementation. 90% test steps in functional and acceptance tests are covered with out modules. So what do we have prepared for 2.2?

    DataFactory

    This module should solve problem of generating and populating data for tests. Right now we have Db module, MongoDb with really limited functionality, methods like haveRecord, seeRecord in various frameworks, and Sequence module for generating unique keys for data. However, they didn’t provide any way to generate data needed exactly for a particular test.

    DataFactory module should solve this with fixture generators, called factories. The original idea take from popular Ruby gem FactoryGirl and implemented in php in FactoryMuffin library, which we use. Laravel 5 users are already aware of using factories. Now you can use them with all PHP frameworks with ActiveRecord pattern and in Doctrine.

    This is how it works. You define rules to generate models:

    <?php
    use League\FactoryMuffin\Faker\Facade as Faker;
    
    $fm->define(User::class)->setDefinitions([
     'name'   => Faker::name(),
        // generate email
     'email'  => Faker::email(),
     'body'   => Faker::text(),
    
     // generate a profile and return its Id
     'profile_id' => 'factory|Profile'
     ]);
    

    and DataFactory creates them using underlying ORM, inserts them to database once you call

    <?php
    $I->have(User::class);
    

    and deletes them after the test. As we said that will work for Doctrine as well, if you are familiar with Nelmio’s Alice, you might find the same idea but with easier syntax to use.

    AngularJS

    AngularJS is probably the most popular framework for building single page web applications. It provides its own tool for acceptance testing - Protractor but what if you already use Codeception and you are not so passionate to switch to NodeJS? Well if you actually do, please check out our side-project CodeceptJS which brings Codeception concepts to JavaScript world. However, in case of Angular testing there is less reasons to switch, as we brought you Protractor experience in new AngularJS module.

    It simply wraps WebDriver module, and adds an asynchronous script between steps to ensure that all client-side operations finished before proceed. This way no new actions are taken before Angular finishes rendering. Also you’ve got new strict locator to use:

    <?php
    $I->see('davert', ['model' => 'user']);
    

    to match elements by their ng-model.

    But enough with modules, we have something more than that!

    Examples

    In PHPUnit you could have one test to be executed several times with different data, using the data provider mechanism. You can do the same in Codeception inside unit tests. But what about functional and acceptance testing? What if you need to run same scenario but passing different values into it? To be honest, the data provider didn’t looked like an elegant way to define a test data. Data was stored in additional method, often few pages below the original test, so it was hard to see the picture in a whole. We introduce a concept of Example, similar to what you might have seen in BDD frameworks. Using the @example annotation you can define data in test annotation and receive from Codeception\Example instance, injected into test:

    <?php
      /**
       * @example(path=".", file="scenario.suite.yml")
       * @example(path=".", file="dummy.suite.yml")
       * @example(path=".", file="unit.suite.yml")
       */
      public function filesExistsAnnotation(FileTester $I, \Codeception\Example $example)
      {
          $I->amInPath($example['path']);
          $I->seeFileFound($example['file']);
      }
    ?>
    

    For REST API testing this might look like:

    <?php
     /**
      * @example ['/api/', 200]
      * @example ['/api/protected', 401]
      * @example ['/api/not-found-url', 404]
      * @example ['/api/faulty', 500]
      */
      public function checkEndpoints(ApiTester $I, \Codeception\Example $example)
      {
        $I->sendGET($example[0]);
        $I->seeResponseCodeIs($example[1]);
      }
    ?>
    

    Data in @example annotation can be defined using JSON objects, JSON-arrays, or Symfony-style annotation. And yes, examples work only in Cests.

    Custom Commands

    Long requested feature that finally was implemented by Tobias Matthaiou allows you ro register custom commands to Codeception runner.

    If you ever created Symfony Console commands you will be familiar in creating custom commands for Codeception. You migth probably use to have your own template generators, perform data migrations, etc. You can register one as simple as you do it for extension:

    extensions:
        commands: [MyApp\MyCustomCommand]
    

    Getting current browser and capabilities in tests

    The last one, simple yet useful thing that might improve your acceptance testing experience. If you want to have different behavior of tests for different browsers, you can get current browser name from a $scenario->current value:

    <?php
    if ($scenario->current('browser') == 'phantomjs') {
      // emulate popups for PhantomJS
      $I->executeScript('window.alert = function(){return true;}'); 
    }
    ?>
    

    in a similar manner you have access to all browser capabilities:

    <?php
    $capabilities = $scenario->current('capabilities');
    if (isset($capabilities['platform'])) {
      if ($capabilities['platform'] == 'Windows') {
        // windows specific hooks
      }
    }
    ?>
    

    That’s all for today, but not for Codeception 2.2 The most important and most impressive feature is waiting for you in next post. Subscribe to our Twitter to not miss it. Stay tuned!


    0 0
  • 03/25/16--18:03: Codeception 2.2 Beta
  • Happy Easter, everyone! Ok, not actually everyone, but Happy Easter to those who celebrate it this weekend. If you do (or even not), you probably will have free time to play with something new. And yes, by something we mean a beta version of Codeception 2.2!

    We already announced lots of changes in Codeception:

    but we forgot the most important one. It is Gherkin Support, which allows you to combine business requirements with functional testing. Yes, *.feature files are now part of a family with Cest, Cept and Tests. Codeception is a tool for running all kind of tests and in this release we significantly improved test internal architecture and test formats.

    But back to Gherkin.

    Feature: ToDo items
      In order to get my tasks done
      As a motivated person
      I need to be able to organize my todo list
    
      Scenario: show current todos
        Given there are todo items:
          | Task             | State   |
          | make Gherkin PR  | opened  |
          | update docs      | opened  |
          | create examples  | closed  |
        When I open my todos
        Then I should see 2 todo items
    

    Complete Guide on BDD with Codeception is not ready yest, but you can start with generating first feature file:

    codecept g:feature <suiteName> <featureName>
    

    We recommend to have a special features folder in acceptance or functional suite, so it could be symlinked to the features dir in root of your project. This way non-technical users can esaily access feature files, without need to examine actual tests.

    Next thing to do is to describe feature with scenarios. When you are done, prepare scenario steps for implementation be running

    codecept gherkin:snippets <suiteName>
    

    You will get a list of methods which should be included into your actor class (let’s say AcceptanceTester). Then you should have it implemented. In theory, you can use any method of any class annotated with @Given, @When, @Then to be the step definition. So don’t worry you will end up with everything to put in one context, you will have option to use multiple contexts depending on role, tags, etc. More about it in BDD guide coming in next weeks.

    For those of you, who set your dependencies as “codeception/codecepton:*” and (with no settings of "minimum-stability": "stable" you will probably have some wonderful time once you get back from holidays. This release is minor, so it doesn’t break everything, but we has breaking changes. We notified of breaking changes in earlier versions by “deprecation warnings”, and we actually removed lots of deprecated stuff. The most important breaking change is proper implementation of Conflicts API. Please make sure you are not using modules of the same kinds in your configs.


    Codeception 2.2-beta is available for installation only via Composer:

    composer require --dev "codeception/codeception:2.2.0-beta"
    

    Next steps to do:

    Stable release will come in next week(s). The release date depends on reported issues and on progress of updating documentation.

    Have nice weekends and testing time! We, Codeception team, hope you will love our product.

    P.S. Reported issues should include a version. Pull Requests should be sent to master branch.


    0 0
  • 06/02/16--18:03: Codeception 2.2 is here
  • Today we are glad to announce that Codeception 2.2 is finally released. The first beta came in March and after there were 2 RC versions. We assume that everyone could already try the new branch and check its new sweet features like Gherkin support, Examples and others. We hope that you will like new version as it brings true multi format support to your testing experience! What other tool can execute Gherkin features and PHPUnit tests? We already support 4 testing formats for code and scenario testing and it is quite easy to introduce more.

    Despite its growth and feature richness Codeception is still the most simple tool for test automation. We updated documentation to keep up to date with latest changes and improved the learning experiences. Also we prepared some Case Study pages which will help you to setup Codeception for your application. Now it is easier to start with Laravel and Symfony apps.

    If you have questions about Codeception you can ask them on our community forum which is PhpTestClub. We plan to build a sustainable community, so If you use Codeception for a long please share your experience there, help others to get into the world of PHP testing.

    How To Upgrade

    We encourage you to UPGRADE to 2.2. Please take a look into changes marked as breaking.

    • The most important change is introduction of module conflicts. If modules share the same interface they probably should not be used together except the cases when the modules are loaded partially. This way you can’t use Laravel5 module with WebDriver but you can use Laravel5 with part: ORM so only ORM actions to be loaded. This change is important in order to avoid confusion in your functional or acceptance tests. If you use 2 similar modules you can’t be sure which one is executed when you call $I->amOnPage. Learn more from previous blogpost.
    • Please also note that Dbh and Laravel4 module were removed from core package and moved into separate packages.
    • Codeception\TestCase was replaced with Codeception\TestInterface to it is a good idea to replace its usages in your Helper files.
    • Cept and Cest formats are no longer extend PHPUnit_Framework_TestCase, so they don’t have expectException, getMock, etc.

    Please read changelog to learn more.

    But wait, you may ask, why there are breaking changes in minor release? Does Codeception follow semver? Not right now but future releases will do. Next major version is going to be Codeception 3.0 which sure will introduce breaking changes for good reason. Also we plan to decouple most of modules from core package to make faster release cycles for them. Codeception 2.2 is also marked as minor that its still support PHP 5.4. However, we encourage you to upgrade for future versions.

    New Features Described


    If you are using 2.1 and you won’t plan to upgrade in nearest future, you can still use releases from 2.2 branch. Minor fixes and improvements will still be accepted there. Also the site will contain documentation for 2.2 and 2.2 versions.

    Try Codeception 2.2 today by installing it via Composer:

    composer require "codeception/codeception: ~2.2" --dev
    

    0 0

    What makes a meaningful test? This question should always be asked. No matter we write your tests first or tests after, they may stay in a project for years and it is pretty important them to test the things that really matters. Badly written tests can slow us down by constantly failing on each implementation change, and such tests can be written no matter you follow TDD or not. The idea of a test is to ensure that software works and not to freeze it at specific point.

    Such situation happens when a test is bound to implementation details. We are choosing to rely on parts which we may consider to be stable but they are not. Whenever we come to unit testing and we are writing a mock we should understand that we change the reality, and how that artificial reality will survive the test of time. Consider using such test written in XIX century as an example:

    Given I am drunk 
    And I am in a pub
    And I want to get home
    When I order a cab
    Then I expect to see 2 horses and carriage
    And they bring me home
    

    By ordering a cab nowadays you probably won’t expect to see 2 horses but you probably will get home even faster. In XXI century we still have pubs and we still need cabs to get home, that’s something stable in our world. The point is: you should not be worried of HOW you will be brought home: by horses, by car, by flying dragons or by teleport. That’s implementation details.

    But how to understand what is stable and what is not? We need to use interfaces. Not that one which is written as interface keyword in PHP but a general term: User Interface, API. And that’s what makes unit testing and browser testing similar: we always need to rely on public interfaces for a test. If break this rule and we start to test private methods or raw values in database we are producing a synthetic tests which will lead us the very tricky path.

    An opposite to follow public APIs is to depend on implementation.

    But back to software development. Here is an example of magento generators written in PhpSpec. Not saying it is bad or not, but we use it to illustrate the point.

    <?php
    class ModuleSpec extends ObjectBehavior
    {
      function let(\Symfony\Component\Filesystem\Filesystem $filesystem)
      {
          $this->beConstructedWith($filesystem);
      }
    
      function it_should_create_a_folder_given_a_valid_path($filesystem)
      {
          $path = 'app/code/test/test';
          $filesystem->exists($path)->willReturn(false);
          $filesystem->mkdir($path, 0700)->willReturn(true);
          $this->make($path)->shouldReturn(true);
      }
    }
    

    The test is very descriptive: there is a class that depends on Filesystem and can be used to create a given folder. However, one line is questionable:

    $filesystem->mkdir($path, 0700)->willReturn(true);
    

    This is the mock that expects mkdir to be called with a specific parameters and to return an expected result. But is that stable to changes? What if at some point mkdir gets deprecated in favor of mkdirPlus, what if a method signature changes?

    Well, in this current case we can be sure that this method won’t be changed. The reason is simple: it’s Symfony, it’s LTS, and its API is stable. But can you say that about internal classes of your application? That they are 100% documented and they won’t change their behavior in future.

    When we change the implementation of Module->make() it still fits expected specification (‘it should create a folder given a valid path’), but the test fails. This happens because the test pretend to know too much. In similar manner a strict master doesn’t just ask apprentice to do the job. He thinks he knows better how to do it and provides him with a detailed instructions. He doesn’t care of the outcome but apprentice should understand the basics: disobedience will be prosecuted.

    But what if the actual result is not important to us? What if we want to ensure that tested method took the right path. That what happens in a test above: the implementation is verified. Indeed, If you are master (senior developer) and you have an apprentice (junior), this works pretty well: you ask them to implement the method just the way you see it.

    This makes a difference between testing behavior or testing result. Probably in most cases you want to test the real outcome, a result, but in some important areas you may need to test the behavior as well.

    To test a result we should rely on public interfaces only. We can validate class only by calling public method of this class and its collaborators In this case we need to call $filesystem->exists('app/code/test/test'), and make a class with in-memory filesystem. We can even make this class without using any mocking framework, just with anonymous classes.

    By making a test bound to implementation we are copying parts of that implementation into a test. This breaks the DRY principle. At some point test and code can be unsynchronized and this leads to fragility. If you change class, the test using a mock of that class may still pass, but actual code using it will fail, and this false positiveness is dangerous.

    Overusing mocks also makes tests longer. Lines of test code you write doesn’t always convert to software quality. Actually, from the software developer experience the true is quite the contrary: the more code you write the more bugs you may introduce. And your tests are just the same as production code: they will evolve with your software and they needs to be maintained.

    So the actionable advice here is to think what is important to you: a behavior or a result (no matter how it is achieved). Try to discover what is stable and what is not, rely on stable parts, and use public interfaces. Thanks for reading.

    Written by Michael Bodnarchuk @davert.

    Thanks to Nick Palamarchuk for the review and ideas.


    0 0

    When you start implement testing in your development process you should always ask: which tests are important for this project. And there is no general answer to this question. “Test everything, everywhere” is not a good solution as well, by writing more and more lines of code you won’t stable software. Tests should be driven by specification, and specification comes from your business. Depending on that you should choose which areas are more important to cover with tests and which are less.

    Part 1: Expectation vs Implementation

    It is always thought that you should have several acceptance tests, a dozen of functional, and a lot of unit or integration tests. However, If you develop a media portal it is more important to you, and for your business to have a good-looking UI. Same story if you develop new To-Do application focused on perfect UX. It is not much business logic for a To-Do app, but you need to ensure that everything user sees is slick. Even you have some internal bugs it is more important to you to assure that all buttons are visible, they can be easily clicked and so on.

    Like in this app:

    That’s right for this type of applications you should revert your testing pyramid. You should have lots of acceptance tests and few unit/integration.

    If UI/UX is a part of your business, concentrate on acceptance (browser) tests. But if you run banking application, ecommerce solution, ERP or CRM, UI is not that important. Those applications include complex domain logic and internal quality is much more important than visual part.

    What if user by a bug creates 3 extra todos in To-Do app? Well, it doesn’t look nice but not a big deal. But what if user by a bug withdraws from ATM 3 times more money they own? That’s a real problem. In case, you deal with money or real things it is critically important to test all the business processes. Test all possible scenarios, all edge cases, make the domain to be 100% covered with tests.

    That brings us to key idea of this post. There are two kinds of IT products:

    • software that automates business
    • software that is a business by itself

    And depending on the kind of software you work on you have different priorities. This also answers a question: “do I need BDD for my project”. In case, you are automating traditional business, you need to translate all business processes into a code, and you need ubiquitous languages for that. In case, you are building a new business, a startup, you probably don’t need that layer of complexity as presumably founders already speak the IT language.

    So learn what is important in your business. What brings you money. What are the risks. Cover that part at first and think from business side and not from technical. This is how a business would understand the real value of automated tests.

    Written by Michael Bodnarchuk

    We provide consulting services on Codeception and automated testing in general.


    0 0
  • 05/08/17--18:03: New Addons Page
  • As you know Codeception has a really nice ecosystem with lots of modules and extensions built by the community. Thanks for everyone who invest their time into building better testing tools. Sometimes we accept 3rd-party modules into Codeception core, sometimes we ask contributors to publish them as standalone packages.

    Today we’d love to announce a minor but very nice change on our website. We updated the Addons page so community-built modules and extensions would look nice there. The new Addons page looks as slick as a marketplace but everything there is for free!

    If you have a helper or an extension that might be useful to others, don’t hesitate to share it! New “cards” format for extensions and modules will make them more visible to others. You can also attach your pictures and set colors for cards to be even more visible (If you think your extensions worth it).

    The philosophy of Codeception is: don’t reinvent the wheel, share testing solutions. Spend more time writing tests and not test support code. Get more powerful tools and make testing fun!

    There are some tools you definitely should not miss:

    and many many others! If you don’t have a module or extension but you still have some experience to share, write a few lines to the Cookbook.

    Thamks for reading. Stay tuned for next cool updates!


    0 0
  • 05/21/17--18:03: Codeception 2.3
  • Today the Codeception 2.3 sees the world. This is a stable release with almost no breaking changes but with new features you will probably like.

    At first, we need to say “thank you” to Luis Montealegre for bringing PHPUnit 6.0 support to Codeception. This is done by using class aliases so this doesn’t break old PHPUnit versions, so as usual, Codeception can be used with PHPUnit 4.8 and higher. However, If you run PHP7+ and you experience PHPUnit issues in this release we recommend you to set phpunit/phpunit package to ^5.7.0 in your composer.json.

    Supporting 3 major versions of PHPUnit may have its issues, so in next release, which is going to be 3.0 we will probably drop support for old PHPUnits and PHP < 7.

    Ok, let’s talk about other changes in this release!

    Installation Templates

    Codeception is a wide-range testing framework and sometimes it is hard to get it configured for a specific case. Nevertheless, you need only acceptance testing Codeception will still install all 3 test suites for you. In 2.3 we prepared installation templates: customized setup wizards that can configure tests for your needs.

    To setup Codeception for acceptance testing with browser only you can run this command

    codecept init acceptance
    

    After answering a few questions, you will be able to execute first browser test

    The setup wizard will prepare only acceptance test suite with tests located in tests directory. Suite configuration will be stored in codeception.yml.

    For QA engineers codecept init acceptance is a new convenient way to start end-to-end testing in PHP.

    We also included API and Unit templates for faster setup of REST API or unit tests:

    codecept init api
    codecept init unit
    

    But the best thing about installation templates that they can be created by anyone! They can be placed inside a separate package and loaded by init command. A template class should be placed into Codeception\Template namespace and then it can be autoloaded. Installation templates are pretty simple, learn how to build your own by taking a look at Acceptance template as an example.

    Configuration Improvements

    Suites Inside Main Config

    Suites can be defined inside main config:

    actor_suffix: Tester
    paths:
        tests: .
        log: _output
        data: _data
        support: _support
    suites:
        unit:
            path: .
            actor: UnitTester
            modules:
                enabled:
                    - Asserts
    

    A good option if you have a single suite.

    Suite Paths

    The suite can have its custom path (specified by path). From config above expects unit tests to be placed into the root directory, where codeception.yml is located (path: tests: . and path: `)

    selection_201

    Suites With No Actor

    Suite can be defined without an actor, which is useful for unit testing

    paths:
        tests: tests
        log: tests/_output
        data: tests/_data
        support: tests/_support
        envs: tests/_envs
    suites:
        unit:
            path: .
            modules:
                enabled:
                    - Asserts
    

    In this case, UnitTester won’t be created, as well as _generated/UnitActions. However, such suites won’t be able to have Cest and Cept files generated.

    Naming Changes

    class_name suite in suite config replaced with actor:

    class_name: UnitTester => actor: UnitTester
    

    actor from global config is replaced with actor_suffix config option (which makes more sense).

    All these changes are backward compatible, so old values in config will work.

    Extensions

    Dynamical loading of extensions was already with --override option but was not very usable. Now extensions can be loaded with --ext option:

    codecept run --ext Recorder
    

    or by providing a full class name

    codecept run --ext "Codeception\Extension\Recorder"
    

    This can be used to enable a custom reporter. For this reason, the new DotReporter has been added:

    codecept run --ext DotReporter
    

    selection_205

    Db Populator

    From the early days of Codeception we had the Db module which was trying to do its best to populate database and clean up it between tests. However, parsing all possible SQL dialects and running them through PHP was not very effective. What if you could use native Database tools to import data instead of doing it from PHP? Why not!

    In Codeception 2.3 we recommend to specify a command to load a database in populator option of Db module:

     modules:
        enabled:
           - Db:
              dsn: 'mysql:host=localhost;dbname=testdb'
              user: 'root'
              password: ''
              cleanup: true # run populator before each test
              populate: true # run populator before all test
              populator: 'mysql -u $user $dbname < tests/_data/dump.sql'
    

    This approach is system-dependent, you can’t use the same config on Windows and Nix systems, but is much faster. Thanks Mauro Asprea @brutuscat for this feature.

    Db module defaults

    Important notice: we changed defaults for Db module, so cleanup and populate options are disabled by default. They were quite dangerous in use, so we decided that you need to set them explicitly in Db module config.


    Codeception 2.2.12 has been released as well. See complete changelog for all notable changes.


    P.S. Codeception is seeking for Symfony module maintainer. If you use Codeception with Symfony and you’d like to improve it, please contact us at team@codeception.com. Maintainer responsibilities are: review issues, pull requests, update symfony demo app and so on. Take a part in project development and make open source brighter!


    0 0

    In last few weeks Codeception received updates aimed to empower acceptance testing. We try to make PHP a better place for browser tests. As you know, QA engineers often prefer other languages like Java or Python for test automation. But PHP is not that bad by itself, it is simpler, it is faster in most cases, it has great IDEs. And for sure, we have Codeception. With it you can write the most complex acceptance tests in a simple scenario-driven manner.

    So what was added in the last release?

    RunProcess

    RunProcess extension was introduced. Use it to easily start/stop Selenium Server, ChromeDriver, or other required services for a test suite.

    Enable it in suite config or in environment:

    # acceptance.suite.yml
    extensions:
        enabled:
            - Codeception\Extension\RunProcess:
                - php -S 127.0.0.1:8000
                - java -jar selenium-server.jar
    
    # or inside environment config
    # in this case, run tests as
    #
    # codecept run --env selenium
    env:
      selenium:
        extensions:
            enabled:
                - Codeception\Extension\RunProcess:
                    - php -S 127.0.0.1:8000
                    - java -jar selenium-server.jar           
    

    Depending on the environment (dev host, CI server) you can easily switch setups if you use environment configs.

    SmartWaits

    This is the new unique feature of Codeception which incorporates implicit waits. By itself, the implicit wait was available in Codeception with wait: option in WebDriver config. However, it was not usable, because by design it slowed down test execution. In this release, we introduce the SmartWait concept. Implicit waits are used only when they are really needed and disabled for all other cases. This makes tests extremely stable and fast.

    Thus, a test won’t fail if expected element didn’t yet appear on a page but waits for it a few seconds more. Just set wait: 5 to WebDriver config to try it and read the documentation.

    Customization

    Codeception took to much of browser control. Let give it back to you. With start: false option you can disable autostart browser before a test, and create a browser session manually. Codeception doesn’t provide actions to start browser session inside a test (because it is supposed you already have one in a test), but you can write a custom method in a helper:

    <?php
    // support/Helper/Acceptance.php:
    // 
    public function startBrowser()
    {
        $this->getModule('WebDriver')->_initializeSession();
    }
    
    public function changeBrowser($browserName)
    {
        $this->getModule('WebDriver')->_restart(['browser' => $browser]);
    }
    
    public function closeBrowser()
    {
        $this->getModule('WebDriver')->_closeSession();
    }
    

    Then these methods can be used inside a test.

    <?php
    // I prepare something for a test
    $I->startBrowser();
    // I do something inside a browser
    $I->closeBrowser();
    

    If you use BrowserStack you can use this features to set a test name dynamically to capabilities. Here is how can you do it in Acceptance Helper:

    <?php
    // inside Helper\Acceptance
    // 
    public function _before(TestInterface $test)
    {
        $webDriver = $this->getModule('WebDriver');
        $name = $test->getMetadata()->getName();
        $webDriver->_capabilities(function($currentCapabilities) use ($name) {
            $currentCapabilities['name'] = $name;
            return new DesiredCapabilities($currentCapabilities);
        });    
        $webDriver->_initializeSession();
    }
    

    Please don’t forget to set start: false in config option, so browser wouldn’t be started twice!

    @prepare

    What if you need to change the configuration for a specific test? Let’s say you want to run all tests in Firefox but this specific one in Chrome? Or something like that.

    We added new annotation @prepare which can be used in Cest and Test formats. It will execute a method which can change the configuration before the module is called.

    <?php
    /**
     * @prepare switchToFirefox
     */
    public function testSomethingInFirefox(AcceptanceTester $I)
    {
        $I->amOnPage('/');
        $I->click('Detect Browser');
        $I->see('Firefox');
    }
    
    protected function switchToFirefox(\Codeception\Module\WebDriver $webDriver)
    {
        $webDriver->_reconfigure(['browser' => 'firefox']);
    }
    

    This @prepare can be used not only for browser tests but everywhere. Use it wisely!


    Is that all? Almost.

    We also updated WebDriver docs to include more options for Local Testing, like ChromeDriver. We also published a reference on running Chrome in Headless mode.

    Update to the latest 2.3.4 version to try all the new features.

    We worked hard to bring all this stuff to you. Now is your turn: please help to spread the word and encourage more of your colleagues to use PHP and Codeception for web testing. Setting up Codeception for web tests nowadays is as simple as running:

    codecept init acceptance
    

    Happy testing!


    0 0

    Before writing any line of test code we should think of how meaningful this test would be. The best code is the code you never write, you know. So if you ever begin to think of which parts of the application you should cover with tests.

    Let’s start with a heart of your application, the Business Logic.

    Layered Architecture

    Depending on architecture Business logic could be in Controller (ough), in Model (ough-ough), or in a special Service layer, which is the most preferred way. Business Logic responsible for taking decisions: which entities are valid and which are not, who can create entities, what happens when the entity is stored. Service layer should delegate the database operations to the infrastructure level.

    Infrastructure level is purely technical. It just takes orders from Service layer and executes them. It doesn’t make any decisions on its own. In most cases, ORM should be at the infrastructure level.

    While models or repositories do not contain any decision making logic it is not that important what ORM type to use: ActiveRecord or DataMapper.

    CommandBus and Domain Events

    One of the solution to separate business logic from everything would be the CommandBus architecture.

    CommandBus is well-known in PHP community. Learn more about it from

    If the application implements CommandBus pattern it is easy to find the business logic. It is located in command handlers. So let’s test them!

    Testing CommandHandler

    CommandHandler takes a command, calls the infrastructure services, and sends the domain events. In theory, CommandHandlers can be tested in isolation, with unit tests. Infrastructure services can be mocked and domain events can be caught. However, those tests may have a little sense. And here is why/

    “Trust me, I will save this to database” - may say the test with mocked infrastructure. But there is no trust to it. To make the command handler reliable and reusable we need to ensure it does what it is expected to do.

    Also, mocks comes with their price. While integration test is really similar to the actual business code, unit test is bloated with mock definitions. It will get hard to maintain and support it really soon:

    <?php
    // is that a business logic test?
    // but no business inside of it
    $eventDispatcher = $this->createMock(EventDispatcher::class);
    $busDispatcher = $this->getMockBuilder(BusDispatcher::class)
       ->setMethods(['dispatch'])
       ->getMock();
    
    $busDispatcher->expects($this->exactly(2))
       ->method('dispatch')
       ->withConsecutive(
           [$this->isInstanceOf(CreateAccount::class)],
           [$this->isInstanceOf(CreateCompany::class)]
       );
    
    $handler = new RegisterUserHandler($eventDispatcher, $busDispatcher);
    // ... 
    

    Here we also mix implementation with a specification, which is a pure sin. What if dispatch method will be renamed? What if we fire more than 2 commands in a call? How is this related to business?

    Even you can mock services you shouldn’t always do it.

    In most cases, business logic should be tested with an integration test. Because the database is an essential part of your application. You can’t deliver an app without a database. The same way you can’t make sure domain logic works as expected when nothing is stored in the database.

    Testing Flarum Forum

    As an example let’s pick Flarum project which is a forum built on top of Illuminate components (Laravel) and Symfony Components. What is most important that it has the Commands and CommandHandlers. By looking at one directory we can learn what Flarum does. That’s awesome!

    Looks like the #1 priority is to get all those Command Handlers tested. We can use StartDiscussionHandler to start.

    For integration tests, we need to initialize Application with its Dependency Injection Container. Then, we fetch StartDiscussionHandler out of it:

    <?php
    protected function _before()
    {
        // initialize Flarum app
        $server = new \Flarum\Forum\Server(codecept_root_dir());
        $app = $server->getApp();
        // get StartDiscussionHandler from container
        $this->handler = $app->make(StartDiscussionHandler::class);
    }
    

    When the handler is prepared we can write the first basic test:

    <?php
    public function testAdminCanStartADiscussion()
    {
        // ARRANGE: create command object with all required params
        $admin = User::find(1); // User #1 is admin
        $command = new StartDiscussion($admin, [ // create command
            'attributes' => [
                'title' => 'How about some tests?',
                'content' => 'We discuss testing in this thread'
            ]
        ], '127.0.0.1');
        // ACT: proceed with handler
        $discussion = $this->handler->handle($command);
        // ASSERT: check response
        $this->assertNotNull($discussion);
        $this->assertEquals('How about some tests?', $discussion->title);
    }
    

    How do you like this test? This test so tiny and so easy to read. Sure, we should add more assertions to find out that all the business rules are applied, but for now it’s ok to try the very basic scenario. Maintaining and extending this test will be a pleasure.

    A bit of Stubs and Mocks

    Integration test represents a part of a system, a working component. Once it triggers the outer service, it should be mocked. In layered architecture, a class from a layer should have access to its neighbors, and to classes of lower levels. So in our case, CommandHandler of business logic can access other command handlers but should be banned from accessing other services or other command buses.

    For sure, mailers, queues, and other asynchronous services should be replaced with stubs.

    Conclusion

    Your application has a heart. Don’t make it die from a heart attack. Write tests. Write meaningful stable tests that will last.

    Written by Michael Bodnarchuk

    We provide consulting services and trainings on Codeception and automated testing in general.


    0 0
  • 02/26/18--17:03: Codeception 2.4 released
  • Hello everyone! We’d like to announce the immediate availability of Codeception 2.4.0. This follows up the PHPUnit 7.0 release and mostly contains compatibility fixes for latest PHPUnit. Internal usage of PHPUnit in Codeception was switched to new namespaced class names. This doesn’t break any public APIs but can break some setups. That’s why this is a minor release with a proper announcement. Please upgrade and unless you have some PHPUnit version conflict you should not experience any issues.

    Codeception 2.4.0 is a stability release. We also dropped unmaintained PHP 5.4 and PHP 5.5. But we keep compatibility with PHP 5.6 so if your company still uses PHP 5 (and we understand business reasons) you can install this update too. Codeception is still going to be maintained with PHP 5.6 compatibility in mind. We don’t break API just for the sake of clean code. We try to make things reliable.

    In the same Codeception 2.3.9 is released time so if you want to get the latest changes and you still use old PHP, upgrade to this version.

    In Codeception 2.4 you will also find new hooks for Cest files:

    • _passed is called when a test inside Cest is successful
    • _failed is called for unsuccessful tests
    • _after is called at the end of a test for both cases.

    Previously _after hook worked as _passed and that behavior was confusing. This was changed so it can be potentially a breaking change as well.

    See changelog for the complete list of fixes and improvements.


    We also launched Codeception Slack to talk about testing and Codeception.

    Join Slack

    We’d also love to see more contributors there. We need more talented developers to help to bring new features and fix the issues. It’s opensource, after all. We face constant challenges: new APIs, new libraries, new approaches, etc. And Codeception as a really big project would benefit from any help. We are open to accepting new maintainers, to discuss trends, to build a better product! As you know, we are merging 90% of submitted Pull Requests so if you are not contributing, please go ahead!


    We try to keep Codeception going and bring more releases to you. If your company uses this framework and you’d like to give back please consider sponsoring Codeception. That’s right. We are asking to invest into open source, to get the features you expect and to give back to open source.

    Sponsor Codeception

    Please show this form to your company bosses. If you want to have stable reliable product it’s ok to pay for that. Be our first official sponsors!

    Yes, we also provide enterprise support and trainings. This is another way you can support the development. Thank you!


    0 0
  • 09/23/18--18:03: Codeception 2.5: Snapshots
  • The Autumn is a good time to start something new. How about new Codeception? As you know, each minor release indicates some important change, new features we wanted to share. This time we want to introduce you to

    Snapshots

    That’s the new feature, which might be useful if you are tired of harcoding your data into fixtures. For instance, when you test the search engine you don’t know the exact list of items to check, however, you are interested that the list would be the same every time for a search term. What about API responses? You don’t want to hardcode it fully but you may want to check that response is the same as it was before.

    So now Codeception can do what php-vcr was doing all the time. Saving data into the file (called snapshot), and comparing with it on next runs.

    This is nice feature for testing REST APIs.

    Let’s assume you have such JSON response.

    {
      "firstName": "John",
      "lastName" : "doe",
      "phoneNumbers": [
        {
          "type"  : "iPhone",
          "number": "0123-4567-8888"
        },
        {
          "type"  : "home",
          "number": "0123-4567-8910"
        }
      ]
    }
    

    And you want to check that phone numbers are the same on each run. For this we can use a snapshot.

    namespace Snapshot;
    
    class PhoneNumberSnapshot extends Codeception\Snapshot {
      
        /** @var \ApiTester */
        protected $i;
    
        public function __construct(\ApiTester $I)
        {
            $this->i = $I;
        }
    
        protected function fetchData()
        {
            // return an array of phone numbers
            return $this->i->grabDataFromResponseByJsonPath('$.phoneNumbers[*].number');
        }  
    
    }
    

    Then in test we can check if data matches the snapshot by calling the snapshot class:

    public function testUsers(ApiTester $I, \Snapshot\PhoneNumberSnapshot $shapsnot)
    {  
      $I->sendGET('/api/users');
      $snapshot->assert();
    }
    

    If the data changes, snapshot is easy to update. Just run the test in --debug mode. The snapshot will be overwritten with a new data.

    So, the good thing about snapshots:

    • you don’t keep flaky data in your code
    • you don’t need to hardcode data values
    • data can be easily updated on change

    Use them!


    There are also some other minor changes:

    • Db module now supports multiple database connections. If you use few databases we got you covered! That was a long awaited feature and finally we have a very nice implementation from @eXorus.
    • --seed parameter added to run so you could rerun the tests in same order. This feature works only if you enabled shuffle mode.
    • Possible breaking: seeLink behavior changed.
      • Previous: $I->seeLink('Delete','/post/1'); matches <a href="/post/199">Delete</a>
      • Now: $I->seeLink('Delete','/post/1'); does NOT match <a href="/post/199">Delete</a>

    See changelog for the complete list of fixes and improvements.

    Thanks to all our contributors who made this release happen!


    Call to Sponsors!

    We try to keep Codeception going and bring more releases to you. If your company uses this framework and you’d like to give back please consider sponsoring Codeception. That’s right. We are asking to invest into open source, to get the features you expect and to give back to open source.

    For instance, how would you like improving stability of WebDriver tests? It would be cool to automatically retry failed steps and rerun failed tests. These could be a very cool features but they can’t be made without your help.

    If you are interested consider sponsoring us:

    Sponsor Codeception

    Yes, we also provide enterprise support and trainings. This is another way you can support the development. Thank you!