Quantcast
Channel: Codeception
Viewing all 321 articles
Browse latest View live

ProTips in Managing UI

$
0
0

Hi, we've decided to start a pro tips post series on Codeception. We hadn't too much time recently, because of the work on our startup Bugira, and the release of community-driven catalog oj Javascript libraries - Jster. You should definitely visit Jster (and add it to bookmarks), as it contains more then 800 javascript libraries for any needs in frontend development. But finally, We can get back to Codeception and tell about it's best usage practices.

Tip 1: Variables for UI elements

You can write pretty complex tests using Codeception. When you have lots of them you may find that changing page UI elements can lead to refactoring all the tests. If you plan to develop a solid automation platform for your web application, consider putting all UI elements into the variables. In this case you can modify your tests pretty easy. If you see like you are creating too much tests, you should think on reorganizing them in a way that can act stable despite of changes in project. It's really hard, features, designs, can constantly change and affect tests. But it's possible to keep your tests stable if you use variables instead of raw values in tests.

Let's say we have an acceptance test tests/acceptance/CreateTaskCept.php:

<?php
$I = new WebGuy($scenario);
$I->wantTo('create todo task');
$I->amOnPage('/tasks');
$I->fillField('New Task','Update a blog');
$I->click('Add');
$I->see('Update a blog', '#tasks');
?>

What if we change the name of buttons and the DOM position of the element where "Update a blog" text should appear? Let's move this elements to a _bootstrap file and push them into variables.

In tests/acceptance/_bootstrap.php:

<?php
$task_add_button = 'Add';
$task_new_field = 'New Task';
$tasks_list = '#tasks';
?>

And in tests/acceptance/CreateTaskCept.php:

<?php
$I = new WebGuy($scenario);
$I->wantTo('create todo task');
$I->amOnPage('/tasks');
$I->fillField($task_fill_field,'Update a blog');
$I->click($task_add_button);
$I->see('Update a blog', $tasks_list);
?>

As you see replacing values with variables shouldn't affect the test readability while you choose proper names for UI elements. We can also recommend to store user names, passwords, etc in bootstrap too. It will be automatically included on each test run. Use it.

Tip 2: Use PageObject

The PageObject is a pattern very popular among QA automation professionals. PageObject allows us to organize variables from a previous tip in more structured way. Instead of using raw variables we will group them into a class and use them in test and helpers. But classes and constants can't be added to _bootstrap, because bootstrap will be loaded before each test, so it will trigger an error: "Fatal: Cannot redeclare class". Let's deal with that by creating a new PHP file for TodoTask page class:

In tests/acceptance/_todoTaskPage.php:

<?php
class TodoTaskPage {
    const URL = '/tasks';

    static $add_button = 'Add';
    static $new_field = 'New Task';
    static $list = '#tasks';   
}
?>

Let's add it to tests/acceptance/_bootstrap.php:

<?php
require_once '_todoTaskPage.php';
?>

In this case you won't get lost in where which UI element is located. Because in PageObject pattern the UI element is bound to a page, which we defined by the URL property. And here is the updated test:

<?php
$I = new WebGuy($scenario);
$I->wantTo('create todo task');
$I->amOnPage(TodoTaskPage::URL);
$I->fillField(TodoTaskPage::$new_field,'Update a blog');
$I->click(TodoTaskPage::$add_button);
$I->see('Update a blog', TodoTaskPage::$list);
?>

After this update we have all the UI elements for todo task page kept in one place. Using and IDE you can easily find the right element and insert it into any test of your suite. Despite the raw variables classes and constants are accessible in helpers and unit tests. Also you can improve the page object classes with PHP's OOP power. If you are using PHP 5.4 you can include global UI elements (menus, breadcrumbs) with traits.

Lets create a trait for menu component in tests/acceptance/_menu.php:

<?php
trait GlobalMenu {
    public static $global_menu = "//div[@id=menu]";

    public static function menuItem($index)
    {
        return self::$global_menu.'/ul/['.$index.']';
    }
}
?>

We add it to tests/acceptance/_bootstrap.php:

<?php
require_once '_menu.php';
require_once '_todoTaskPage.php';
?>

And we are using it in previously declared TodoTaskPage:

<?php
class TodoTaskPage {

    use GlobalMenu;

    const URL = '/tasks';

    static $add_button = 'Add';
    static $new_field = 'New Task';
    static $list = '#tasks';   
}
?>

And now we can use properties and methods of a trait included in class. Thus, TodoTaskPage, as any class that uses GlobalMenu trait can access a menu. Let's write a new test for checking that menu contains a button to create a task.

<?php
$I = new WebGuy($scenario);
$I->wantTo('check task creation link is in a menu');
$I->amOnPage(TodoTaskPage::URL);
$I->see('Create Task', TodoTaskPage::menuItem(3));
?>

If you think on layout components as a traits and pages as classes, you can reorganize your UI elements in a very flexible way. Changes in markup or texts won't affect a tests, as you can easily update all the tests that rely on that element. You can even teach your template developer to update the corresponding PageObject each time she changes template.

Conclusion

Codeception is simple, yet powerful tool for test automation. If you plan to use test automation for the enterprise level projects, it's better to start with proper organization. As we think on architecture before writing the first line of code, we should think on scalable architecture of tests platform. Using PageObject and variables can dramatically improve the reusability (and keep KISS and DRY principles) of your tests.


Protips 2 - Extending Modules

$
0
0

Today, we will continue covering advanced topics on Codeception. The main power of Codeception is it's modules, there are already plenty of them, even though you might want to extend one of them. Let's say you are using Symfony2 and you want to add additional actions, you actually miss in current module, like dealing with FOSUserBundle: registering, authenticating, etc. You can create a helper and connect with Symfony2 module, get access to dependency injection container and do anything you need there. Sometimes you might want to change the module initialization scripts. This happens if your application configured in a custom way and module doesn't work properly because of it.

In both cases you might want to improve current Symfony2 (or any other module) implementation. That's pretty easy, because we can use just the simple OOP inheritance. You can create your own module Symfony2Helper and inherit it from default Symfony2 module. This module will act as a regular Helper, and should be placed to tests/_helpers dir.

In the next example we will redefine initialization for working with Symfony2 applications located in app/frontend path with Kernel class located in app.

<?php
class Symfony2Helper extends \Codeception\Module\Symfony2
{
    // overriding standard initialization
    public function _initialize() {
        // bootstrap
        require_once getcwd().'app/frontend/bootstrap.php.cache';
        // kernel class
        require_once  getcwd().'/FrontendKernel.php';

        $this->kernel = new FrontendKernel('test', true);
        $this->kernel->boot();

        $dispatcher = $this->kernel->getContainer()->get('event_dispatcher');
        $dispatcher->addListener('kernel.exception', function ($event) {
            throw $event->getException();
        });
    }    
}
?>

Please, refer to the _initialize method implementation to understand the default behavior. To get the idea whether you need to inherit and redeclare methods of module, you need to review it's code. If you see your requirements can't be met by using config options, please override.

By using inheriteance you can redeclare all the methods or initialization hooks you don't like. The API of parent module is pretty clean and even when you use the phar version of Codeception, you can read it's code on GitHub. If you stuck because Codeception module works improperly, you are free to fix that.

And yep, if your improved module is worth sharing with community, submit a patch and it will be added to official distribution. After all, that's how the Open Source works.

Thanks for reading. Use Codeception professionally.

Stop Simulating BDD, Write Tests

$
0
0

I keep receiving one question. Well, actually two: "What makes Codeception different comparing to Behat". And the next one: "Why can't I write Codeception tests in human language like Behat does". I think the time has come to answer this questions.

As you know, the Behat project is not only a tool but a methodology. The Behavior Driven Development concept requires that the User Stories to be written, and then executed as a test. Behat is a right tool for teams who strictly follow the BDD methodology. But what if you don't? For what in the world you need tests written in plain English?

If you write a story like this, using Behat and MinkContext for example,

Given I am on "/product/1"
When I click "Purchase"
And I fill in "name" with "Michael"
And I fill in "credit" with "321123123"
And I click "Order"
Then I should see "The product was purchased!"

you actually write test, and not a story. Because the feature you describe won't change If you rename field "credit" to "credit card", but the test will fail. You may change the text "The product was purchased" to "iPhone5 was purchased. Thank you!" and that will make test fail too. So you will rewrite the user story every time you change every detail that may affect passing test. In this case the manager who wrote this spec will debug and update user story to make the test pass.

And so your manager becomes tester. His role is now not only to deliver specifications to team but also to make this specifications written as tests and make them pass.

But what If you want one additional step to the scenario?

Given I am "/product/1"
When I click "Purchase"
Then I should see "1 product" in the bin

For this step see "1 product" in the bin you should write a custom step. If only your manager knew PHP for it! Ideally to become manager + tester + PHP guru. But no, manager needs to ask your developer to write the necessary step. Thus, already two people are writing the test. And one of them knows PHP. And when you will hire a QA will you require a knowledge of PHP? Well, yep. Thus because you don't want to disturb your PHP devs and assign them additional tasks.

When your product growths your tests becomes more and more complex. And you need to keep them updated for each change. When you use the plain English text, you don't have a control over tests. When your form don't have a label you need to write custom step instead of I fill in, because it's actually wrong to use CSS or XPath selectors inside a Behat feature. It's not plain English text anymore with CSS. You need PHP developers to create additional steps.

Why Codeception is better testing tool? Well, just because it is a testing tool at first. Spend a day, to teach your manager/tester/qa PHP DSL, install a Netbeans or PHPStorm (or any other IDE), and start writing tests. Even when tests looks similarly to Behat features they give more power and control over the process.

<?php
$I = new WebGuy($scenario);
$I->amOnPage('/product/1');
$I->click('Purchase');
$I->fillField('name', 'Michael');
$I->fillField('credit', '321123123');
$I->click('Order');
$I->see('The product was purchased!');

This test in the process of evolution can be transformed into something more complex. You can use CSS and XPath everywhere, so when you change the "Order" button with an image, you can use it's CSS and quickly update a test to make it pass. Also, you can move repeatable elements into variables and classes and make a test that looks like this:

<?php
$I = new WebGuy($scenario);
$I->amOnPage(ProductPage::URL);
$I->click('#purchase');
$I->fillField('name', $user_name);
$I->fillField('credit', $user_credit);
$I->click('img a#order');
$I->see(ProductPage::$successMessage);

Just the same scenario. Well it's a bit harder to read, as we added CSS selectors, but as you see this test pretty much more flexible. It can be easily refactored and improved without writing any custom method to helper.

Don't fall into a marketing trap. You will find yourself writing tests two times: as a feature in plain English and in code with PHP.

It's better to have one person who can take a full control over test automation then, delegate some tasks to developers, some tasks to managers. If you are developer and you work on your own project. Why PHP is not suitable for you? Why do you want to write code 2 times: once in feature with Gherkin and second time in Context. Is it KISS? Is it DRY? I don't think so.

Will Codeception support plain text test scenarios? Really, I thought on that. But I don't want to put the limits. Using plain text makes impossible to use variables, loops, grabbers, and thus use PageObject pattern. Which is very important for solid and concrete test automation platform.

Behat is great BDD tool and it's author @everzet also states: don't use it for functional testing. Choose the right tool for job.

Disclaimer: I'm not very good in theory, I'm very practical guy. I won't refer you to any published books or methodologies. But If you want an authority... Well, David Heinemeier Hansson (the creator of Rails) wrote in his blog: 1"Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests". And Behat is Cucumber for PHP. DHH is known for his dislike to popular BDD tools like RSpec or Cucumber. But he is very pragmatic and very scrupulous in testing.

Codeception released with CodeCoverage support

$
0
0

We'd like to announce new Codeception 1.5 major release. This time our improvements are CodeCoverage and Remote CodeCoverage added. These features allows you to collect code coverage reports for all your tests: unit, functional, and acceptance and merge them together. So now you can review what parts of your applications are tested and which are not even for Selenium and PhpBrowser tests which are executed on a webserver.

Read more in the new Guides chapter.

There is no magic in local codecoverage. XDebug and PHP_CodeCoverage libraries do their job. The tricky thing is remote codecoverage. We attach small script into application's front controller. When a special header is sent this script starts to collect coverage information. And in the end of tests, this data is merged, serialized and sent back to Codeception. So you can test and collect coverage report even on staging servers in real environment.

code coverage

Thanks to tiger-seo for codecoverage feature. He did a great job developing a remote script c3.php which is a unique in it's way.

But back to Codeception. As you may've noticed our website is updated too. Documentation was improved, search was added, and nice Quickstart page added too. So if you didn't try Codeception yet, it's very easy to start now. In only 6 steps.

Modules

Two useful modules were introduced by judgedim. We have support of MongoDb and Redis now. They both can clean up your storages between tests and Mongo can perform checks in your collections. So If you are working with NoSQL databases, you should try them in your tests. If you use different NoSQL databases, please submit your patches, and they will be included in next release.

UX Improvements

Now you can execute a test by providing relative path to test, like.

php codecept.phar tests/acceptance/SignInCept.php

This small tweak imprioves user experience for *nix users as they can use autocompletion when running a test.

Also you can run test from one specific directory, i.e. match a group of tests:

php codecept.phar tests/acceptance/admin

Bugfixes

Composer package is works again. It's really hard to follow the stability in the world of constant changes, so we recommend use of phar for testing, just because it's prepackaged and always runs as expected. But if you use Composer it's easy to add Codeception to your vendors and receive all new updates with new release. Don't forget to mark @stable Codeception version.

Install

As usual, Codeception 1.5.0 can be downloaded from site,

installed via PEAR

$ pear install codeception/Codeception

or via Composer

$ php composer.phar update

Thanks

As you may've noticed all that guys who took part in developing Codeception are now shown on every page of this site. Thn this way we say thank for all our contributors and all guys who support this project, for all companies that adopt Codeception in their workflow.

Codecepion 1.5.1

$
0
0

You know, it's a common situation when just after the brand major release comes a small release with patches and bugfixes. Yeah, Codeception 1.5.1 has nothing to surprise you, still it's reommended to upgrade if you are running the phar vs composer issue. As we discovered with the help of alexshelkov, that Codeception packaged phar was trying to load PHP libraries from a local composer installation. So if you was trying phar version in project that already use composer, you may have seen strange errors.

This bug was fixed and last version 1.5.1, ready for download.

A simple improvement was introduced by DevMan. It allows you to execute Codeception from any directory if you set a path to config file with a -c parameter.

php codecept.phar run -c /var/myproject/codeception.yml

And finally we are back green on Travis. We had some issues with Travis infrastructure (not the tests), but we managed to solve them. Did you know that it takes 8 minutes to run all internal tests of Codeception? Yeah, we've got lots of tests )

Build Status

Update

Please redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

Codecepion 1.5.2

$
0
0

Yet another minor release. It introduces more compact form for unit tests.

<?php
use Codeception\Util\Stub;

class UserRepositoryTest extends Codeception\TestCase\Test
{
    protected function _before()
    {
        $this->user = new User();
    }

    public function testUserIsNotAdmin()
    {
        $this->assertFalse($user->isAdmin());
    }

}
?>

We moved all Codeception's setUp and tearDown routine into parrent class and left you more clean _before and _after methods for your preperations. They act just the same as setUp and tearDown for PHPUnit. So _before is executed before each test and _after after it. Tests should look more clean now. If your tests already have setUp and tearDown methods nothing is changed for them.

Accessing Modules in Unit Tests

Another nice feature for Symfony2 users and not only them. Form now on you can access a module from your unit test. This is done to access public properties of your module. For example, here is how you can retrieve a Symfony Dependency Injection Container:

<?php
protected function _before() 
{
    /**
   * @var $container Symfony\Component\DependencyInjection\Container
   */
    $container = $this->getModule('Symfony2')->container;
    $this->em = $container->get('doctrine');
}
?>

Also we'd like to announce the upcoming modules for Yii and Laravel frameworks.

In the meantime we will release more complete guides on How to integrate framework to Codeception.

Bugfixes

  • Composer build is finally is stable and doesn't require minimal-stability: dev. Thanks to: jacobkiers.
  • Adding support of pecl_http by phpcodemonkey.
  • Methods of Guy received documentation and examples (phar bug).
  • generate:cest command requires a tested class name no more.
  • PostgreSQL driver improved .

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

or via PEAR:

$ pear install codeception/Codeception

Connecting PHP Frameworks. Part 1

$
0
0

As you know, Codeception has various modules for functional testing of PHP applications. Really, you could use Selenium, but as an advice, please leave Selenium for QAs.

As a developer you need to get more technical details for each test. If test fails you need to see the actual stack trace. As a developer you need to have all test running much faster. You can't wait for an hour before pushing your commit.

If you ever had tried Codeception with Symfony2 you should know how fast are functional tests. You should know how helpful is printed information from profiler: queries used, user authentication status, etc.

In case you want to try functional tests with your own framework you should develop module for it. Basically it's not that hard as you think it is. But some frameworks suit better for integrations and some need additional hooks. In this post I will concentrate on integrating one of the modern PHP frameworks that use HttpKernel Symfony Component.

There are Laravel 4 and Drupal 8 that adopted HttpKernel interface. And more frameworks to come...

So in case you want to integrate Laravel or Drupal, or you know that your framework uses HttpKernel, you should follow this guide.

If you are not aware of what HttpKernel class is, please follow Fabien Potencier's' series of posts Create your own framework... on top of the Symfony2 Components

Contributor's Notice

First of all, how do we treat the user contributions? In Codeception we have liberal politics for accepting pull requests. The only thing, we can't test your implementation, as we just don't have experience in this frameworks. So when you commit module please test your module, and prepare to be it's maintainer. So you will need to write proper documentation and provide support it. Answer to questions on GitHub, in Twitter, etc. Yep, this is opensource.

But we will help you with that. The more developers will use your framework the more contributions your module will receive. So try to encourage framework community to test your module, use it and improve.

When your module is complete it will be packaged with Codeception and it's reference will be published on this site. This is done to make Codeception with all it's modules work out of the box from one phar archive.

Technical Implementation

So you decided to create a module for Codeception that provides integration with HttpKernel. Hope you do!

Check how it is done for Symfony2

  1. We load and initialize HttpKernel class in _initialize method.
  2. Before each test we create a HttpKernel\Client class for this kernel in _before method
  3. We shut down client after each test in _after method.

Let's narrow it to example.

<?php
namespace Codeception\Module;
use Symfony\Component\HttpKernel\Client;

class MyFrameworkModule extends \Codeception\Util\Framework {

    public function _initialize()
    {
        // $app implements HttpKernelInterface
        $app = require_once \Codeception\Configuration::projectDir().'/app.php';
        $app->setEnvironment('test');
        $this->kernel = $app;
    }

    public function _before(\Codeception\TestCase $test)
    {
        $this->client = new Client($this->kernel);
        $this->client->followRedirects(true);
    }

    public function _after(\Codeception\TestCase $test)
    {
        $this->kernel->shutdown();       
    }
}
?>

And basically that's all you need for integration. The Client class has everything to simulate requests and parse responses. Every module that extends Codeception\Util\Framework class will have actions: click, see, fillField, defined and documented in Codeception\Util\FrameworkInterface. This actions will work in just the same manner as for other frameworks. And it's really cool, that testing client is not aware of framework it is testing. All methods and their behavior are just the same. So tests for Symfony2, Zend, or your newly integrated frameworks will look just the same.

Still you may want to add something special for your framework. Maybe some additional initialization steps, or new actions. Let's say a framework you integrate have methods to authenticate a user by name. Why not to use this ability and to make a short cut for logging in?

<?php
namespace Codeception\Module;
use Symfony\Component\EventDispatcher\EventDispatcher;
use Symfony\Component\HttpKernel\Client;

class MyFrameworkModule extends \Codeception\Util\Framework {

    public function _initialize()
    {
        // $app implements HttpKernelInterface
        $app = require_once Codeception\Configuration::projectDir().'/app.php';
        $app->setEnvironment('test');
        $this->kernel = $app;
    }

    public function amLoggedAs($username)
    {
        $this->kernel->getService('session')->authenticate($user);

        $role = $this->kernel->getService('session')->getRole();
        $permission = $$this->kernel->getService('session')->getPermissions();

        // let's display additional information
        $this->debugSection('Role', $role);
        $this->debugSection('Permissions', json_encode($permission));
    }
}
?>

So now we can write a test like this:

<?php
$I = new TestGuy($scenario);
$I->amLoggedAs('davert');
$I->amOnPage('/user/profile');
$I->see('Davert');
?>

As you see, framework integration allows us to access it's internals. Services, classes, configurations, etc. Please add methods that you think may be useful for other developers that will write functional tests. Also you can display some technical details with debug and debugSection methods.

Let other modules get access to framework internals too. In our case we can define kernel as public property and make it accessible from user helper classes.

<?php
class MyFrameworkModule extends \Codeception\Util\Framework {
    public $kernel;
}    
?>

In helper:

<?php 
class TestHelper extends Codeception\Module {

    public function doSomeTrickyStuff()
    {
        $kernel = $this->getModule('MyFrameworkModule')->kernel;
        $kernel->doTrickyStuff();
    }
}
?>

In case you want to provide flexibility you can add configurable parameters to your module. Let's update our example to use additional parameters.

<?php
class MyFrameworkModule extends \Codeception\Util\Framework {
    
    // paramter with default var
    protected $config = array('file_name' => 'app.php');

    // a parameter that we can't guess 
    protected $requiredFields = array('app_name');

    public function _initialize()
    {
        // $app implements HttpKernelInterface
        $app = require_once Codeception\Configuration::projectDir().$this->config['file_name'];
        $app->setEnvironment('test');
        $app->init($this->config['app_name']);
        $this->kernel = $app;
    }
}    
?>

Module won't start if app_name parameter is not set. But file_name parameter is optional and can be redefined.

In the end here how should look functional.suite.yml that includes your newly developed module:

class_name: TestGuy
modules:
  enabled: [MyFrameworkModule, TestHelper]
  config:
      MyFrameworkModule:
          app_name: blog

Keep in mind that main principles of all Codeception modules are simple and smart. This means, if you can guess some parameters - do not require them to be explicitly set.

Conclusion

It's pretty simple to integrate any framework if it's modern and built with Symfony Components. If you have various projects using this framework and you want to get it tested well, try to develop integration. It's not that hard: only 3 methods to implement. Share your work with others, and let it be added to main package.

We appreciate all contributions and all frameworks. Let's unite the PHP world!

Codeception 1.5.3

$
0
0

Urgent fix for Selenium2 module. It appeared that Selenium sessions were not restarting. The Travis-CI didn't reveal this problem. Selenium tests were passing as usual, just because there were no session related tasks. Main purpose for 1.5.3 was to release this fix. Some other fixes are also included, but it's the most important one.

Modules

In this release we also introduce very basic implementation of Yii module by Ragazzo. If you use Yii, please try it and help us improve Yii integration. To start using, please review the module documentation and issue description on Github.

Also we've got some improvements for AMQP module. Not only queues are cleaned up between tests, but also you can check if messages came to queue, grab them from queues, or publish new messages. This module was tested with RabbitMQ, but it should work with other AMQP providers too. This module is based on php-amqplib.

Generators

From now on you can generate classical PHPUnit tests, with no Codeception stuff in it. Just the plain old PHPUnit tests that you can run natively with PHPUnit itself. For this we have new command generate:phpunit

Use it as regular:

php codecept.phar generate:phpunit NewTest

Also generators now support namespaces. So when you type class name with namespaced, Codeception will generate directories (according to PSR-0) and will put namespace into your newly generated file.

Check out sample usages.

php codecept.phar generate:cest \\Entity\\User
php codecept.phar generate:test \\Entity\\User
php codecept.phar generate:phpunit unit \Entity\\User
php codecept.phar generate:phpunit unit "\Entity\User"

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Testing Symfony2 Apps with Codeception

$
0
0

From the beginning of it's existence Codeception was in good relations with Symfony2 framework. Codeception was built on Symfony Components and uses BrowserKit and HttpKernel components for launching functional tests. It's a shame we didn't have a complete Symfony2 integration tutorial before. But we will try to fill this gap today.

What benefits you get by using Codeception with Symfony2? Let's list all of them:

  • user-friendly syntax for functional tests
  • access to container in unit tests
  • testing REST and SOAP services built on Symfony
  • fastest data cleanup when using with Doctrine2

The installation is pretty simple. You can use Composer (as you are used to it), but we'd recommend to try phar package. In this case you can avoid unnecessary dependencies. But all this versions are equal. And when you installed Codeception and executed a bootstrap command you should configure your functional test suite.

In file tests/functional.suite.yml:

class_name: TestGuy
modules:
    enabled: [Symfony2, Doctrine2, TestHelper]

And nothing more. You just need to declare that you will be using Symfony2 and Doctrine2. The Symfony2 module will search for AppKrenel on initialization (in app) and use it for functional tests. Doctrine2 module will find that Symfony2 module is declared and will try to receive default database connection from container. Of course, If you use non-standard setup this behavior can be reconfigured.

Functional Testing

Let's create a first functional test. We use generate:cept functional TestName command.

<?php
$I = new TestGuy($scenario);
$I->wantTo('log in to site');
$I->amOnPage('/');
$I->click('Login');
$I->fillField('username', 'admin');
// ....
?>

And so on. Unlike standard Symfony2 tests you don't need to deal with filters, CSS, and XPaths. Well, you can use CSS or XPath in any selector, if you need to. But on start you can keep your test simple and compact.

The commands we use here are common for most modules that perform testing of web application. That means, that once you discover a need for Selenium, this test can be executed inside a web browser using Selenium2 module. But some commands are unique to Symfony2 module. For example, you can use seeEmailIsSent command that checks if application has submitted an email during the last request. Check Symfony2 module reference for all commands we provide.

Unit Testing

Codeception's unit tests are a bit improved PHPUnit's tests. Inside a unit tests you have access to initialized modules and guy classes.

<?php
class PaginateTest extends \Codeception\TestCase\Test
{
    private $serviceContainer;
    protected $codeGuy;

    protected function _before()
    {
        // accessing container
        $this->serviceContainer = $this->getModule('Symfony2')->container;
        $this->serviceContainer->enterScope('request');
    }

    public function testDefaults()
    {
        // using container
        $this->serviceContainer->set('request', new Request(), 'request');
        $paginate = new Paginate($this->getModule('Symfony2')->container);
        $this->assertEquals(1, $paginate->getCurrentPage());
        $this->assertEquals(0, $paginate->getCount());
        $this->assertEquals(20, $paginate->getLimit());

        // checking data in repository
        $this->codeGuy->seeInRepository('SettingsBundle:Settings', ['pager_limit' => 20]);
    }
}    
?>    

If you want to bypass Codeception hooks for PHPUnit you can create a plain PHPUnit test with generate:phpunit command and get a traiditional PHPUnit's test. Then you can put this test into bundle.

Conclusion

Ok, you will probably ask: why is it better then Behat. We have a wide answer for that. A short is: Codeception is for testing, Behat is for Behavior Driven Development. If you need a professional testing tool that supports PageObject pattern, complex Locators, refactoring capabilities and CodeCoverage - Codeception is a good choice for that.

We say thanks to @everzet for wonderful Mink (that is used for acceptance tests) and to Sebastian Bergmann for it's PHPUnit. Codeception uses their powers, but makes them a bit simpler in use.

Codeception 1.5.4. Skip Tests, UTF fixes, etc.

$
0
0

Today we are releasing Codeception 1.5.4. This is minor bugfixing release consisting of GitHub issues. And a small yet awaitied feature introduced: ability to skip and mark Cepts as incomplete.

It's pretty simple, though.

<?php
$scenario->skip();

$I = new WebGuy($scenario);
$I->wanTo('do something, but I would rather not');
$I->amOnPage('/');
//.....
?>

This makes this test to be skipped. Similarly it can be marked as incomplete.

<?php
$scenario->incomplete();

$I = new WebGuy($scenario);
$I->wanTo('do something, but I would rather not');
$I->amOnPage('/');
//.....
?>

The skip and incomplete methods can take one argument, that descrines a reason why this tests were marked this way.

<?php
$scenario->skip('waiting for new markup');
$scenario->incomplete('welcome page appearence not tested');
?>

Bugfixes

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Codeception 1.5.5 and Roadmap Announced

$
0
0

Yes, here is a new version of Codception with more features and bugfixes in it. We have a flexible relase cycle, so the new version comes when we have a set of updates that you might be needed. We want to say "thank you" for all contributors, and for everyone who helps making Codeption better. It's always nice to get pull requests or follow discussions in our issues secition. So let us sum up the work was done during last 2 weeks and share it with you.

Use Codeception in Different Places

All codeception commands got -c option to provide a custom path to tests. The exception is bootstrap command. It accepts a custom directory as a second argument:

php codecept.phar bootstrap ~/mynewproject
php codecept.phar generate:cept acceptance Login -c ~/mynewproject
php codecept.phar generate:test unit MyFirstTest -c ~/mynewproject
php codecept.phar run -c ~/mynewproject
php codecept.phar generate:scenarios acceptance -c ~/mynewproject

Alternatively you may specify path to codeception.yml file in -c option: php codecept.phar run -c ~/mynewproject/codeception.yml

Thus, you don't need to keep your codecept.phar in the root of your project anymore. Use -c option and one local runner for all your projects.

Skipping Tests

Skipping and marking tests of incomplete was improved. We did a new solid implementation for it (it was very basic in 1.5.4). Now If a test is marked to skip, no modules will be touched.

<?php
$scenario->skip('this is not ready yet, move along');

$I = new WebGuy($scenario);
$I->wanTo('do something, but I would rather not');
$I->amOnPage('/');
//.....
?>

This feature required to rework some core classes (like Step and TestCase and Scenario) but hopefully we simplified our code.

Bugfixes

  • acceptPopup with Selenium 2 does not trigger Exception any more
  • error handling was improved to skip blocked with @ alerts, yet to throw ErrorException for notices, warnings, errors.
  • ZombieJS configuration was fixed. Now the url parameter is required to specify app's local url.
  • REST seeStatusCodeIs method works correctly with Symfony2 now.

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Roadmap

For the first time we will announce the roadmap for Codeception. Actually we need your opinion: what features you'd like to see in new releases, and what things can be postponed. The list below is not a precise plan, but just a list of features we have in mind:

  • Make a PageObject pattern first-class citizen in Codeception. Add generators and guides to use PageObjects (for acceptance tests).
  • Multiple sessions for tests execution (see discission on GitHub)
  • Silex, Laravel 4, Zend Framework 2, Drupal 8, Phalcon integrations. The key problem here: we can't do this on our own. We need a real users of these frameworks, to create integration and test it on their projects. We have reworked functional testing guide to help you with this. Also use GitHub or personal contacts if you want to make a module.
  • Scenario Unit Tests to be rethinked. We have 2 options here: dump scenario driven unit tests (or mark them as deprecated) or rework them. Actually we need your real opinion. Here is an example of what new Cests may look like. They will dramatically improve the way you work with mocks and stubs in PHP. But will you use it? Please, let us know what you think.

To summarize: we'd appreciate contributions, feedbacks and ideas for next releases.

Codeception 1.5.6

$
0
0

Codeption 1.5.6 released with varous fixes to Selenium2 module (popups, screenshots) an other bugfixes. We added a bunch of tests for Selenium2 and they are runnng on TravisCI. Also, Codeception is now compatble with Symony 2.2. And few features were intrduced.

Basic Fixtures

Very basic implementation of fixtures was added to Db and Mongo modules. Now you can write something like:

<?php
// for Db
$I->haveInDatabase('users', array('name' => 'davert'));
// for Mongo
$I->haveInCollection('users', array('name' => 'davert'));
?>

to add data for a test. For Db module, the inserted record will be cleaned up on text test, even If cleanup option is set to false. So haveInDatabase method cleans data after itself. Something similar exists for Doctrine2 module. Yet, it looks not like for Db and Mongo implementation:

<?php
$I->persistEntity(new \Entity\User, array('name' => 'Miles'));

Error Handling

From now on you can set custom error handling parameters per suite or globally.

class_name: WebGuy
modules:
    enabled: [PhpBrowser, WebHelper]
error_level: "E_ALL & ~E_STRICT & ~E_DEPRECATED"

Actually, E_ALL & ~E_STRICT & ~E_DEPRECATED is default error level for now. This may be changed in future.

Bugfixes

  • popups and modals in Selenium2 are fully tested and work correctly now
  • screenshots in Selenium2 module is saved correctly
  • seeInField method works correctly even if field is not in form
  • fix to AMQP module to push message to queue
  • Symfony 2.2 compatibility

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Headless Browser Testing with Selenium2 and PhantomJS

$
0
0

This is a guest blogpost by Jon Phipps. Jon explains how to use PhantomJS -- the most anticipated testing backend in Codeception.

If you're running acceptance tests in Codeception that require interaction with JavaScript, or have scripts that manipulate the DOM, the speedy Php Browser driver won't help you since it doesn't support JavaScript. The best options for running tests using JavaScript are the Selenium2 and ZombieJS drivers.

The Selenium2 driver actually loads and runs an active browser session, manipulating the browser just as a human would. ZombieJS is a 'headless' browser that provides all of the features of a regular browser, but without a display interface. Without the extra time spent waiting for the display to actually render, a headless browser like ZombieJS can run far faster than a normal browser, so you're tests will execute in as little as half the time. But ZombieJS requires installing Node.js and can be a little buggy, plus it has its own API (which has both pros and cons). The Selenium2 driver is well tested and implements a standard API -- the WebDriver Wire Protocol -- across all of the browsers it has drivers for.

Now there's a headless browser that includes a WebDriver Wire Protocol implementation -- PhantomJS. The latest version of PhantomJS is an easy to install, stand-alone binary that doesn't require installing Node.js or any other dependencies, and ships with its own 'Ghost Driver' for implementing the WebDriver Wire Protocol. Which means you can drive it using the Selenium2 driver in Codeception, and anything that you can test in Chrome, Firefox, Safari, or IE using Selenium2, you can now test in half the time using PhantomJS

To get started, if you haven't installed Selenium2, you just need to follow the instructions on the Codeception web site for installing and running the Selenium2 driver.

Next download PhantomJS. The binary is setup and ready to use for Linux, Mac OSX, and Windows. Put it, or a symlink to it, somewhere in your path so you can launch it from anywhere.

You're done with installation!

Now open up a new terminal window and fire up an instance of Selenium Server, leaving the terminal window open:

$ java -jar selenium-server-standalone-2.0b2.jar

This will launch the server listening on the default port of 4444. You should see something like this in the terminal:

May 10, 2013 9:41:38 AM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
09:41:46.852 INFO - Java: Apple Inc. 20.45-b01-451
09:41:46.857 INFO - OS: Mac OS X 10.7.5 x86_64
09:41:46.941 INFO - v2.32.0, with Core v2.32.0. Built from revision 6c40c18
09:41:47.774 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
09:41:47.775 INFO - Version Jetty/5.1.x
09:41:47.775 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
09:41:47.776 INFO - Started HttpContext[/selenium-server,/selenium-server]
09:41:47.777 INFO - Started HttpContext[/,/]
09:41:47.983 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@1f25fefa
09:41:47.983 INFO - Started HttpContext[/wd,/wd]
09:41:48.011 INFO - Started SocketListener on 0.0.0.0:4444
09:41:48.011 INFO - Started org.openqa.jetty.jetty.Server@16a4e743

If you already have a listener on that port, you'll see a handy error message:

09:50:34.172 WARN - Failed to start: SocketListener0@0.0.0.0:4444
Exception in thread "main" java.net.BindException: 
Selenium is already running on port 4444. Or some other service is.

Next, open up a new terminal window and launch PhantomJS, telling it to use the its built-in WebDriver extension to talk to Selenium on the port Selenium is listening to, and leave that window open too:

$ phantomjs --webdriver=4444

You should see a response like this in your terminal:

PhantomJS is launching GhostDriver...
[INFO  - 2013-05-10T14:11:05.076Z] GhostDriver - Main - running on port 4444

Now just change the browser setting in your acceptance.suite.yml file (an example file is on the Selenium2 driver page) to browser: phantomjs. If you're changing modules then you should also run php codecept.phar build.

Check it out by doing a fresh Codeception run:

$ php codecept.phar run acceptance

Your tests should now run quietly and silently, and much faster.

You should see some output in your PhantomJS terminal window providing some useful feedback on this session's capabilities provisioning. This happens on every run (the output below are the defaults):

[INFO  - 2013-05-10T14:33:43.796Z] Session [9dbc5700-b97e-11e2-8dc9-976d2e8732bf] - 
CONSTRUCTOR - Desired Capabilities:{
  "browserName" : "PhantomJS",
  "version" : "9",
  "platform" : "ANY",
  "browserVersion" : "9",
  "browser" : "firefox",
  "name" : "Codeception Test",
  "deviceOrientation" : "portrait",
  "deviceType" : "tablet",
  "selenium-version" : "2.31.0"
}
[INFO  - 2013-05-10T14:33:43.796Z] Session [9dbc5700-b97e-11e2-8dc9-976d2e8732bf] - 
CONSTRUCTOR - Negotiated Capabilities: {
  "browserName" : "phantomjs",
  "version" : "1.9.0",
  "driverName" : "ghostdriver",
  "driverVersion" : "1.0.3",
  "platform" : "mac-10.7 (Lion)-32bit",
  "javascriptEnabled" : true,
  "takesScreenshot" : true,
  "handlesAlerts" : false,
  "databaseEnabled" : false,
  "locationContextEnabled" : false,
  "applicationCacheEnabled" : false,
  "browserConnectionEnabled" : false,
  "cssSelectorsEnabled" : true,
  "webStorageEnabled" : false,
  "rotatable" : false,
  "acceptSslCerts" : false,
  "nativeEvents" : true,
  "proxy" : {
    "proxyType" : "direct"
  }
}
[INFO  - 2013-05-10T14:33:43.796Z] SessionManagerReqHand - _postNewSessionCommand - 
New Session Created: 9dbc5700-b97e-11e2-8dc9-976d2e8732bf

You can adjust these capabilities in your acceptance.suite.yml file like so:

modules:
   enabled: [Selenium2]
   config:
      Selenium2:
         url: 'http://localhost/'
         browser: phantomjs
         capabilities:
             webStorageEnabled: true

I have no idea if capabilities from the larger list of Selenium DesiredCapabilities that are not on the list you see reported from the driver are enabled for PhantomJS.

Headless testing can be a bit of a challenge, since it's impossible to 'see' what failed. But in this case, Codeceptions default logging and screenshot capture on failure can be extremely helpful, since you can then actually see the state of the browser at the point of failure.

There's quite a bit more that you can do with PhantomJS. The CasperJS project makes good use of the PhantomJS API and if you already have NodeJS installed it's a quick and easy way to play with some of the PhantomJS capabilities. CapserJS, aside from requiring NodeJS, only drives PhantomJS. So its not a reasonable alternative to Codeception. Maybe at some point there will be a native Mink driver for the PhantomJS API which will more fully exploit it.

Happy testing.

Testing WordPress Plugins with Codeception. Part 1

$
0
0

WordPress has dozens of plugins. And probably you have developed your own. How would you know your plugin works for everyone? Does it conflicts with others? Does it work after your last change?

Oh yes, releasing the plugin means to make it reliable. This why we highly recommend to use automated testing in your development. Its not that really hard as you expect. Quite the contrary: automated testing saves your time for clicking and filling forms on a page.

In this post we will examine how can you test a sample WordPress plugin with Codeception framework. Our patient is User Submitted Posts plugin by Jeffrey Starr.

This plugin allows regular users to submit a posts to your WordPress blog.

We assume a plugin is already installed and activated.

Plugins Activated

And we created "Submit a Post" page with a plugin snippet included on it. Through this page we expect user stories to be submitted.

Submit Post

This plugin has lots of settings that include post fields, available categories, security issues, etc. What we want to do is try to switch different options and try to submit a post. Depending on option switched we will see some changes in the form, moderation or publishing process.

Automation of testing allows us to write a script one time and replay it whenever a change is introduced. Will the plugin work when WordPress hits new release? Will the plugin work when we introduce a new option? Without automated testing we would spend hours to check the same stuff over and over. Lets spend a few hours writing a proper tests, and save days on manual testing.

Install Codeception

Codeception can be easily installed. It requires PHP 5.3 and higher. You can download it here or grab from site with wget:

wget http://codeception.com/codecept.phar

Downloaded codecept.phar file should be placed into your WordPress directory. This file should be executed with PHP from console

Console

To install it run the bootstrap command:

php codecept.phar bootstrap

This creates a new directory tests with different folders in it.

Tests

There are 3 different folders (called suites) to store tests: acceptance, functional, and unit. For our purposes we will need only the acceptance suite. Acceptance tests will replay our actions in browser.

To speed up the tests we are not using real browser here, but we use its emulator, a so-called PHPBrowser, based on curl calls.

First Test

Let's create a test named "SubmitPost". We will need generate:cept command for that.

php codecept.phar generate:cept acceptance SubmitPost

A newly generated test will look like this:

 
<?php
$I = new WebGuy($scenario);
$I->wantTo('perform actions and see result');

?>

This PHP code is written in the way you would describe your actions while you are testing a plugin. What we will actually try to check? Let's define a scenario:

  • enter a site
  • go to "submit posts" page
  • fill all fields
  • submit a post
  • check a message for valid submission is shown
  • admin should see the post
  • and publish it
  • post should be seen on site

Lets write the first step into a test.

 
<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');

?>

We moved to the front page of a site:

WordPress

To move to the "submit posts" page (our step #2) we will need to click on "Submit a Post" menu item:

 
<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');

?>

That's right, everything is as simple as you would tell it to a mate. As we are on page submission page we got a form and a few fields to fill in.

Submit

We are going to publish the review for the "Game of Drones" book into related category. Follow the code:

 
<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','http://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('Post Category', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
?>

After a post is submitted a message Success! Thank you for your submission. is shown.

Sent

Our test wouldn't be a real test without at least one verification. The result of our current actions is this "Success" message. If we don't see it on a screen, we assume that test failed.

 
<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','http://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('Post Category', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');
?>

Execute Test

As we have one assertion with the see command in the end, it's a good idea to try this test in action. As it was said, we will execute this test with the PHPBrowser, i.e. browser emulator. We should point it to the right URL to access our site. For such parameters a config file acceptance.suite.yml exists.

Config

We need to start a web server and specify proper local url of wordpress site with a plugin we are testing. We are ready to execute this test with run command.

php codecept.phar run

And ups...

TestFailed

Looks like everything were ok, before we tried to select option Post Category. We saw in previous screen, there was such field in a list, but why WebGuy couldn't find it on a page? It was there.

Well, here is the answer. WebGuy tries to fetch a form element by its label, whenever a label tag has attribute for that points to the id of a field , we can select this field by label.

SelectField

But as you see, unlike the previous Post Tags field, the Post Category select tag doesn't have an id. Thus, the label do not point to it, as we would expect.

How do we resolve this situation? If we can't match the field by its label, we can probably match it by CSS, pretty similar to how we would do that in jQuery:

 
<?php
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
?>

Let's execute the test once again... And we see it has passed.

Passed

Great, still we only verified the "Success" message was shown. We didn't check that admin can actually see the user submitted post. This is where we start the next part of this tutorial.

The final test code we got today is:

 
<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','http://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');
?>

As you see it is pretty straight forward. And there is nothing hard in writing such tests. But execution of this test took much less then we would reproduce this steps manually, in a browser. Also we can replay this test in any time. Isn't that a good reason to start testing today?

In Next Series:

  • We will finish the test to verify that admin can publish user submitted post.
  • We will rework the test make it even more compact and readable.
  • We will learn how to deal with data in Codeception.

Before we proceed, try to reproduce the following steps locally and prepare the testing environment. You can download this demo project or clone it from GitHub. And try to write and run some tests on your own.

Nothing is Untestable: AspectMock in Action

$
0
0

“Nothing is True, Everything is Permitted” William S. Burroughs

We already announced AspectMock, the mocking framework that may dramatically change the way you do testing in PHP. In this video this Jeffrey Way shows how AspectMock is different from others. In this post we will demonstrate its powers too, and we will try to break some stereotypes about PHP testing.

To get the code tested, you should always keep in mind how you would write a test for it. We know unit testing requires some good practices to follow and bad practices to avoid.

For example, you should not use singletons. They are bad. Why? Code that use singletons can't be tested.

But what if we could mock singletons:

<?php
function testSingleton()
{
    $class = MySingleton::getInstance();
    $this->assertInstanceOf('MySingleton', $class);
    test::double('MySingleton', ['getInstance' => new DOMDocument]);
    $this->assertInstanceOf('DOMDocument', $class);
}
?>

And with AspectMock we really do it - the test is passing. Then should we still consider a singleton to be a bad practice?

Beyond Good and Evil

Classes and methods in PHP are declared statically and can't be changed in runtime. This can be treated as language limitation. Dependency Injection pattern can be used as a workaround for this limitation and widely used for testing. AspectMock breaks the limitation. The same can probably be achieved with Runkit extension. But AspectMock doesn't require you to install additional extensions, and uses only PHP methods to do its job.

"Testability" should not be used as argument deciding what design pattern is right to use and what is not. When you develop with PHP you should always rely on common sense only. Production code should be efficient, fast, readable, and maintainable. The tests should not introduce redundant abstractions to the production code.

Real World Experience With Yii2

Let's get hands on AspectMock. We will use a demo application from the upcoming Yii2 framework. Despite having dependency injection container, Yii2 does not use it for models. It relies on static calls to global Yii class.

Take a look into LoginForm model of advanced application from the Yii2 repo.

Here is the source code:

<?php
namespace common\models;

use Yii;
use yii\base\Model;

class LoginForm extends Model
{
    public $username;
    public $password;
    public $rememberMe = true;

    public function rules()
    {
        return array(
            // username and password are both required
            array('username, password', 'required'),
            // password is validated by validatePassword()
            array('password', 'validatePassword'),
            // rememberMe must be a boolean value
            array('rememberMe', 'boolean'),
        );
    }

    public function validatePassword()
    {
        $user = User::findByUsername($this->username);
        if (!$user || !$user->validatePassword($this->password)) {
            $this->addError('password', 'Incorrect username or password.');
        }
    }

    public function login()
    {
        if ($this->validate()) {
            $user = User::findByUsername($this->username);
            Yii::$app->user->login($user, $this->rememberMe ? 3600*24*30 : 0);
            return true;
        } else {
            return false;
        }
    }
}
?>

As you see, it can't be tested in classical unit testing. The only option we have here is to write an integration test for this class. But with AspectMock we can easily get this class tested with 100% code coverage.

Let's test successful and unsuccessful login scenarios in LoginForm.

LoginForm relies on User class. That's why to write a test, we will mock some of its methods. We will create a mock with test::double call. In a second argument we are passing the methods that are goint to be replaced and the values they should return.

<?php
    public function setUp()
    {
        test::double('common\models\User', [
            'findByUsername' => new User,
            'getId' => 1,
        ]);

    }
?>    

With this User::findByUsername() will always return an empty User instance. And user id will always be 1. For user to log in we need that $user->validatePassword() returned true. We will mock that call in a test.

<?php
public function testCanLoginWhenValid()
{
    $user = test::double('common\models\User', ['validatePassword' => true]);

    $model = new LoginForm();
    $model->username = 'davert';
    $model->password = '123456';

    $this->assertTrue($model->login());
    $user->verifyInvoked('findByUsername',['davert']);
    $user->verifyInvoked('validatePassword',['123456']);
}
?>    

Additionally we did a check that validatePassword method was called, and user was found by findByUsername call. In production environment, this methods would use the database.

The same way we can check that user can't log in with invalid password:

<?php
public function testCantLoginWhenInvalid()
{
    $user = test::double('common\models\User', ['validatePassword' => false]);

    $model = new LoginForm();
    $model->username = 'davert';
    $model->password = '123456';

    $this->assertFalse($model->login());
    $user->verifyInvoked('findByUsername',['davert']);
    $user->verifyInvoked('validatePassword',['123456']);
}
?>    

And in the end we will also check that user can't be logged in without a password.

<?php
public function testCantLoginWithoutPassword()
{
    test::double('common\models\User', ['validatePassword' => true]);
    $model = new LoginForm();
    $model->username = 'davert';
    $this->assertFalse($model->login());
    $model->password = '123456';
    $this->assertTrue($model->login());
}    
?>

If we execute this tests with Codeception we will see all them pass successfully:

passed

If you want to see this with your own eyes, clone this application from Github and run Codeception tests:

php vendor/bin/codecept run

Pay attention to tests/_bootstrap.php file where AspectMock Kernel is initialized. Yii autoloader was loaded through AspectKernel as well. That is important to point AspectMock to a custom autoloader if you do not rely on Composer's autoloader entirely.

How it Works

There are no magical meadows and mighty unicorns in a hat. Still AspectMock uses something really powerful to break the rules. You may have heard of Aspect Oriented Programming. Go AOP framework, developed by @lisachenko does awesome job to bring the AOP to PHP world. It intercepts all method calls and allows to put your own advices for them. The AspectMock is just an advice on top of Go Aop.

Go Aop scnans all libraries and enhances include and require statements with PHP filters. Go adds a parent proxy class to any loaded PHP class on the fly. So If we get back to Yii2 example, User::findByUsername call will invoke that method on a proxy class.

Conclusions

AspectMock still considered to be an experimental project. But it has a wide potential. It is very simple and easy to use. It has very tiny api easy to remember and understand. That's why tests developed with AspectMock are very clean and readable.

AspectMock is not a testing tool for the bad code. The good code is efficient code. WordPress is much popular then any PHP framework, because of its efficiency. Magento does not have unit tests (only integration), but is the most popular ecommerce platform. We can't say how many there are unit tests in Facebook, but we can bet, it started without unit tests. Code should do its job. Code should be readable and maintanable. Overusing dependency injection does not make the code more efficient in any sense. By the way, in Ruby dependency injection is not widely used, but as you may know ruby developers are very passionate about testing.

AspectMock is not a tool for newbies who just didn't manage to learn the good practices. It is advanced tool, that require you to set dependencies explicitly in a test. That may require deep knowledge on internals of framework you use.

You can try it on your own project. If you have code parts that can't be unit tested in classical manner, then AspectMock can do a job for you.


Testing WordPress Plugins with Codeception. Part 2

$
0
0

In previous part of this tutorial we installed Codeception and got a simple test for User Submitted Posts plugin. We tested that user can send a post and see a message that the post was successfully sent. Nothing more.

Let's remind the test SubmitPostCept we done previously:

<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','http://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');
?>

But well, we didn't verify that the admin has received this post. And it can be published after a moderation. Thus, we will require few more steps to make a test complete. We will need to login to WordPress admin dashboard.

Dashboard

Test commands which allow us to do that are pretty obvious. To keep the code listing shorter we won't show the code from previous lesson. But you should understand that we just append new commands to the previous steps.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
?>

Then we go to Posts section to get all the post listed. We expect to see the "Game of Drones" post in the list.

Posts

Let's also check that it was not published by default and it is in Pending state. Game of Drones Review - Pending should be found inside the table html tag, right? We can even specify the CSS class .posts for that table.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
?>

That's right, the see command we use for assertions (and the test will fail if assertions fail) has a second parameter which allow us to narrow the results from the whole page, to a particular area on a page, which we can define by CSS:

Posts CSS

What's left? We need to review and publish this post, right? The result we expect... Well, we will trust the notifications once again, and message Post published should be enough for us.

Published

And that's how we will do that in test.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review','table.posts');
$I->click('Publish');
$I->see('Post published');
?>

And that would be a good point to execute tests once again. Probably you remember, that you need execute codecept.phar with run parameter from console.

php codecept.phar run

What we are seeing now? Oh no. Just another fail.

Fail

What could go wrong? Hopefully Codeception gives us a suggestion to look for the complete HTML code in _log directory. This directory, actually tests/_log, was meant to store all data related to test execution. We can see there two log files, and the file which we actually need: SubmitPostCept.php.page.fail.html.

Log

This file, named after our test name, stores the HTML that was on page, before the fail. Let's open it in a browser to get a clue what might go wrong here.

Publish

It looks like we are still on the Edit Post page we were before. And the post status is still Pending Review. It looks like click('Publish') command didn't do its job.

Take a look on screenshot. as you can see, word Publish occur several times on a page. We can assume that we were clicking wrong one. What should we do in this case? We can specify the exact that blue Publish button with CSS.

Codeception can use CSS instead of names for clicking the elements. The button have id=publish attribute, thus, it can be found by #publish selector. That what we would probably do if we were using jQuery.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review');
$I->see('Edit Post');
$I->click('#publish'); // id of "Publish" button
$I->see('Post published');
?>

And that fixes the test. You can check that by executing test once again. Let's not trust the notification test and check that a post really is on site. We can click View Post link right near the Post updated message.

Page

What we will check that there is site motto on a page (thus we know, we see the blog theme), and that post title is shown in .entry-title class. And yep, we see this story is "epic and amazing". Let's face the final and complete code of our test:

<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','http://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');

$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review');
$I->see('Edit Post');
$I->click('#publish');
$I->see('Post published');
$I->click('View Post');
$I->see('Just another WordPress site');
$I->see('Game of Drones Review','.entry-title');
$I->see('This story is epic and characters are amazing.');
?>

Well, we did a good job. But the test is like too long. And we can't understand what was going on here. We can add comments to the code, and Codeception has some valuable helpers to add extra text informations with expect and amGoingTo commands.

<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');

$I->amGoingTo('submit a post as a regular user');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','http://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');

$I->amGoingTo('log in as admin');
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');

$I->expect('submitted post was added to a list');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review');

$I->amGoingTo('publish this post');
$I->see('Edit Post');
$I->click('#publish');
$I->see('Post published');

$I->expect('post is available on blog');
$I->click('View Post');
$I->see('Just another WordPress site');
$I->see('Game of Drones Review','.entry-title');
$I->see('This story is epic and characters are amazing.');
?>

We divided our test scenario into logical parts. At least we have left some notice about what is going on and what we were going to achieve. If we execute test with --steps option, we will get a output with all passed steps listed:

php codecept.phar run --steps

Final

Our comments were added to list of passed steps, thus we can easily understand what was going on. And that's quite enough for today. Our test is pretty mature, and covers not only plugin functionality, but WordPress core functions too. Whenever this plugin gets updated, this test should pass to ensure we did everything right.

But the only thing left. After we executed tests several times, we got this picture on Posts screen.

Previous Posts

We have attack of clones here. Yep, each test created its own post and published it. Probably, it is not a good idea to pollute the blog with dozens of posts with similar names. We use post title and content in a test, so probably we can't be sure, what post we are dealing with: current one or the post from a previous test.

That will lead us to the idea of data cleanup. Probably we should delete post after it was published, to revert all our changes. Alternatively we can install WordPress Reset Plugin to reset WordPress to its initial state. In both cases we will need to append some steps to our test to get data cleaned before the next test.


Let's make this your home task. To not start with scratch, you can download code of this tutorial or clone it on Github.

In next lesson we will try to refactor this test and get a few more of them. Don't worry, they will be much shorter than this one. See you soon!

Codeception 1.6.5: Debug and Bugfixes

$
0
0

This is a minor release, mostly done to fix some bugs, you have encountered. Please, submit your Pull Requests for the bugs critical of yours. Most of pull requests are accepted, but if you will start a proposal, we can recommend you the best way to implement the fix.

Selenium2 Compatiblity

Selenium2 server v.2.34 was released recently and to use it you need to update Codeception. If you use phar version, you should replace your old codecept.phar with new one.

Debug Levels

At least one useful feature we prepared for you. Debug output in PhpBrowser and REST modules was extended with additional information that will be printed in debug mode:

debug

Tests would be much easier to debug when you see reponse headers, status codes and client cookies. Don't forget to add --debug to run your PhpBrowser acceptance tests, option to see that.

Title Actions

Two basic yet useful actions were added to all web interaction modules.

<?php
$I->seeInTitle('My Blog | My Post #1');
$I->dontSeeInTitle('Her Blog');
?>

Should be useful, right?

Single Test

Finally you can now execute a single test from Cest or Test testcases.

php codecept.phar run tests/unit/UserModelTest.php:testSave

In this case we will execute only testSave test out of UserModelTest TestCase. The same works for Cests. You may write only the beginning of test name, to execute it.

Bugfixes

  • fix to correct displaying of non-latin characters in html-report by shofel
  • --xml output for Codeception\TestCase\Test fixed
  • fixed unserialize error during code coverage. Anyway, if you ever seen this, you didn't setup coverage correctly.
  • Interactive console console command does not boot with error stacktrace.
  • Clearing only tables and not views in Db->cleanup()
  • PDO $dbh is now passed to Db module corretcly #414

Release Plan

Also we are planning to get more stable releases, and follow the Semantic Versioning. This means that current stable branch is 1.6. If you submit patches and bugfixes, you should propose them into 1.6 branch. Experimental features should go to master.

release branch status
Stable 1.6 Build Status Latest Stable
Development master Build Status Dependencies Status

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Codeception 1.6.6: Sequences

$
0
0

A minor release with one major announcement. In 1.6.6 most bugfixes were included from pull requests. Thanks for everyone who did the contributions.

Sequence

Also a very tiny new module was included. It's Sequence which was created to generate unique data for your tests. Sequnces become pretty handy if you don't do database cleanup before each tests. They guarantee you can get unique names and values for each test.

<?php
$I = new WebGuy\PostSteps($scenario);
$I->wantTo('create a post');
$I->createPost('Post'.sq(1), 'Lorem Ipsum');
$I->see('Post created sucessfully');
$I->see('Post'.sq(1), '#posts');
?>

No matter how much times you execute this test, each time you see a new post is created with different name.

Bugfixes

  • Remote codecoverage now works with Selenium2 module. Please update c3.php file to use it.
  • IoC container access in Laravel4 module by @allmyitjason.
  • PostgreSQL driver fixes by @mohangk and @korotovsky .
  • don't rollback for inactive transaction in Dbh module by @sillylogger
  • fix to guy classes generation with namespaces by @vinu.
  • SQLite improvements by @piccagliani

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Living on the Edge: WebDriver

In July a group of Facebook developers set the goal to write the complete new Selenium Webdriver bindings. They decided to do it finally the right way, with the same WebDriver interface it is used in other languages like Java and C#. Ironically, Selenium2 module of Codeception uses the old webdriver bindings from Facebook.They were very hard in use, and had lots of issues. Most common issues were solved in Element34 fork, which was then forked by Instaclick to bring namespaces and PSR-0, which was then used in Mink's Selenium2Driver, and Mink was used in Codeception.

Pretty tricky, right?

Currently there are 3 WebDriver bindings in PHP.

  • Selenium2TestCase of PHPUnit which is the most old, the most complete and the most OOP webdriver implementation. But if you have worked with its api, you understand how robust it is to learn and use.
  • Element34 fork based on initial facebook/webdriver bindings, but with hooks to solve common pitfalls.
  • Selenium2Driver of Mink which incorporates Element34 bindings and Syn.js library to perform most of interactions via JavaScript bypassing WebDriver API.

Ok. Now we have new facebook webdriver bindings. They are in active development and they lack proper documentation. But the good part of it, that even without documentation you will easily learn how to work with them. Any question on StackOverflow with examples in Java or C# will work in PHP just the same way.

In Codeception master we created a new WebDriver module which uses new webdriver bindings. This module will be included into first 1.7 release, but it won't be recommended for regular use before the stable version of php-webdriver is released.

To try the new WebDriver you should switch to dev-master in composer.json or use the pre-prelease phar package.

WebDriver module does not implement everyhting the Selenium2 module has. There is no right clicks, drag and drops, and more. But there are few handy improvements:

  • submitForm action as in PhpBrowser.
  • waitForElement action to wait for element to appear on page.
  • selectOption and checkOption now both work with radio buttons.
  • seeElementInDOM to check the invisible elements.
  • waitForElementChange to wait that element changed
  • implicit waits with wait config prameter (default is 5 secs).
  • maximizeWindow specially for @aditya- :).

In all other ways its pretty compatible with Selenium2 module. Try it on your own risk or wait for stable versions.

Understanding AspectMock

$
0
0

As you may know, AspectMock is non-ordinary mocking framework that can override any method of any class in your application. This is practically useful If you want to unit test code, which was not aimed to be testable from start. Also AspectMock gives you power to write efficient code at first, and not affect production code with testing design.

Test Design

Even AspectMock proposes a flexibility in testing, it doesn't drive you into bad application design. If you use classes globally (without injecting them) or you use static properties, methods, or singletones, it's all right while they are defined as your internal API. Such API methods should be well documented, especially for cases, where they should be used, and where not.

If we use ActiveRecord pattern, we can assume that all models are inherited from ActiveRecord class. The only point in which our models is accessing database is save method of that class. Thus, we need only to block its call, If we don't want the database to be hit.

<?php
test::double('ActiveRecord', ['save' => false]);
$user = new User(['name' => 'davert']);
$user->save(); // false
?>

Sure, integration testing using database gives us more reliable results. And no one ignores that fact. But unit tests allows to cover more cases, without implementing and loading fixtures. They are much faster too.

Features and Drawbacks

AspectMock may sounds cool for you, but you feel that there should be pitfalls. Let's be honest, and list all of them here.

  • The most common issue is to get AspectMock installed. We won't list different configuration options here, they are well documented in Github readme. But the idea is pretty simple: you should include directories with files expected to be mocked. If you don't rely completely on autoloader from composer, you should include your autoloaders too.
  • AspectMock will slow down execution in about 20%. That's because all methods of all classes are intercepted.

You may be curious If AspectMock affect the stack traces? The answer is no. AspectMock (starting from 0.4) does not change the line order in mocked classes, thus you get truth worthy information in stack trace. Sure, AspectMock changes those files, a bit, but more about that in next section.

Can I debug my tests when using AspectMock? And here are the good news: Sure, you can! In Debug mode you will see your classes, with no mock including in them.

To summarize: AspectMock has no side effects on unit testing process. Its magic is properly hidden to not affect the development.

Dark Magic Inside

Before implementing AspectMock into your project you might want to know, how it works in details. AspectMock is powered by Go Aop Framework.

Go AOP Framework uses php stream wrappers with filter to parse PHP files before requiring them. That may even happen in runtime. Thus, by analyzing file, we can find all its methods, and inject mocking code into it. To do so, all requiresshould include a filter. This will look like:

<?php
require 'myfile.php';
?>

will be replaced with

<?php
require 'php://read=go.source.transforming.loader/resource=myfile.php';
?>

That will make PHP file to be parsed before loading, and changed on the fly. Go AOP is pretty smart to cache already parsed files.

Now time comes for AspectMock. For every method of every class, AspectMock inserts one line into very beginning of method definition. This sample class

<?php
class User {
    
    function setName($name)
    {
        $this->name = $name;
    }    

}
?>

will be replaced with:

<?php
class User {
    
    function setName($name)
    { if (($__am_res = __amock_before($this, __CLASS__, __FUNCTION__, array($name), false)) !== __AM_CONTINUE__) return $__am_res; 
        $this->name = $name;
    }    

}
?>

As you see only one line is added. If a stub was registered for this method, its result will be returned, and method itself won't be invoked.

If you will enter into class in debug mode, you won't see the line injected by AspectMock. But you will notice its there, even not show, in step by step debug.

And that's probably all the dark magic you should be aware of. Probably it's not too tricky and you will get along with it.

Conclusion

Unit tests are important part of testing pyramid. They are fast and they are flexible. You should not ignore them just because it may be hard for you to implement them. Its not that hard anymore. With AspectMock you can get a good code coverage with less efforts for any kind of modern php application.

Codeception 1.6.7

$
0
0

This release is minor, yet fixes lots of bugs you might have encountered. If you didn't encounter them, well, then, lucky you. Still it's a good idea to upgrade. Maybe you will encounter some bugs after that :)

Ok, let's turn off the irony mode, and post a regular list of changes.

  • fix to 80 port issue, if you had problems connecting to server via PhpBrowser, then this must be it. Thanks to @tiger-seo.
  • fix in REST module when using application/json header and passing parameters.
  • seeJsonResponseContains of REST can now search on all nesting levels.
  • fix to Sequence module.
  • Step class code improved by guilhermeFranco
  • Using suite with defined namespace was improved by @Borales.
  • Generators fixes by @piccagliani and davert.

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update
Viewing all 321 articles
Browse latest View live




Latest Images