Quantcast
Channel: Codeception
Viewing all 321 articles
Browse latest View live

Protips 2 - Extending Modules

$
0
0

Today, we will continue covering advanced topics on Codeception. The main power of Codeception is it's modules, there are already plenty of them, even though you might want to extend one of them. Let's say you are using Symfony2 and you want to add additional actions, you actually miss in current module, like dealing with FOSUserBundle: registering, authenticating, etc. You can create a helper and connect with Symfony2 module, get access to dependency injection container and do anything you need there. Sometimes you might want to change the module initialization scripts. This happens if your application configured in a custom way and module doesn't work properly because of it.

In both cases you might want to improve current Symfony2 (or any other module) implementation. That's pretty easy, because we can use just the simple OOP inheritance. You can create your own module Symfony2Helper and inherit it from default Symfony2 module. This module will act as a regular Helper, and should be placed to tests/_helpers dir.

In the next example we will redefine initialization for working with Symfony2 applications located in app/frontend path with Kernel class located in app.

<?php
class Symfony2Helper extends \Codeception\Module\Symfony2
{
    // overriding standard initialization
    public function _initialize() {
        // bootstrap
        require_once getcwd().'app/frontend/bootstrap.php.cache';
        // kernel class
        require_once  getcwd().'/FrontendKernel.php';

        $this->kernel = new FrontendKernel('test', true);
        $this->kernel->boot();

        $dispatcher = $this->kernel->getContainer()->get('event_dispatcher');
        $dispatcher->addListener('kernel.exception', function ($event) {
            throw $event->getException();
        });
    }    
}
?>

Please, refer to the _initialize method implementation to understand the default behavior. To get the idea whether you need to inherit and redeclare methods of module, you need to review it's code. If you see your requirements can't be met by using config options, please override.

By using inheriteance you can redeclare all the methods or initialization hooks you don't like. The API of parent module is pretty clean and even when you use the phar version of Codeception, you can read it's code on GitHub. If you stuck because Codeception module works improperly, you are free to fix that.

And yep, if your improved module is worth sharing with community, submit a patch and it will be added to official distribution. After all, that's how the Open Source works.

Thanks for reading. Use Codeception professionally.


Stop Simulating BDD, Write Tests

$
0
0

I keep receiving one question. Well, actually two: "What makes Codeception different comparing to Behat". And the next one: "Why can't I write Codeception tests in human language like Behat does". I think the time has come to answer this questions.

As you know, the Behat project is not only a tool but a methodology. The Behavior Driven Development concept requires that the User Stories to be written, and then executed as a test. Behat is a right tool for teams who strictly follow the BDD methodology. But what if you don't? For what in the world you need tests written in plain English?

If you write a story like this, using Behat and MinkContext for example,

Given I am on "/product/1"
When I click "Purchase"
And I fill in "name" with "Michael"
And I fill in "credit" with "321123123"
And I click "Order"
Then I should see "The product was purchased!"

you actually write test, and not a story. Because the feature you describe won't change If you rename field "credit" to "credit card", but the test will fail. You may change the text "The product was purchased" to "iPhone5 was purchased. Thank you!" and that will make test fail too. So you will rewrite the user story every time you change every detail that may affect passing test. In this case the manager who wrote this spec will debug and update user story to make the test pass.

And so your manager becomes tester. His role is now not only to deliver specifications to team but also to make this specifications written as tests and make them pass.

But what If you want one additional step to the scenario?

Given I am "/product/1"
When I click "Purchase"
Then I should see "1 product" in the bin

For this step see "1 product" in the bin you should write a custom step. If only your manager knew PHP for it! Ideally to become manager + tester + PHP guru. But no, manager needs to ask your developer to write the necessary step. Thus, already two people are writing the test. And one of them knows PHP. And when you will hire a QA will you require a knowledge of PHP? Well, yep. Thus because you don't want to disturb your PHP devs and assign them additional tasks.

When your product growths your tests becomes more and more complex. And you need to keep them updated for each change. When you use the plain English text, you don't have a control over tests. When your form don't have a label you need to write custom step instead of I fill in, because it's actually wrong to use CSS or XPath selectors inside a Behat feature. It's not plain English text anymore with CSS. You need PHP developers to create additional steps.

Why Codeception is better testing tool? Well, just because it is a testing tool at first. Spend a day, to teach your manager/tester/qa PHP DSL, install a Netbeans or PHPStorm (or any other IDE), and start writing tests. Even when tests looks similarly to Behat features they give more power and control over the process.

<?php
$I = new WebGuy($scenario);
$I->amOnPage('/product/1');
$I->click('Purchase');
$I->fillField('name', 'Michael');
$I->fillField('credit', '321123123');
$I->click('Order');
$I->see('The product was purchased!');

This test in the process of evolution can be transformed into something more complex. You can use CSS and XPath everywhere, so when you change the "Order" button with an image, you can use it's CSS and quickly update a test to make it pass. Also, you can move repeatable elements into variables and classes and make a test that looks like this:

<?php
$I = new WebGuy($scenario);
$I->amOnPage(ProductPage::URL);
$I->click('#purchase');
$I->fillField('name', $user_name);
$I->fillField('credit', $user_credit);
$I->click('img a#order');
$I->see(ProductPage::$successMessage);

Just the same scenario. Well it's a bit harder to read, as we added CSS selectors, but as you see this test pretty much more flexible. It can be easily refactored and improved without writing any custom method to helper.

Don't fall into a marketing trap. You will find yourself writing tests two times: as a feature in plain English and in code with PHP.

It's better to have one person who can take a full control over test automation then, delegate some tasks to developers, some tasks to managers. If you are developer and you work on your own project. Why PHP is not suitable for you? Why do you want to write code 2 times: once in feature with Gherkin and second time in Context. Is it KISS? Is it DRY? I don't think so.

Will Codeception support plain text test scenarios? Really, I thought on that. But I don't want to put the limits. Using plain text makes impossible to use variables, loops, grabbers, and thus use PageObject pattern. Which is very important for solid and concrete test automation platform.

Behat is great BDD tool and it's author @everzet also states: don't use it for functional testing. Choose the right tool for job.

Disclaimer: I'm not very good in theory, I'm very practical guy. I won't refer you to any published books or methodologies. But If you want an authority... Well, David Heinemeier Hansson (the creator of Rails) wrote in his blog: 1"Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests". And Behat is Cucumber for PHP. DHH is known for his dislike to popular BDD tools like RSpec or Cucumber. But he is very pragmatic and very scrupulous in testing.

Codeception released with CodeCoverage support

$
0
0

We'd like to announce new Codeception 1.5 major release. This time our improvements are CodeCoverage and Remote CodeCoverage added. These features allows you to collect code coverage reports for all your tests: unit, functional, and acceptance and merge them together. So now you can review what parts of your applications are tested and which are not even for Selenium and PhpBrowser tests which are executed on a webserver.

Read more in the new Guides chapter.

There is no magic in local codecoverage. XDebug and PHP_CodeCoverage libraries do their job. The tricky thing is remote codecoverage. We attach small script into application's front controller. When a special header is sent this script starts to collect coverage information. And in the end of tests, this data is merged, serialized and sent back to Codeception. So you can test and collect coverage report even on staging servers in real environment.

code coverage

Thanks to tiger-seo for codecoverage feature. He did a great job developing a remote script c3.php which is a unique in it's way.

But back to Codeception. As you may've noticed our website is updated too. Documentation was improved, search was added, and nice Quickstart page added too. So if you didn't try Codeception yet, it's very easy to start now. In only 6 steps.

Modules

Two useful modules were introduced by judgedim. We have support of MongoDb and Redis now. They both can clean up your storages between tests and Mongo can perform checks in your collections. So If you are working with NoSQL databases, you should try them in your tests. If you use different NoSQL databases, please submit your patches, and they will be included in next release.

UX Improvements

Now you can execute a test by providing relative path to test, like.

php codecept.phar tests/acceptance/SignInCept.php

This small tweak imprioves user experience for *nix users as they can use autocompletion when running a test.

Also you can run test from one specific directory, i.e. match a group of tests:

php codecept.phar tests/acceptance/admin

Bugfixes

Composer package is works again. It's really hard to follow the stability in the world of constant changes, so we recommend use of phar for testing, just because it's prepackaged and always runs as expected. But if you use Composer it's easy to add Codeception to your vendors and receive all new updates with new release. Don't forget to mark @stable Codeception version.

Install

As usual, Codeception 1.5.0 can be downloaded from site,

installed via PEAR

$ pear install codeception/Codeception

or via Composer

$ php composer.phar update

Thanks

As you may've noticed all that guys who took part in developing Codeception are now shown on every page of this site. Thn this way we say thank for all our contributors and all guys who support this project, for all companies that adopt Codeception in their workflow.

Codecepion 1.5.1

$
0
0

You know, it's a common situation when just after the brand major release comes a small release with patches and bugfixes. Yeah, Codeception 1.5.1 has nothing to surprise you, still it's reommended to upgrade if you are running the phar vs composer issue. As we discovered with the help of alexshelkov, that Codeception packaged phar was trying to load PHP libraries from a local composer installation. So if you was trying phar version in project that already use composer, you may have seen strange errors.

This bug was fixed and last version 1.5.1, ready for download.

A simple improvement was introduced by DevMan. It allows you to execute Codeception from any directory if you set a path to config file with a -c parameter.

php codecept.phar run -c /var/myproject/codeception.yml

And finally we are back green on Travis. We had some issues with Travis infrastructure (not the tests), but we managed to solve them. Did you know that it takes 8 minutes to run all internal tests of Codeception? Yeah, we've got lots of tests )

Build Status

Update

Please redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

Codecepion 1.5.2

$
0
0

Yet another minor release. It introduces more compact form for unit tests.

<?php
use Codeception\Util\Stub;

class UserRepositoryTest extends Codeception\TestCase\Test
{
    protected function _before()
    {
        $this->user = new User();
    }

    public function testUserIsNotAdmin()
    {
        $this->assertFalse($user->isAdmin());
    }

}
?>

We moved all Codeception's setUp and tearDown routine into parrent class and left you more clean _before and _after methods for your preperations. They act just the same as setUp and tearDown for PHPUnit. So _before is executed before each test and _after after it. Tests should look more clean now. If your tests already have setUp and tearDown methods nothing is changed for them.

Accessing Modules in Unit Tests

Another nice feature for Symfony2 users and not only them. Form now on you can access a module from your unit test. This is done to access public properties of your module. For example, here is how you can retrieve a Symfony Dependency Injection Container:

<?php
protected function _before() 
{
    /**
   * @var $container Symfony\Component\DependencyInjection\Container
   */
    $container = $this->getModule('Symfony2')->container;
    $this->em = $container->get('doctrine');
}
?>

Also we'd like to announce the upcoming modules for Yii and Laravel frameworks.

In the meantime we will release more complete guides on How to integrate framework to Codeception.

Bugfixes

  • Composer build is finally is stable and doesn't require minimal-stability: dev. Thanks to: jacobkiers.
  • Adding support of pecl_http by phpcodemonkey.
  • Methods of Guy received documentation and examples (phar bug).
  • generate:cest command requires a tested class name no more.
  • PostgreSQL driver improved .

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

or via PEAR:

$ pear install codeception/Codeception

Connecting PHP Frameworks. Part 1

$
0
0

As you know, Codeception has various modules for functional testing of PHP applications. Really, you could use Selenium, but as an advice, please leave Selenium for QAs.

As a developer you need to get more technical details for each test. If test fails you need to see the actual stack trace. As a developer you need to have all test running much faster. You can't wait for an hour before pushing your commit.

If you ever had tried Codeception with Symfony2 you should know how fast are functional tests. You should know how helpful is printed information from profiler: queries used, user authentication status, etc.

In case you want to try functional tests with your own framework you should develop module for it. Basically it's not that hard as you think it is. But some frameworks suit better for integrations and some need additional hooks. In this post I will concentrate on integrating one of the modern PHP frameworks that use HttpKernel Symfony Component.

There are Laravel 4 and Drupal 8 that adopted HttpKernel interface. And more frameworks to come...

So in case you want to integrate Laravel or Drupal, or you know that your framework uses HttpKernel, you should follow this guide.

If you are not aware of what HttpKernel class is, please follow Fabien Potencier's' series of posts Create your own framework... on top of the Symfony2 Components

Contributor's Notice

First of all, how do we treat the user contributions? In Codeception we have liberal politics for accepting pull requests. The only thing, we can't test your implementation, as we just don't have experience in this frameworks. So when you commit module please test your module, and prepare to be it's maintainer. So you will need to write proper documentation and provide support it. Answer to questions on GitHub, in Twitter, etc. Yep, this is opensource.

But we will help you with that. The more developers will use your framework the more contributions your module will receive. So try to encourage framework community to test your module, use it and improve.

When your module is complete it will be packaged with Codeception and it's reference will be published on this site. This is done to make Codeception with all it's modules work out of the box from one phar archive.

Technical Implementation

So you decided to create a module for Codeception that provides integration with HttpKernel. Hope you do!

Check how it is done for Symfony2

  1. We load and initialize HttpKernel class in _initialize method.
  2. Before each test we create a HttpKernel\Client class for this kernel in _before method
  3. We shut down client after each test in _after method.

Let's narrow it to example.

<?php
namespace Codeception\Module;
use Symfony\Component\HttpKernel\Client;

class MyFrameworkModule extends \Codeception\Util\Framework {

    public function _initialize()
    {
        // $app implements HttpKernelInterface
        $app = require_once \Codeception\Configuration::projectDir().'/app.php';
        $app->setEnvironment('test');
        $this->kernel = $app;
    }

    public function _before(\Codeception\TestCase $test)
    {
        $this->client = new Client($this->kernel);
        $this->client->followRedirects(true);
    }

    public function _after(\Codeception\TestCase $test)
    {
        $this->kernel->shutdown();       
    }
}
?>

And basically that's all you need for integration. The Client class has everything to simulate requests and parse responses. Every module that extends Codeception\Util\Framework class will have actions: click, see, fillField, defined and documented in Codeception\Util\FrameworkInterface. This actions will work in just the same manner as for other frameworks. And it's really cool, that testing client is not aware of framework it is testing. All methods and their behavior are just the same. So tests for Symfony2, Zend, or your newly integrated frameworks will look just the same.

Still you may want to add something special for your framework. Maybe some additional initialization steps, or new actions. Let's say a framework you integrate have methods to authenticate a user by name. Why not to use this ability and to make a short cut for logging in?

<?php
namespace Codeception\Module;
use Symfony\Component\EventDispatcher\EventDispatcher;
use Symfony\Component\HttpKernel\Client;

class MyFrameworkModule extends \Codeception\Util\Framework {

    public function _initialize()
    {
        // $app implements HttpKernelInterface
        $app = require_once Codeception\Configuration::projectDir().'/app.php';
        $app->setEnvironment('test');
        $this->kernel = $app;
    }

    public function amLoggedAs($username)
    {
        $this->kernel->getService('session')->authenticate($user);

        $role = $this->kernel->getService('session')->getRole();
        $permission = $$this->kernel->getService('session')->getPermissions();

        // let's display additional information
        $this->debugSection('Role', $role);
        $this->debugSection('Permissions', json_encode($permission));
    }
}
?>

So now we can write a test like this:

<?php
$I = new TestGuy($scenario);
$I->amLoggedAs('davert');
$I->amOnPage('/user/profile');
$I->see('Davert');
?>

As you see, framework integration allows us to access it's internals. Services, classes, configurations, etc. Please add methods that you think may be useful for other developers that will write functional tests. Also you can display some technical details with debug and debugSection methods.

Let other modules get access to framework internals too. In our case we can define kernel as public property and make it accessible from user helper classes.

<?php
class MyFrameworkModule extends \Codeception\Util\Framework {
    public $kernel;
}    
?>

In helper:

<?php 
class TestHelper extends Codeception\Module {

    public function doSomeTrickyStuff()
    {
        $kernel = $this->getModule('MyFrameworkModule')->kernel;
        $kernel->doTrickyStuff();
    }
}
?>

In case you want to provide flexibility you can add configurable parameters to your module. Let's update our example to use additional parameters.

<?php
class MyFrameworkModule extends \Codeception\Util\Framework {
    
    // paramter with default var
    protected $config = array('file_name' => 'app.php');

    // a parameter that we can't guess 
    protected $requiredFields = array('app_name');

    public function _initialize()
    {
        // $app implements HttpKernelInterface
        $app = require_once Codeception\Configuration::projectDir().$this->config['file_name'];
        $app->setEnvironment('test');
        $app->init($this->config['app_name']);
        $this->kernel = $app;
    }
}    
?>

Module won't start if app_name parameter is not set. But file_name parameter is optional and can be redefined.

In the end here how should look functional.suite.yml that includes your newly developed module:

class_name: TestGuy
modules:
  enabled: [MyFrameworkModule, TestHelper]
  config:
      MyFrameworkModule:
          app_name: blog

Keep in mind that main principles of all Codeception modules are simple and smart. This means, if you can guess some parameters - do not require them to be explicitly set.

Conclusion

It's pretty simple to integrate any framework if it's modern and built with Symfony Components. If you have various projects using this framework and you want to get it tested well, try to develop integration. It's not that hard: only 3 methods to implement. Share your work with others, and let it be added to main package.

We appreciate all contributions and all frameworks. Let's unite the PHP world!

Codeception 1.5.3

$
0
0

Urgent fix for Selenium2 module. It appeared that Selenium sessions were not restarting. The Travis-CI didn't reveal this problem. Selenium tests were passing as usual, just because there were no session related tasks. Main purpose for 1.5.3 was to release this fix. Some other fixes are also included, but it's the most important one.

Modules

In this release we also introduce very basic implementation of Yii module by Ragazzo. If you use Yii, please try it and help us improve Yii integration. To start using, please review the module documentation and issue description on Github.

Also we've got some improvements for AMQP module. Not only queues are cleaned up between tests, but also you can check if messages came to queue, grab them from queues, or publish new messages. This module was tested with RabbitMQ, but it should work with other AMQP providers too. This module is based on php-amqplib.

Generators

From now on you can generate classical PHPUnit tests, with no Codeception stuff in it. Just the plain old PHPUnit tests that you can run natively with PHPUnit itself. For this we have new command generate:phpunit

Use it as regular:

php codecept.phar generate:phpunit NewTest

Also generators now support namespaces. So when you type class name with namespaced, Codeception will generate directories (according to PSR-0) and will put namespace into your newly generated file.

Check out sample usages.

php codecept.phar generate:cest \\Entity\\User
php codecept.phar generate:test \\Entity\\User
php codecept.phar generate:phpunit unit \Entity\\User
php codecept.phar generate:phpunit unit "\Entity\User"

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Testing Symfony2 Apps with Codeception

$
0
0

From the beginning of it's existence Codeception was in good relations with Symfony2 framework. Codeception was built on Symfony Components and uses BrowserKit and HttpKernel components for launching functional tests. It's a shame we didn't have a complete Symfony2 integration tutorial before. But we will try to fill this gap today.

What benefits you get by using Codeception with Symfony2? Let's list all of them:

  • user-friendly syntax for functional tests
  • access to container in unit tests
  • testing REST and SOAP services built on Symfony
  • fastest data cleanup when using with Doctrine2

The installation is pretty simple. You can use Composer (as you are used to it), but we'd recommend to try phar package. In this case you can avoid unnecessary dependencies. But all this versions are equal. And when you installed Codeception and executed a bootstrap command you should configure your functional test suite.

In file tests/functional.suite.yml:

class_name: TestGuy
modules:
    enabled: [Symfony2, Doctrine2, TestHelper]

And nothing more. You just need to declare that you will be using Symfony2 and Doctrine2. The Symfony2 module will search for AppKrenel on initialization (in app) and use it for functional tests. Doctrine2 module will find that Symfony2 module is declared and will try to receive default database connection from container. Of course, If you use non-standard setup this behavior can be reconfigured.

Functional Testing

Let's create a first functional test. We use generate:cept functional TestName command.

<?php
$I = new TestGuy($scenario);
$I->wantTo('log in to site');
$I->amOnPage('/');
$I->click('Login');
$I->fillField('username', 'admin');
// ....
?>

And so on. Unlike standard Symfony2 tests you don't need to deal with filters, CSS, and XPaths. Well, you can use CSS or XPath in any selector, if you need to. But on start you can keep your test simple and compact.

The commands we use here are common for most modules that perform testing of web application. That means, that once you discover a need for Selenium, this test can be executed inside a web browser using Selenium2 module. But some commands are unique to Symfony2 module. For example, you can use seeEmailIsSent command that checks if application has submitted an email during the last request. Check Symfony2 module reference for all commands we provide.

Unit Testing

Codeception's unit tests are a bit improved PHPUnit's tests. Inside a unit tests you have access to initialized modules and guy classes.

<?php
class PaginateTest extends \Codeception\TestCase\Test
{
    private $serviceContainer;
    protected $codeGuy;

    protected function _before()
    {
        // accessing container
        $this->serviceContainer = $this->getModule('Symfony2')->container;
        $this->serviceContainer->enterScope('request');
    }

    public function testDefaults()
    {
        // using container
        $this->serviceContainer->set('request', new Request(), 'request');
        $paginate = new Paginate($this->getModule('Symfony2')->container);
        $this->assertEquals(1, $paginate->getCurrentPage());
        $this->assertEquals(0, $paginate->getCount());
        $this->assertEquals(20, $paginate->getLimit());

        // checking data in repository
        $this->codeGuy->seeInRepository('SettingsBundle:Settings', ['pager_limit' => 20]);
    }
}    
?>    

If you want to bypass Codeception hooks for PHPUnit you can create a plain PHPUnit test with generate:phpunit command and get a traiditional PHPUnit's test. Then you can put this test into bundle.

Conclusion

Ok, you will probably ask: why is it better then Behat. We have a wide answer for that. A short is: Codeception is for testing, Behat is for Behavior Driven Development. If you need a professional testing tool that supports PageObject pattern, complex Locators, refactoring capabilities and CodeCoverage - Codeception is a good choice for that.

We say thanks to @everzet for wonderful Mink (that is used for acceptance tests) and to Sebastian Bergmann for it's PHPUnit. Codeception uses their powers, but makes them a bit simpler in use.


Codeception 1.5.4. Skip Tests, UTF fixes, etc.

$
0
0

Today we are releasing Codeception 1.5.4. This is minor bugfixing release consisting of GitHub issues. And a small yet awaitied feature introduced: ability to skip and mark Cepts as incomplete.

It's pretty simple, though.

<?php
$scenario->skip();

$I = new WebGuy($scenario);
$I->wanTo('do something, but I would rather not');
$I->amOnPage('/');
//.....
?>

This makes this test to be skipped. Similarly it can be marked as incomplete.

<?php
$scenario->incomplete();

$I = new WebGuy($scenario);
$I->wanTo('do something, but I would rather not');
$I->amOnPage('/');
//.....
?>

The skip and incomplete methods can take one argument, that descrines a reason why this tests were marked this way.

<?php
$scenario->skip('waiting for new markup');
$scenario->incomplete('welcome page appearence not tested');
?>

Bugfixes

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Codeception 1.5.5 and Roadmap Announced

$
0
0

Yes, here is a new version of Codception with more features and bugfixes in it. We have a flexible relase cycle, so the new version comes when we have a set of updates that you might be needed. We want to say "thank you" for all contributors, and for everyone who helps making Codeption better. It's always nice to get pull requests or follow discussions in our issues secition. So let us sum up the work was done during last 2 weeks and share it with you.

Use Codeception in Different Places

All codeception commands got -c option to provide a custom path to tests. The exception is bootstrap command. It accepts a custom directory as a second argument:

php codecept.phar bootstrap ~/mynewproject
php codecept.phar generate:cept acceptance Login -c ~/mynewproject
php codecept.phar generate:test unit MyFirstTest -c ~/mynewproject
php codecept.phar run -c ~/mynewproject
php codecept.phar generate:scenarios acceptance -c ~/mynewproject

Alternatively you may specify path to codeception.yml file in -c option: php codecept.phar run -c ~/mynewproject/codeception.yml

Thus, you don't need to keep your codecept.phar in the root of your project anymore. Use -c option and one local runner for all your projects.

Skipping Tests

Skipping and marking tests of incomplete was improved. We did a new solid implementation for it (it was very basic in 1.5.4). Now If a test is marked to skip, no modules will be touched.

<?php
$scenario->skip('this is not ready yet, move along');

$I = new WebGuy($scenario);
$I->wanTo('do something, but I would rather not');
$I->amOnPage('/');
//.....
?>

This feature required to rework some core classes (like Step and TestCase and Scenario) but hopefully we simplified our code.

Bugfixes

  • acceptPopup with Selenium 2 does not trigger Exception any more
  • error handling was improved to skip blocked with @ alerts, yet to throw ErrorException for notices, warnings, errors.
  • ZombieJS configuration was fixed. Now the url parameter is required to specify app's local url.
  • REST seeStatusCodeIs method works correctly with Symfony2 now.

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Roadmap

For the first time we will announce the roadmap for Codeception. Actually we need your opinion: what features you'd like to see in new releases, and what things can be postponed. The list below is not a precise plan, but just a list of features we have in mind:

  • Make a PageObject pattern first-class citizen in Codeception. Add generators and guides to use PageObjects (for acceptance tests).
  • Multiple sessions for tests execution (see discission on GitHub)
  • Silex, Laravel 4, Zend Framework 2, Drupal 8, Phalcon integrations. The key problem here: we can't do this on our own. We need a real users of these frameworks, to create integration and test it on their projects. We have reworked functional testing guide to help you with this. Also use GitHub or personal contacts if you want to make a module.
  • Scenario Unit Tests to be rethinked. We have 2 options here: dump scenario driven unit tests (or mark them as deprecated) or rework them. Actually we need your real opinion. Here is an example of what new Cests may look like. They will dramatically improve the way you work with mocks and stubs in PHP. But will you use it? Please, let us know what you think.

To summarize: we'd appreciate contributions, feedbacks and ideas for next releases.

Codeception 1.5.6

$
0
0

Codeption 1.5.6 released with varous fixes to Selenium2 module (popups, screenshots) an other bugfixes. We added a bunch of tests for Selenium2 and they are runnng on TravisCI. Also, Codeception is now compatble with Symony 2.2. And few features were intrduced.

Basic Fixtures

Very basic implementation of fixtures was added to Db and Mongo modules. Now you can write something like:

<?php
// for Db
$I->haveInDatabase('users', array('name' => 'davert'));
// for Mongo
$I->haveInCollection('users', array('name' => 'davert'));
?>

to add data for a test. For Db module, the inserted record will be cleaned up on text test, even If cleanup option is set to false. So haveInDatabase method cleans data after itself. Something similar exists for Doctrine2 module. Yet, it looks not like for Db and Mongo implementation:

<?php
$I->persistEntity(new \Entity\User, array('name' => 'Miles'));

Error Handling

From now on you can set custom error handling parameters per suite or globally.

class_name: WebGuy
modules:
    enabled: [PhpBrowser, WebHelper]
error_level: "E_ALL & ~E_STRICT & ~E_DEPRECATED"

Actually, E_ALL & ~E_STRICT & ~E_DEPRECATED is default error level for now. This may be changed in future.

Bugfixes

  • popups and modals in Selenium2 are fully tested and work correctly now
  • screenshots in Selenium2 module is saved correctly
  • seeInField method works correctly even if field is not in form
  • fix to AMQP module to push message to queue
  • Symfony 2.2 compatibility

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Scenario Unit Tests Deprecation Announcement

$
0
0

We decided to get rid of the scenario unit tests. The main reason is: the concept was interesting, but current realization is a bit outdated. It would take to much time to maintain and support them. Hopefully, not to much developers used them anyway ) They are cool if you are dealing with mocks in your tests, but they don't bring any value if you write a plain unit test. We've seen too much improperly written unit tests in Cest format so we'd actually recommend you to switch to classical unittests with codeGuy helpers.

So now recommended are Codeception driven unit tests, in PHPUnit like format:

<?php
class UserTest extends \Codeception\TestCase\Test {

  function _before()
  {
    $this->user = new User();
  }


  function testUserIsSaved()
  {
    $this->user->name = 'miles';
    $this->user->save();
    $this->codeGuy->seeInDatabase('users', array('name' => 'miles'));
  }  
}
?>

If you still interested in scenario driven tests, leave your feedback or even propose your concept. Here is one of them The real idea behind scenario driven unit tests to have code well described in a human language. This is especially useful when dealing with mocks and stubs. In this case we can implement some 'magic' methods that allows us to simplify manipulation with code and write only definitions. What methods do we need, what methods do we test, what we expect to see. No unnescessary technlical details. Only logic and dependencies.

The Cest format itself is not going to be deprecated. It would be proposed to use in functional or acceptance tests as an alternative. If you like to write tests in a class (rather then plain PHP file), please use Cest format. More announcement comes on this soon. Expect a new vewsion.

Scenario driven tests will be removed in 1.7 version of Codeception. So migrate them to regular unit tests.

Update! Codeception 1.5.7 is out.

$
0
0

Great news. Codeception got attention from Nettuts community and we are happy to welcome you in the family ) What you should know about Codeception release cycle is that releases are not breaking the BCs and they appear quite often. Even minor releases introduces lots of new features. And today we'd like to announce some of them.

More Current Url Methods

As were pointed, the seeInCurrentUrl method is not enough to test urls. So we got much more "currentUrl" methods:

<?php
// to match root url
$I->seeCurrentUrlEquals('/');
$I->seeCurrentUrlMatches('~$/user/.*?~')
$url = $I->grabFromCurrentUrl();
$user_id = $I->grabFromCurrentUrl('~/user/(\d+)~');
?>

And yes, we have correspoinding dontSee methods too.

Click on context

Now you can perform clicks in context elements:

<?php
$I->click('Login', '#nav');
$I->click('Login', '//html/body/div/a');
?>

Also you can now try to use Cest classes for acceptance and functional testing. As scenario-driven unit tests are going to be deprecated, Cests will be used as Cepts, but in a format of a class. The Cest generator was updated accrodingly.

Bugfixes

  • Yii module should work as expected as all reauired files are included in package.
  • Fix to Kohana module.
  • $scenario->preload() is back

Announcing: The Platform

Next major release will be called The Platform. In Platform we will introduce lots of a new concepts designed to build a real test automation platform.

  • Extensions - classes that can hook into any events and produce their own. Extend anything.
  • Tags - unite tests into groups and provide a similar hooks for them.
  • Conditional Asserts - don't stop a test if assert fails.
  • PageObjects - new classes for defining UI maps.
  • Controllers - classes to define similar scenario logic.
  • Multiple Sessions - ability to execute same tests with different config (i.e. inside chrome then firefox)

More about PageObjects and Controllers you can read on PHPMaster.

Help from Community

And we'd like to ask you for help (once again). Please pass this survey. We develop a bugtracker Bugira specially for web developments. And to get it running we need your feedback.

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

P.S. It's only 3 minutes of your time: pass this survey. Thanks

The Platform (1.6.0): Interactive Console

$
0
0

Today we start new The Platform iteration. After 1.6 Codeception would become much more friendly and flexible in testing. Yet, no of announced features are introduced today. It's really hard to keep promises especially if you come to idea of feature that would be really really helpful for acceptance testing.

Meet Interactive Console

Within the interactive console you can try various commands without running a test. It is quite useful when you deal with Selenium sessions for example. It always takes a long time to start and shutdown Selenium session. So it's unpleasent to see that your test fails and it should checked saved and executed once again and once again.

Interactive console

Interactive console should simplify writing tests as you can play with different selectors and commands. Also it may be useful when you want to try your new RESTful API. In console you can execute Codeception commands as you use in tests, but receive immediate result.

Try using the console:

php codecept.phar console suitename

Every command is exuecuted with eval. So check a your input to be valid PHP. And yes, you can skip $I-> in the begining and ; in the end. But all the brackets and quites are required.

Low Level APIs

Sometimes it happens that Codeception limits you in your actions. Probably you could solve the issue within a Selenium, but Codeception Modules may not provide full access to it. Well, before this day.

Now you can access launching backends directly in tests:

<?php
$I->amOnPage('/');
$I->executeInSelenium(function(\WebDriver\Session $webdriver) {
   $webdriver->element('id','link')->click('');
});
?>

For Selenium, Selenium2 modules commands executeInSelenium were introduced. For PhpBrowser you can use executeInGuzzle which provides you access to Guzzle HTTP Client.

Zend Framework 2 Initial Support

Thanks to bladeofsteel you can try to use Codeception with Zend Framework 2. Please try to use ZF2 module on real Zend Framework 2 applications. ZF2 module is in alpha now, so your feedbacks, issues, and pull requests are highly appreciated.

Various additions

  • defer-flush option to Run was added by phpcodemonkey. It helps to overcome output errors for session in functional tests.
  • amHttpAuthenticated command added to all framework modules and PhpBrowser
  • do not use steps option by default if it's not a one test executed by tigerseo
  • fixes to AMQP module. Still it is in beta.
  • you can use Cest files for writing functional and acceptance tests.
  • CURL parameters can be set for PHPBrowser (SSL checks turned off by default).

Sorry if some bugs were not fixed. Any help and pull requests are highly appreciated.

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Soon a new Guides will be added to documentation replacing the Scenario Unit Tests guides. Follow the updates.

Specification or Testing: The Comparison of Behat and Codeception

$
0
0

This is guest post by Ragazzo. He uses Behat as well as Codeception for making his project better. He was often asked to do a comparison between Codeception, Behat, and PhpUnit. In this post he explains the commons and different parts of this products.

In nowadays almost everyone knows how it is important to have stable and tested application. It is very difficult to create good application without involving different specific tools in your project. It is very important to develop your application through testing, according to TDD principle. There are different ways to write tests in PHP:

  • Classical unit-tests with PhpUnit;
  • Functional (integration / end-to-end) tests with PhpUnit
  • Functional web tests with Codeception / Behat+Mink
  • Acceptance web tests with Codeception / Behat+Mink.

My vision of BDD.

Main difference between usual functional and acceptance tests is that in acceptance tests you can not manipulate directly with some "underground" application code. All you can do is just interact with things that was given to you by application. In functional tests you can interact with application internals, proceed some checks and conditions that can involve application parts not available for end-user.

If we are talking about "classical" functional tests also known as "end-to-end" or "integration" tests then simple example can be when you have to test some behavior of 2-3 components together, like logging, writing configs, etc. For functional tests (but not web) I prefer to use Behat by itself. And I have some reasons for that.

Behavior Driven Development helps you to start developing project from "how it must behave" and not from "what concrete components it must include in particular".

It can be said that you write your thoughts about behavior of your application. And this helps you to build not simply a couple of classes powered by unit-tests, but an application with good design based on specific behavior. Behat is very powerful tool, but it has some common pitfalls that maybe looks weird to the user:

  • It could be said that it is a feature driven development. Thats why it is not so easy to use subcontexts in Behat. For example, if you have 30-40 tests it could be hard to maintain them, because of the same PhpDoc definitions for matching methods.
  • If you use Behat+Mink for functional and acceptance tests, it also requires from you a good knowledge of Mink's internals to create your own methods to interact with a page.

Lets see the example on how you can determine what components you need in your application in a "BDD" way. Here we are creating dummy application for managing football clubs. Lets write some 2-3 scenarios in Gherkin:

#football_clubs.feature

Feature: football_clubs
  As a football club owner
  I want to perform different actions with it.

  Scenario: transfer player to the club
    Given I have a football club "MyClub"
    Given Football club budget is 100 mln. dollars
    Given Football player "GoodSkilledPlayer" costs 35 mln. dollars
    When I try to buy this player
    And He see is available for transfer
    Then I sign player contract with 10 mln. dollars salary
    And I see player club is now "MyClub"

  Scenario: get list of injured players
    Given I have a football club "MyClub"
    When I get a list of injured players
    Then I see players in list:
    """
    FirstFootballPlayer
    SecondFootballPlayer
    """
    And I dont see players in list:
    """
    ThirdFootballPlayer
    """

Thats all. We just wrote some thoughts on how an application should behave. From here we know what exactly we need: "Football Player" object, "Football Club" object, "Transfers Market" object. If you know how they must behave you do not need to think for hours on what unit-test you need to write, and whether they are really needed in the component behavior.

Lets see another example of functional tests with Behat, for example we need to test config files of our system. Feature for that can be like this:

#config.feature
Feature:
  In order to proceed config files for MyModule
  As a system root user i should be able
  to write correct info in files

  Scenario: write some users "context" info
  Given I have users in system:
    | username | email              |
    | first    | first@example.com  |
    | second   | second@example.com |
  When I dump users info to config file
  Then I should see
  """
  #users
  username=first
  email=first@example.com

  username=second
  email=second@example.com
  """

As you see it is easy to read and also easy to understand. Thats why I prefer Behat for "classic" functional tests against usage of PhpUnit to test behavior of 2-3 components.

Ways of Writing Acceptance Tests

Now lets talk about functional and acceptance tests for web. Now we know difference between acceptance and functional tests, so in web tests difference is the same. There is a couple of tools to use for web functional and acceptance tests:

  • PhpUnit + Selenium
  • Behat + Mink
  • Codeception

I do prefer Codeception over Behat+Mink for web acceptance and functional tests for these reasons:

  • In web it is not very important to develop pages from how they need to behave. So there all power of Gherkin could not be applied. In this case Gherkin works just as simple translator and nothing more. My decision is to use right things for right purposes.
  • As I've pointed out above you also need to know Mink "internals" (API) to proceed some your custom methods.
  • Testing web-pages can lead to many "Ambiguous" or "Redundant" Behat feature steps.

Of course there are some things to avoid this, but i prefer "easy-to-learn things" in this situation.

In that case Codeception offeres you a simple and very easy way to test your web application. Also I need to notice that Codeception takes low-level Mink's API and provides a high-level commands for end-user.

For example lets see simple Codeception Cept test case:

<?php
$I = new TestGuy($scenario);
$I->wantTo('Test index page');
$I->amOnPage('/index.php');
$I->see('My Web Application','#header #logo');
$I->click('Login');
$I->see('Login','h1');
$I->see('Username');
$I->amGoingTo('login in the test app');
$I->fillField('#LoginForm_username','demo');
$I->fillField('#LoginForm_password','demo');
$I->click('#login-form input[type="submit"]');
$I->seeLink('Logout (demo)');
$I->click('Logout (demo)');
$I->seeLink('Login');
?>

As you can see it is easy to read, and one really great thing is it is easy to write, because of auto-completion. In Behat when using Gherkin you can get some "Undefined" steps errors, just because you mistyped something. And it happens to be annoying.

Codeception bundled with modules for functional and acceptance tests: PhpBrowser (functional tests over curl), Selenium (1,2) for acceptance tests, etc. There are also some nice modules for REST, SOAP and XML-RPC that can help you to test your API.

It is need to be noticed that there are two types of web tests in Codeception: some require webserver and others not - all requests are processed directly by your applcation using Symfony's BrowserKit Component.

Overall, my choice in Codeception is to use "PhpBrowser" module (requires a web-server) for headless web functional tests, and "Selenium2" module for complete web acceptance tests. Codeception also can generate BDD-style scenarios for you based on your Cepts.

Also need to notice that maybe Codeception is not a "true" BDD, unlike Behat. Behat+Gherkin helps you to use and design your application. Codeception helps you to test it nicely.

Links to read and get more info

This post was intended to be a start to series. In next posts you will learn how to use newcoming Cest format for testing and integrate Codeception with Continious integration tools. Stay tuned!


Codeception 1.6.1: Laravel and Groups

$
0
0

That was about a month of hard work on Codeception 1.6.1. Probably the hardest part was to provide more open architecture for testing framework. Right now it's hard to customize anything in Codeception. We are going to fix that soon with new event system. We are going to release Extension (for hooking all the events) and Group classes (for events related to groups) in next release.

Hey, did you mention groups?

Yep, you are right! From now on you can use group tests and execute them separately. You have a few ways to add test to a group. If it is a Cept scenario all of this methods will work.

<?php
$scenario->group('admin');
$scenario->group(array('admin','critical'));
$scenario->groups(array('admin','critical'));

$I = new WebGuy($scenario);
// ...
?>

For Cest and Test files you can use @group annotiation similarly as in PHPUnit:

<?php
 /**
  * @group model
  */ 
  public function testAdminIsCreated()
  {
    // test goes here
  }

?>

To execute tests of one group use:

php codecept.phar run -g admin php codecept.phar run --group admin

To execute tests of several groups just pass multiple group options to runner.

Functional Tests in Classes

As you know, we deprecated Cest format for unit testing and we propose you to use it for functional and acceptance tests. It should work well for you now. Soon we will publish a tutorial and guide for how to to this. Basically you can generate a Cest file:

php codecept.phar generate:cest acceptance MyCest

And write tests inside a newly created MyCest class.

<?php
class MyCest 
{
  public function loginPage(\WebGuy $I)
  {
    $I->amOnPage('/login');
    $I->see('Login', 'h1');
    $this->checkCopyrightIncluded($I);
  }

  public function registrationPage(\WebGuy $I)
  {
    $I->amOnPage('/register');
    $I->see('Register', 'h1');
    $this->checkCopyrightIncluded($I);
  }  

  protected function checkCopyrightIncluded($I)
  {
    $I->see('(c) 2013', '#footer');
  }
}
?>

All public methods of Cest classes become tests. You can use private and prected methods and properties to share data and functionality over the tests. Also you can use setup/teardown preperations in _before and _after methods. If you want to make a bunch of simple tests and you don't want to create a file for each - Cest is your choice. More docs on it coming.

Good news to Laravel fans

Yes, yes. Finally we've got a module for Laravel4 framework functional testing. We couldn't test it well, as there are not to much Laravel4 apps available on GitHub. But we run some basic tests on a CRUD app, and it worked nicely. Thanks to HttpFoundation component inside Laravel4 framework.

Larvael module is zero configuration. You should just bootstrap a project in your Laravel app directory and include Laravel4 module into your functional test suite.

Take a look into sample Laravel project prepared my Jeffrey Way and Michael Bodnarchuk with sample test included.

Various changes and fixes.

  • added seeOptionIsSelected methods for all modules that share FrameworkInterface.
  • added resizeWindow method to Selenium2 module (thanks to Jaik Dean)
  • Codeception is PHP 5.5 compatible. We learned that by launching tests on Travis (thanks to Tiger-Seo)
  • fix for screenshots snapshot filenames in Cest files
  • fix to getStatusCode method in frameworks tests

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Playng with Cests

$
0
0

This is the second guestpost from Ragazzo. He writes about using Cest format in functional tests.

As you already saw it is easy to use flat Cept files in Codeception for your tests. But what if you want to test CRUD operations so a Cept can take 50-70 rows long. In this case it would not be so "easy-to-read".

This is where Codeception's Cests are really good to use. Cests are simple classes with no parents. You can use them like your classic PHPUnit classes. All that need to know is that each public Cest method is a separated scenario, which could be represented by a Cept file.

The other issue we want to solve with Cest files is UI mapping. It's ok to start testing with relying on button names and field labels. But the more UI elements come to page, the more complex our selectors become. There should be a way to manage those UI elements for tests. It's better not to put too much CSS/XPath code into scenario as it affetcs readbility of code and hardens its support.

Lets see example of testing simple Yii 1.1 application page in Cest-way with PhpBrowser module:

<?php

class IndexCest
{

  /**
   * user name text field
   * @type string
   */
  public $username = '#LoginForm_username';

  /**
   * user password text field
   * @type string
   */
  public $password = '#LoginForm_password';

  /**
   * submit button
   * @type string
   */
  public $enterButton = '#login-form input[type=submit]';

  /**
   *
   * @param \TestGuy $I
   * @param \Codeception\Scenario $scenario
   */
  public function check_basic_login_logout(\TestGuy $I, $scenario)
  {
    $I->wantTo('Test index page');
    $I->amOnPage('/');
    $I->see('My Web Application','#header');
    $I->click('Login');
    $I->amGoingTo('login in the test app');
    $I->fillField($this->username,'demo');
    $I->fillField($this->password,'demo');
    $I->click($this->enterButton);
    $I->seeLink('Logout (demo)');
    $I->click('Logout (demo)');
    $I->seeLink('Login');
  }

}
?>

As you see, it was easy to add move out selectors into UI properties if we are inside a calss.

Example of how to test CRUD operations with Selenium2 module is below. Please notice that all support methods are protected. They are called from test methods and won't be executed as test themeselves.

<?php
/**
 * "Users" module CrudCest class file
 * @author Ragazzo
 */

class CrudCest
{

  /**
   * your fields definitions goes 
   * here with UI-mappings
   * and other fields.
   *
   */

  private $_userId;

  /**
   * new user created attributes
   */
  private $_newAttrs = array(
    'name'  => 'Ragazzo',
    'email' => 'someRagazzoEmail@example.com',
    'skype' => 'someRagazzoSkype',
  );

  /**
   * edited user attributes
   */
  private $_editAttrs = array(
    'skype' => 'newRagazzoSkype',
  );

  /**
   *
   * @param \WebGuy $I
   * @param \Codeception\Scenario $scenario
   */
  public function test_basic_users_module_crud(\WebGuy $I, $scenario)
  {
    $I->wantTo('create user and test basic crud in "Users" module');
    $I->am('system root user');
    $I->loginIntoTheSystem('somelogin','somepassword',$I);
    $this->create_new_user($I, $scenario);
    $this->update_created_user($I, $scenario);
    $this->delete_created_user($I, $scenario);
  }

  /**
   *
   * @param \WebGuy $I
   * @param \Codeception\Scenario $scenario
   */
  protected function create_new_user($I, $scenario)
  {
    $I->amGoingTo('create new user');
    $I->amOnPage('/users');

    $I->see('Users','.breadcrumbs')
    $I->see('Create','.btn');

    $I->click('Create','.btn');
    $I->seeInCurrentUrl('users/create');
    $I->see('Create','.breadcrumbs');

    $I->amGoingTo('submit form without required fields');
    $I->click($this->saveButton);

    $I->expectTo('see required validation errors');
    $I->see('Name field can not be empty');

    $I->amGoingTo('submit correctly filled form');
    $I->fillField($this->name, $this->_newAttrs['name']);
    $I->fillField($this->email, $this->_newAttrs['email']);
    $I->fillField($this->skype, $this->_newAttrs['skype']);
    $I->click($this->saveButton);

    $I->expectTo('see correctly saved user info');
    $I->seeInCurrenturl('users/view');
    $I->see('View','.breadcrumbs');
    $I->see('Delete','.btn');
    $I->see('Edit','.btn');
    $I->seeElement('.detail-view');
    $I->see($this->_newAttrs['name'],'.detail-view');
    $I->see($this->_newAttrs['email'],'.detail-view');
    $I->see($this->_newAttrs['skype'],'.detail-view');

    $this->_userId = $I->grabFromCurrentUri('~/id/(\d+)/~'); 
  }

  /**
   *
   * @param \WebGuy $I
   * @param \Codeception\Scenario $scenario
   */
  protected function update_created_user($I, $scenario)
  {
    $I->amGoingTo('change created user attributes');
    $I->amOnPage($this->editUrl.$this->_userId);
    $I->fillField($this->skype, $this->_editAttrs['skype']);
    $I->click($this->saveButton);

    $I->expectTo('see that attributes has changed');
    $I->seeInField($this->skype,$this->_editAttrs['skype']);
  }

  /**
   *
   * @param \WebGuy $I
   * @param \Codeception\Scenario $scenario
   */
  protected function delete_created_user($I, $scenario)
  {
    $I->amOnPage('/users');
    $I->see($this->_newAttrs['name'],'#users-grid');
    $I->click($this->_newAttrs['name'],'#users-grid');
    $I->see('Delete','.btn');
    $I->click('Delete','.btn');
    $I->seeInCurrentUrl('/users');
    $I->dontSee($this->_newAttrs['name'],'#users-grid');
  }

}
?>

In this way you can use Cests classes for some tasks where it is difficult to use flat Cepts. You also can use Cests classes as PageObjects.

When you write tests with Codeception it is good to be verbose and use different methdos for that, so use them:

  • $I->am to say who you are and define your role;
  • $I->wantTo to say what feature you want to test;
  • $I->amGoingTo to say what you gonna do next;
  • $I->expect and $I->expectTo to say what do expect next to see in your test case as a user.

Use this commands instead of comments in your tests. They will be displayed in scenario output and will provide you with additional information on steps taken.

We can say to Ragazzo for sharing his experience in using the Cest classes. Stil it might be a point of discussion, should a CRUD be tested in one test or in four. In both cases Cest classes are a good places to keep everything you need to test dealing with one entity. A user, in this case.

Headless Browser Testing with Selenium2 and PhantomJS

$
0
0

This is a guest blogpost was brought to you by Jon Phipps. Jon explains how to use PhantomJS -- the most anticipated testing backend in Codeception.

If you're running acceptance tests in Codeception that require interaction with JavaScript, or have scripts that manipulate the DOM, the speedy Php Browser driver won't help you since it doesn't support JavaScript. The best options for running tests using JavaScript are the Selenium2 and ZombieJS drivers.

The Selenium2 driver actually loads and runs an active browser session, manipulating the browser just as a human would. ZombieJS is a 'headless' browser that provides all of the features of a regular browser, but without a display interface. Without the extra time spent waiting for the display to actually render, a headless browser like ZombieJS can run far faster than a normal browser, so you're tests will execute in as little as half the time. But ZombieJS requires installing Node.js and can be a little buggy, plus it has its own API (which has both pros and cons). The Selenium2 driver is well tested and implements a standard API -- the WebDriver Wire Protocol -- across all of the browsers it has drivers for.

Now there's a headless browser that includes a WebDriver Wire Protocol implementation -- PhantomJS. The latest version of PhantomJS is an easy to install, stand-alone binary that doesn't require installing Node.js or any other dependencies, and ships with its own 'Ghost Driver' for implementing the WebDriver Wire Protocol. Which means you can drive it using the Selenium2 driver in Codeception, and anything that you can test in Chrome, Firefox, Safari, or IE using Selenium2, you can now test in half the time using PhantomJS

To get started, if you haven't installed Selenium2, you just need to follow the instructions on the Codeception web site for installing and running the Selenium2 driver.

Next download PhantomJS. The binary is setup and ready to use for Linux, Mac OSX, and Windows. Put it, or a symlink to it, somewhere in your path so you can launch it from anywhere.

You're done with installation!

Now open up a new terminal window and fire up an instance of Selenium Server, leaving the terminal window open:

$ java -jar selenium-server-standalone-2.0b2.jar

This will launch the server listening on the default port of 4444. You should see something like this in the terminal:

May 10, 2013 9:41:38 AM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
09:41:46.852 INFO - Java: Apple Inc. 20.45-b01-451
09:41:46.857 INFO - OS: Mac OS X 10.7.5 x86_64
09:41:46.941 INFO - v2.32.0, with Core v2.32.0. Built from revision 6c40c18
09:41:47.774 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
09:41:47.775 INFO - Version Jetty/5.1.x
09:41:47.775 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
09:41:47.776 INFO - Started HttpContext[/selenium-server,/selenium-server]
09:41:47.777 INFO - Started HttpContext[/,/]
09:41:47.983 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@1f25fefa
09:41:47.983 INFO - Started HttpContext[/wd,/wd]
09:41:48.011 INFO - Started SocketListener on 0.0.0.0:4444
09:41:48.011 INFO - Started org.openqa.jetty.jetty.Server@16a4e743

If you already have a listener on that port, you'll see a handy error message:

09:50:34.172 WARN - Failed to start: SocketListener0@0.0.0.0:4444
Exception in thread "main" java.net.BindException: 
Selenium is already running on port 4444. Or some other service is.

Next, open up a new terminal window and launch PhantomJS, telling it to use the its built-in WebDriver extension to talk to Selenium on the port Selenium is listening to, and leave that window open too:

$ phantomjs --webdriver=4444

You should see a response like this in your terminal:

PhantomJS is launching GhostDriver...
[INFO  - 2013-05-10T14:11:05.076Z] GhostDriver - Main - running on port 4444

Now just change the browser setting in your acceptance.suite.yml file (an example file is on the Selenium2 driver page) to browser: phantomjs. If you're changing modules then you should also run php codecept.phar build.

Check it out by doing a fresh Codeception run:

$ php codecept.phar run acceptance

Your tests should now run quietly and silently, and much faster.

You should see some output in your PhantomJS terminal window providing some useful feedback on this session's capabilities provisioning. This happens on every run (the output below are the defaults):

[INFO  - 2013-05-10T14:33:43.796Z] Session [9dbc5700-b97e-11e2-8dc9-976d2e8732bf] - 
CONSTRUCTOR - Desired Capabilities:{
  "browserName" : "PhantomJS",
  "version" : "9",
  "platform" : "ANY",
  "browserVersion" : "9",
  "browser" : "firefox",
  "name" : "Codeception Test",
  "deviceOrientation" : "portrait",
  "deviceType" : "tablet",
  "selenium-version" : "2.31.0"
}
[INFO  - 2013-05-10T14:33:43.796Z] Session [9dbc5700-b97e-11e2-8dc9-976d2e8732bf] - 
CONSTRUCTOR - Negotiated Capabilities: {
  "browserName" : "phantomjs",
  "version" : "1.9.0",
  "driverName" : "ghostdriver",
  "driverVersion" : "1.0.3",
  "platform" : "mac-10.7 (Lion)-32bit",
  "javascriptEnabled" : true,
  "takesScreenshot" : true,
  "handlesAlerts" : false,
  "databaseEnabled" : false,
  "locationContextEnabled" : false,
  "applicationCacheEnabled" : false,
  "browserConnectionEnabled" : false,
  "cssSelectorsEnabled" : true,
  "webStorageEnabled" : false,
  "rotatable" : false,
  "acceptSslCerts" : false,
  "nativeEvents" : true,
  "proxy" : {
    "proxyType" : "direct"
  }
}
[INFO  - 2013-05-10T14:33:43.796Z] SessionManagerReqHand - _postNewSessionCommand - 
New Session Created: 9dbc5700-b97e-11e2-8dc9-976d2e8732bf

You can adjust these capabilities in your acceptance.suite.yml file like so:

modules:
   enabled: [Selenium2]
   config:
      Selenium2:
         url: 'http://localhost/'
         browser: phantomjs
         capabilities:
             webStorageEnabled: true

I have no idea if capabilities from the larger list of Selenium DesiredCapabilities that are not on the list you see reported from the driver are enabled for PhantomJS.

Headless testing can be a bit of a challenge, since it's impossible to 'see' what failed. But in this case, Codeceptions default logging and screenshot capture on failure can be extremely helpful, since you can then actually see the state of the browser at the point of failure.

There's quite a bit more that you can do with PhantomJS. The CasperJS project makes good use of the PhantomJS API and if you already have NodeJS installed it's a quick and easy way to play with some of the PhantomJS capabilities. CapserJS, aside from requiring NodeJS, only drives PhantomJS. So its not a reasonable alternative to Codeception. Maybe at some point there will be a native Mink driver for the PhantomJS API which will more fully exploit it.

Happy testing.

Continuous Integration In Practice: Codeception, Jenkins, Xvfb

$
0
0

Another guest post from Ragazzo containing some practical tips on configuring CI server. If you ever wanted to ask how you could use Codeception with Jenkins, you are in a right place.

It is very good to automate maximum of the things you can. Once we automated testing process, build should be automated too. I use Jenkins as my primary continuous integration server. It can be installed and executed on all popular operating systems with Java enabled.

As for PHPUnit I use JUnit format for error reports and some Jenkins plugins to make it work with Code Coverage. For Behat and Codeception I also use JUnit output log format and Jenkins PHPUnit Plugin.

Thus, my application have 4 jobs in Jenkins they named in that way:

  • MyProject_Testing_Unit # via phpunit
  • MyProject_Testing_Func # via behat
  • MyProject_Testing_Func_Web # codeception functional
  • MyProject_Testing_Accep_Web # codeception acceptance

And of course one job for bulding packages for demo and for developer:

  • MyProject_Build_Dist

I scheduled jobs to run one by one so the first goes Unit, then it triggers Func, then Func triggers Func_Web and so on. Build is not triggered automatically, I start it by myself.

Configuring Builds

Lets pay attention on two of them that include Codeception. I'm using Ant build tool to execute them. Bascially for JUnit output you need nothing more then running codeception with --xml option. Like this:

codecept.phar --xml run functional

This will produce the report.xml file in tests/_log directory, which will be analyzed by Jenkins. In my Ant task I'm also clearing a _log directory before running tests.

<project name="MyProject_Testing_Func_Web" default="build" basedir=".">
<target name="clean">
  <delete dir="${basedir}/build/src/protected/tests/codeception/tests/_log" includes="**/*" />
</target>
<target name="codeception">
  <exec dir="${basedir}/build/src/protected/tests/codeception" executable="php" failonerror="true">
    <arg line="codecept.phar --xml run functional" />
  </exec>
</target>
<target name="build" depends="clean, codeception"/>
</project>

Also you may want to add the --html option for execution. That would produce a readable HTML report, that you can show to your boss. You will need HTML Publisher Plugin to display it in Jenkins web interface.

Using Selenium2 and Xvfb

To run acceptance web tests correctly on server you don't need to install desktop environment like Gnome or XFCE just to launch a browser. Better to use Virtual Framebuffer Xvfb for running Selenium tests without showing them on screen.

You need to install Xfvb and then run Selenium2 with this options:

export DISPLAY=":1" && java -jar selenium-server-standalone.jar

Then start your Codeception acceptance tests as usual. By the way, I suggest you to use this Xvfb init script in Ubuntu.

There is nothing hard in setting up Jenkins. You can use Codeception with it or in any other CI server. If you already did that, please share your experience with community. We really need more personal experiences and blogposts from the community. Documentation can't cover all use cases, that's why your thought are much appreciated. If you don't have a blog we can publish your blogpost here.

P.S. Just to mention, If you are using Bamboo CI server, pls enable the --no-exit option to run tests correctly.

Codeception 1.6.2: Subdomains Support

$
0
0

Looks like a good time for a new release. We decided to stick to features that was asked a lot and to merge all uesful PRs that were submitted.

One of them is Subdomain support. If you were wondering how to deal with subdomains of your app, we have a new amOnSubdomain command for you. It will prepend a subdomain name to your current url you put in config.

<?php
// configured url is http://user.site.com
$I->amOnSubdomain('user');
$I->amOnPage('/');
// moves to http://user.site.com/
?>

But... well, this would be much easier to implement by anyone if only you had an option to dynamically pass configuration to modules. And yes, it's also possible now. You can use new _reconfigure method of a module. In helper you can access it and replace the keys with required values.

<?php
$this->getModule('Selenium2')->_reconfigure(array('browser' => 'chrome'));
?>

In the end of a test, all changed options will be rolled back to configuration values.

Some new nice improvements and fixes from our contributors:

  • Cookie methods like getCookie, setCookie can now be used in acceptance tests (thanks igorsantos07)
  • Curl options are now correctly passed to PhpBrowser (thanks to mrtimp)
  • Also X-Requested-With is cleared in Ajax requests of PhpBrowser (thanks to vinu)
  • Non-strict matching for links (with normalize-space XPath) was proposed by brutuscat.
  • Symfony1 module fixed (by walkietalkie)
  • Lots of small fixes and tests by Ragazzo.
  • fix to haveInDatabase method by davert.

Update

redownload your codeception.phar for update:

wget http://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update
Viewing all 321 articles
Browse latest View live


Latest Images