Quantcast
Channel: Codeception

Testing WordPress Plugins with Codeception. Part 2

0
0

In previous part of this tutorial we installed Codeception and got a simple test for User Submitted Posts plugin. We tested that user can send a post and see a message that the post was successfully sent. Nothing more.

Let’s remind the test SubmitPostCept we done previously:

<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','https://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');
?>

But well, we didn’t verify that the admin has received this post. And it can be published after a moderation. Thus, we will require few more steps to make a test complete. We will need to login to WordPress admin dashboard.

Dashboard

Test commands which allow us to do that are pretty obvious. To keep the code listing shorter we won’t show the code from previous lesson. But you should understand that we just append new commands to the previous steps.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
?>

Then we go to Posts section to get all the post listed. We expect to see the “Game of Drones” post in the list.

Posts

Let’s also check that it was not published by default and it is in Pending state. Game of Drones Review - Pending should be found inside the table html tag, right? We can even specify the CSS class .posts for that table.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
?>

That’s right, the see command we use for assertions (and the test will fail if assertions fail) has a second parameter which allow us to narrow the results from the whole page, to a particular area on a page, which we can define by CSS:

Posts CSS

What’s left? We need to review and publish this post, right? The result we expect… Well, we will trust the notifications once again, and message Post published should be enough for us.

Published

And that’s how we will do that in test.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review','table.posts');
$I->click('Publish');
$I->see('Post published');
?>

And that would be a good point to execute tests once again. Probably you remember, that you need execute codecept.phar with run parameter from console.

php codecept.phar run

What we are seeing now? Oh no. Just another fail.

Fail

What could go wrong? Hopefully Codeception gives us a suggestion to look for the complete HTML code in _log directory. This directory, actually tests/_log, was meant to store all data related to test execution. We can see there two log files, and the file which we actually need: SubmitPostCept.php.page.fail.html.

Log

This file, named after our test name, stores the HTML that was on page, before the fail. Let’s open it in a browser to get a clue what might go wrong here.

Publish

It looks like we are still on the Edit Post page we were before. And the post status is still Pending Review. It looks like click('Publish') command didn’t do its job.

Take a look on screenshot. as you can see, word Publish occur several times on a page. We can assume that we were clicking wrong one. What should we do in this case? We can specify the exact that blue Publish button with CSS.

Codeception can use CSS instead of names for clicking the elements. The button have id=publish attribute, thus, it can be found by #publish selector. That what we would probably do if we were using jQuery.

<?php
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review');
$I->see('Edit Post');
$I->click('#publish'); // id of "Publish" button
$I->see('Post published');
?>

And that fixes the test. You can check that by executing test once again. Let’s not trust the notification test and check that a post really is on site. We can click View Post link right near the Post updated message.

Page

What we will check that there is site motto on a page (thus we know, we see the blog theme), and that post title is shown in .entry-title class. And yep, we see this story is “epic and amazing”. Let’s face the final and complete code of our test:

<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','https://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');

$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review');
$I->see('Edit Post');
$I->click('#publish');
$I->see('Post published');
$I->click('View Post');
$I->see('Just another WordPress site');
$I->see('Game of Drones Review','.entry-title');
$I->see('This story is epic and characters are amazing.');
?>

Well, we did a good job. But the test is like too long. And we can’t understand what was going on here. We can add comments to the code, and Codeception has some valuable helpers to add extra text informations with expect and amGoingTo commands.

<?php
$I = new WebGuy($scenario);
$I->wantTo('submitted a post by user and publish it as admin');

$I->amGoingTo('submit a post as a regular user');
$I->amOnPage('/');
$I->click('Submit a Post');
$I->fillField('Your Name', 'Michael');
$I->fillField('Your URL','https://drone-rules.com');
$I->fillField('Post Title', 'Game of Drones Review');
$I->fillField('Post Tags', 'review book rob-starkraft');
$I->selectOption('select[name=user-submitted-category]', 'Game of Drones');
$I->fillField('Post Content', 'This story is epic and characters are amazing.');
$I->click('Submit Post');
$I->see('Success! Thank you for your submission.');

$I->amGoingTo('log in as admin');
$I->amOnPage('/wp-login.php');
$I->fillField('Username', 'admin');
$I->fillField('Password','admin');
$I->click('Log In');
$I->see('Dashboard');

$I->expect('submitted post was added to a list');
$I->click('Posts');
$I->see('Game of Drones Review','table.posts');
$I->click('Game of Drones Review');

$I->amGoingTo('publish this post');
$I->see('Edit Post');
$I->click('#publish');
$I->see('Post published');

$I->expect('post is available on blog');
$I->click('View Post');
$I->see('Just another WordPress site');
$I->see('Game of Drones Review','.entry-title');
$I->see('This story is epic and characters are amazing.');
?>

We divided our test scenario into logical parts. At least we have left some notice about what is going on and what we were going to achieve. If we execute test with --steps option, we will get a output with all passed steps listed:

php codecept.phar run --steps

Final

Our comments were added to list of passed steps, thus we can easily understand what was going on. And that’s quite enough for today. Our test is pretty mature, and covers not only plugin functionality, but WordPress core functions too. Whenever this plugin gets updated, this test should pass to ensure we did everything right.

But the only thing left. After we executed tests several times, we got this picture on Posts screen.

Previous Posts

We have attack of clones here. Yep, each test created its own post and published it. Probably, it is not a good idea to pollute the blog with dozens of posts with similar names. We use post title and content in a test, so probably we can’t be sure, what post we are dealing with: current one or the post from a previous test.

That will lead us to the idea of data cleanup. Probably we should delete post after it was published, to revert all our changes. Alternatively we can install WordPress Reset Plugin to reset WordPress to its initial state. In both cases we will need to append some steps to our test to get data cleaned before the next test.


Let’s make this your home task. To not start with scratch, you can download code of this tutorial or clone it on Github.

In next lesson we will try to refactor this test and get a few more of them. Don’t worry, they will be much shorter than this one. See you soon!


Codeception 1.6.5: Debug and Bugfixes

0
0

This is a minor release, mostly done to fix some bugs, you have encountered. Please, submit your Pull Requests for the bugs critical of yours. Most of pull requests are accepted, but if you will start a proposal, we can recommend you the best way to implement the fix.

Selenium2 Compatiblity

Selenium2 server v.2.34 was released recently and to use it you need to update Codeception. If you use phar version, you should replace your old codecept.phar with new one.

Debug Levels

At least one useful feature we prepared for you. Debug output in PhpBrowser and REST modules was extended with additional information that will be printed in debug mode:

debug

Tests would be much easier to debug when you see reponse headers, status codes and client cookies. Don’t forget to add --debug to run your PhpBrowser acceptance tests, option to see that.

Title Actions

Two basic yet useful actions were added to all web interaction modules.

<?php
$I->seeInTitle('My Blog | My Post #1');
$I->dontSeeInTitle('Her Blog');
?>

Should be useful, right?

Single Test

Finally you can now execute a single test from Cest or Test testcases.

php codecept.phar run tests/unit/UserModelTest.php:testSave

In this case we will execute only testSave test out of UserModelTest TestCase. The same works for Cests. You may write only the beginning of test name, to execute it.

Bugfixes

  • fix to correct displaying of non-latin characters in html-report by shofel
  • --xml output for Codeception\TestCase\Test fixed
  • fixed unserialize error during code coverage. Anyway, if you ever seen this, you didn’t setup coverage correctly.
  • Interactive console console command does not boot with error stacktrace.
  • Clearing only tables and not views in Db->cleanup()
  • PDO $dbh is now passed to Db module corretcly #414

Release Plan

Also we are planning to get more stable releases, and follow the Semantic Versioning. This means that current stable branch is 1.6. If you submit patches and bugfixes, you should propose them into 1.6 branch. Experimental features should go to master.

release branch status
Stable 1.6 Build Status Latest Stable
Development master Build Status Dependencies Status

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Codeception 1.6.6: Sequences

0
0

A minor release with one major announcement. In 1.6.6 most bugfixes were included from pull requests. Thanks for everyone who did the contributions.

Sequence

Also a very tiny new module was included. It’s Sequence which was created to generate unique data for your tests. Sequnces become pretty handy if you don’t do database cleanup before each tests. They guarantee you can get unique names and values for each test.

<?php
$I = new WebGuy\PostSteps($scenario);
$I->wantTo('create a post');
$I->createPost('Post'.sq(1), 'Lorem Ipsum');
$I->see('Post created sucessfully');
$I->see('Post'.sq(1), '#posts');
?>

No matter how much times you execute this test, each time you see a new post is created with different name.

Bugfixes

  • Remote codecoverage now works with Selenium2 module. Please update c3.php file to use it.
  • IoC container access in Laravel4 module by @allmyitjason.
  • PostgreSQL driver fixes by @mohangk and **@korotovsky **.
  • don’t rollback for inactive transaction in Dbh module by @sillylogger
  • fix to guy classes generation with namespaces by @vinu.
  • SQLite improvements by @piccagliani

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Living on the Edge: WebDriver

In July a group of Facebook developers set the goal to write the complete new Selenium Webdriver bindings. They decided to do it finally the right way, with the same WebDriver interface it is used in other languages like Java and C#. Ironically, Selenium2 module of Codeception uses the old webdriver bindings from Facebook.They were very hard in use, and had lots of issues. Most common issues were solved in Element34 fork, which was then forked by Instaclick to bring namespaces and PSR-0, which was then used in Mink’s Selenium2Driver, and Mink was used in Codeception.

Pretty tricky, right?

Currently there are 3 WebDriver bindings in PHP.

  • Selenium2TestCase of PHPUnit which is the most old, the most complete and the most OOP webdriver implementation. But if you have worked with its api, you understand how robust it is to learn and use.
  • Element34 fork based on initial facebook/webdriver bindings, but with hooks to solve common pitfalls.
  • Selenium2Driver of Mink which incorporates Element34 bindings and Syn.js library to perform most of interactions via JavaScript bypassing WebDriver API.

Ok. Now we have new facebook webdriver bindings. They are in active development and they lack proper documentation. But the good part of it, that even without documentation you will easily learn how to work with them. Any question on StackOverflow with examples in Java or C# will work in PHP just the same way.

In Codeception master we created a new WebDriver module which uses new webdriver bindings. This module will be included into first 1.7 release, but it won’t be recommended for regular use before the stable version of php-webdriver is released.

To try the new WebDriver you should switch to dev-master in composer.json or use the pre-prelease phar package.

WebDriver module does not implement everyhting the Selenium2 module has. There is no right clicks, drag and drops, and more. But there are few handy improvements:

  • submitForm action as in PhpBrowser.
  • waitForElement action to wait for element to appear on page.
  • selectOption and checkOption now both work with radio buttons.
  • seeElementInDOM to check the invisible elements.
  • waitForElementChange to wait that element changed
  • implicit waits with wait config prameter (default is 5 secs).
  • maximizeWindow specially for @aditya- :).

In all other ways its pretty compatible with Selenium2 module. Try it on your own risk or wait for stable versions.

Understanding AspectMock

0
0

As you may know, AspectMock is non-ordinary mocking framework that can override any method of any class in your application. This is practically useful If you want to unit test code, which was not aimed to be testable from start. Also AspectMock gives you power to write efficient code at first, and not affect production code with testing design.

Test Design

Even AspectMock proposes a flexibility in testing, it doesn’t drive you into bad application design. If you use classes globally (without injecting them) or you use static properties, methods, or singletones, it’s all right while they are defined as your internal API. Such API methods should be well documented, especially for cases, where they should be used, and where not.

If we use ActiveRecord pattern, we can assume that all models are inherited from ActiveRecord class. The only point in which our models is accessing database is save method of that class. Thus, we need only to block its call, If we don’t want the database to be hit.

<?php
test::double('ActiveRecord', ['save' => false]);
$user = new User(['name' => 'davert']);
$user->save(); // false
?>

Sure, integration testing using database gives us more reliable results. And no one ignores that fact. But unit tests allows to cover more cases, without implementing and loading fixtures. They are much faster too.

Features and Drawbacks

AspectMock may sounds cool for you, but you feel that there should be pitfalls. Let’s be honest, and list all of them here.

  • The most common issue is to get AspectMock installed. We won’t list different configuration options here, they are well documented in Github readme. But the idea is pretty simple: you should include directories with files expected to be mocked. If you don’t rely completely on autoloader from composer, you should include your autoloaders too.
  • AspectMock will slow down execution in about 20%. That’s because all methods of all classes are intercepted.

You may be curious If AspectMock affect the stack traces? The answer is no. AspectMock (starting from 0.4) does not change the line order in mocked classes, thus you get truth worthy information in stack trace. Sure, AspectMock changes those files, a bit, but more about that in next section.

Can I debug my tests when using AspectMock? And here are the good news: Sure, you can! In Debug mode you will see your classes, with no mock including in them.

To summarize: AspectMock has no side effects on unit testing process. Its magic is properly hidden to not affect the development.

Dark Magic Inside

Before implementing AspectMock into your project you might want to know, how it works in details. AspectMock is powered by Go Aop Framework.

Go AOP Framework uses php stream wrappers with filter to parse PHP files before requiring them. That may even happen in runtime. Thus, by analyzing file, we can find all its methods, and inject mocking code into it. To do so, all requiresshould include a filter. This will look like:

<?php
require 'myfile.php';
?>

will be replaced with

<?php
require 'php://read=go.source.transforming.loader/resource=myfile.php';
?>

That will make PHP file to be parsed before loading, and changed on the fly. Go AOP is pretty smart to cache already parsed files.

Now time comes for AspectMock. For every method of every class, AspectMock inserts one line into very beginning of method definition. This sample class

<?php
class User {
	
	function setName($name)
	{
		$this->name = $name;
	}	

}
?>

will be replaced with:

<?php
class User {
	
	function setName($name)
	{ if (($__am_res = __amock_before($this, __CLASS__, __FUNCTION__, array($name), false)) !== __AM_CONTINUE__) return $__am_res; 
		$this->name = $name;
	}	

}
?>

As you see only one line is added. If a stub was registered for this method, its result will be returned, and method itself won’t be invoked.

If you will enter into class in debug mode, you won’t see the line injected by AspectMock. But you will notice its there, even not show, in step by step debug.

And that’s probably all the dark magic you should be aware of. Probably it’s not too tricky and you will get along with it.

Conclusion

Unit tests are important part of testing pyramid. They are fast and they are flexible. You should not ignore them just because it may be hard for you to implement them. Its not that hard anymore. With AspectMock you can get a good code coverage with less efforts for any kind of modern php application.

Codeception 1.6.7

0
0

This release is minor, yet fixes lots of bugs you might have encountered. If you didn’t encounter them, well, then, lucky you. Still it’s a good idea to upgrade.

Here is the list of changes.

  • fix to 80 port issue, if you had problems connecting to server via PhpBrowser, then this must be it. Thanks to @tiger-seo.
  • fix in REST module when using application/json header and passing parameters.
  • seeJsonResponseContains of REST can now search on all nesting levels. Details below.
  • fix to Sequence module.
  • Step class code improved by guilhermeFranco
  • Using suite with defined namespace was improved by @Borales.
  • Generators fixes by @piccagliani and davert.

Rest Module Changes Example

added on 09/15/2013

seeJsonResponseContains behavior is less strict now. Now you it checks for any inclusion of provided json in Response, and works for unordered arrays as well. Here is an example.

<?php
$response = '{"ticket": {"title": "Bug should be fixed", "user": {"name": "Davert"}, "labels": null}}';
$I->seeResponseContainsJson(array('ticket' => array('title' => 'Bug should be fixed')));
$I->seeResponseContainsJson(array('name' => 'Davert'));
$I->seeResponseContainsJson(array('user' => array('name' => 'Davert')));
?>

In previous versions, only the first assertion would pass. But now seeJsonResponseContains will scan for inclusion on all levels of response json.

There was no other way to get it tested unordered arrays. Until 1.6.7. Here is a sample response:

[
    {
        "id": 1,
        "name": "Dalhousie Website",
        "url": "https://www.dal.ca",
        "hashed_url": "ad920603a18be8a41207c0529200af45"
    },
    {
        "id": 2,
        "name": "Wikipedia - Brasil",
        "url": "https://pt.wikipedia.org/wiki/Brasil",
        "hashed_url": "01203b7a9a2a27c22bc828419d4b7dc9"
    }
]

And test part that was failing in previous versions:

<?php
$I->seeResponseContainsJson(['url' =>  "https://pt.wikipedia.org/wiki/Brasil"]);
?>

Example taken from #330 The case here, if you didn’t know the index of element, you couldn’t verify it with seeJsonResponseContains. If you were using indexes, tests still should work for you.

Another change is checks for null values in response. Actually, we deserialized them in empty strings "", and this was a regular workaround to replace null with "" in tests as proposed in #381. But if you did that, you should update your tests for 1.6.7, as null work as expected now.

So this is the changes that might affect your tests. If you got issues. seeJsonResponseContains has much more detailed output on fails, so it would be easier to find the cause and get it fixed.

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

New Fashioned Classics: BDD Specs in PHPUnit

0
0

One of the most important requirements for tests is maintainability. The tests can live in a project for a months or even years. One day it may happen that some old tests start failing and the team got no idea about what the test does.

Does this test checks something important? Maybe specification has changed and the test should be rewritten? The team who worked on tests that early days wrote them only to make them pass. Team is not sure what was the purpose of test.

In such situations it would have been nice if a developer who wrote the test at least has left some comments. But not. Usually no one documents tests. Test passes and developer is completely satisfied with that fact.

Proper test structure and readability is the only way to maintainable tests. Tests must not turn to a legacy code. Is there a way to write better tests?

The rule, dictated by BDD, is quite simple: write tests for specifications. Do not test just methods, test the behavior. As you know, there are plenty of BDD frameworks that replace classical unit testing with specification testing. You may have heard of RSpec in Ruby, Jasmine, Mocha and others in JavaScript.

If you ever did testing in JavaScript you know how popular mentioned BDD frameworks are. Why can’t we have something similar in PHP? We got PHPSpec which is nice, but looks much different from mentioned frameworks. What if we want something more usual? Something like Jasmine in PHP?

Even if we had such BDD framework none will ever adopt it as we have PHPUnit for all kind of testing in PHP. We won’t switch PHPUnit in favor of some geeky BDD tool. But actually to write BDD-styled tests, inspired by Jasmine we don’t need to do dramatic changes.

We can use Specify, a simple trait inserted into your PHPUnit’s TestCase that allows you to store several specifications in a test and write them in BDD way.

At first we will write down some specifications in a body of typical PHPUnit’s test:

<?php
// this is just a PHPUnit's testcase
class PostTest extends PHPUnit_Framework_TestCase {

	use Codeception\Specify;

	// just a regular test declaration
	public function testPublication()
	{
		$this->specify('post can be published');
		$this->specify('post should contain a title');
		$this->specify('post should contain a body');
		$this->specify('author of post should not be banned');		
	}
}
?>

Pretty sweet, we started with describing things before the test. But can’t we do the same with comments? specify method is much better then comments as it introduces code blocks into PHPUnit.

To see it in action, let’s write the tests.

<?php
// this is just a PHPUnit's testcase
class PostTest extends PHPUnit_Framework_TestCase {

	use Codeception\Specify;

	// just a regular test declaration
	public function testPublication()
	{
		$this->post = new Post;
		$this->post->setAuthor(new User());

		$this->specify('post can be published', function() {
			$this->post->setTitle('Testing is Fun!');
			$this->post->setBody('Thats for sure');
			$this->assertTrue($this->post->publish());
		});

		$this->specify('post should contain a title', function() {
			$this->assertFalse($this->post->publish());
			$this->assertArrayHasKey('title', $this->post->errors());		
		});

		$this->specify('post should contain a body', function() {
			$this->assertFalse($this->post->publish());
			$this->assertArrayHasKey('body', $this->post->errors());		
		});

		$this->specify('author of post should not be banned', function() {			
			$this->post->getAuthor()->setIsBanned(true);

			$this->post->setTitle('Testing is Fun!');
			$this->post->setBody('Thats for sure');			

			$this->assertFalse($this->post->publish());
			$this->assertArrayHasKey('author', $this->post->errors());
		});		
	}
}
?>

This code blocks will be executed inside the same test. But it’s important to notice that each code block is isolated, thus, when an assertion inside a block fails, the test won’t stop the execution. That is how the specify is different from comments.

Now we’ve got a list of specification and code examples for each case. If one day our site will allow micro-blogging, we can easily find post should contain a body specification and remove it. That’s pretty flexible, thus maintainable.

Please note, that all the specification are grouped by context. In plain PHPUnit you would create each code block as a separate method. This way it’s pretty hard to all the tests related to one specific feature, in our case - publishing.

Ok, we got nice specifications. But can we also replace classical asserts with some more BDD stuff? Sure. We have another tiny package Verify which is also inspired by Jasmine. Assert keyword is replaced either with expect (as Jasmine does) or verify. This asserts change the order of assetion to improve readability.

Let’s rewrite our test with Verify so you could feel the difference.

<?php
// this is just a PHPUnit's testcase
class PostTest extends PHPUnit_Framework_TestCase {

	use Codeception\Specify;

	// just a regular test declaration
	public function testPublication()
	{
		$this->post = new Post;
		$this->post->setAuthor(new User());

		$this->specify('post can be published', function() {
			$this->post->setTitle('Testing is Fun!');
			$this->post->setBody('Thats for sure');
			expect_that($this->post->publish());
		});

		$this->specify('post should contain a title', function() {
			expect_not($this->post->publish());
			expect($this->post->errors())->hasKey('title');		
		});

		$this->specify('post should contain a body', function() {
			expect_not($this->post->publish());
			expect($this->post->errors())->hasKey('body');		
		});

		$this->specify('author of post should not be banned', function() {			
			$this->post->getAuthor()->setIsBanned(true);

			$this->post->setTitle('Testing is Fun!');
			$this->post->setBody('Thats for sure');			

			expect_not($this->post->publish());
			expect($this->post->errors())->hasKey('author');
		});		
	}
}
?>

Basically it’s a deal of habit. The expect(XX)->toBe(YY) style is more natural for reading as you read from left to right. But If you got used to assertXXX syntax (where the code is read from right), you can skip Verify library.


With just a few tiny libraries we converted a classical flat PHPUnit test into separate code blocks driven by specification. Test looks very similar to what we saw in Jasmine, just as we intended.

And the piece of advice to your team: introduce a rule “no assertion without specification” to make tests readable and easy to maintain.

You can install Specify and Verify via Composer:

php composer.phar require "codeception/specify:*" --dev
php composer.phar require "codeception/verify:*" --dev

To introduce specify codeblocks, just add use Codeception\Specify into any TestCase file. BDD styled assertions with expect keywords are installed automatically.

Hint: Isolated codeblocks are especially useful for testing exceptions.

Codeception 1.6.8

0
0

Yet another minor release before the 1.7 comes.

In 1.7 you will see new Selenium WebDriver module, better output formatter moved to Symfony Components, and other useful features. It will come shortly during next week(s). But for now lets list bugfixes contributed by our community members. Thank you all!

Bugfixes

  • ZF2 module fixes by Marcelo Araújo.
  • Cleanup for custom enum types in PostgreSQL module by korotovsky.
  • Disabled fields are sent by submitForm method no more. Thx to TrustNik.
  • seeResponseIsXml added to REST module by FnTm. Expect more REST/Xml features in 1.7
  • WebDriver module usage fixed.
  • Fixed unit testing results display when used with CodeGuy object
  • seeElement and dontSeeElement work correctly now.
  • Conditional Asserts work as expected now.
  • seeCookie can be used with different domain set. Thx vkn.
  • fixed submitForm for forms that don’t have submit buttons in PhpBrowser by vkn.

Translations

It’s good for you to know, that Russian and Portuguese translations of Codeception Guides were started. If you know either of this langueges (or even both, he-he) please, help us get translations done! By now several chapters are already translated, but you know, there are still lot of work,

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Codeception 1.7: WebDriver

0
0

This is new Codeception with awaited WebDriver module in it. WebDriver module is new incarnation of Selenium implementation. As it was mentioned in previous post this WebDriver module is based on facebook/php-webdriver bindings. The most valuable thing in new Selenium bindings that they can be used just the same way Selenium is used in Java. It’s very important project for PHP community, and we say “thank you” to all the Facebook behind it. One day PHP will be used in acceptance testing as widely as Java or Ruby is used.

WebDriver

WebDriver module is pretty new, yet you may want to switch to it from Selenium2. It uses just the same interface, so migration should come smoothly. If you have issues, then no hurry, stay with Selenium2 for a while. Let’s list a few features that WebDriver module has:

Implicit waits

<?php
$I->waitForText('foo', 30); // secs
$I->waitForElement('#agree_button', 30);
$I->waitForElementChange('#menu', function(\WebDriverElement $el) {
    return $el->isDisplayed();
}, 100);
$I->waitForJS("return $.active == 0;", 60);
?>

Better Keyboard Manipoluation

<?php
// <input id="page" value="old" />
$I->pressKey('#page','a'); // => olda
$I->pressKey('#page',array('ctrl','a'),'new'); //=> new
$I->pressKey('#page',array('shift','111'),'1','x'); //=> old!!!1x
$I->pressKey('descendant-or-self::*[@id='page']','u'); //=> old!!!1xu
$I->pressKey('#name', array('ctrl', 'a'), WebDriverKeys::DELETE); //=>''
?>

submitForm

A submit form method was implemented in WebDriver.

<?php
$I->submitForm('#login', array('login' => 'davert', 'password' => '123456'));
?>

Common Selenium API

Thus If you got any Selenium questions any answers on StackOverflow will help you. PHP implementation is so close to Java that you can use any answer in PHP.

You can invoke WebDriver methods directly with executeInSelenium

<?php
$I->executeInSelenium(function(\WebDriver $webdriver) {
  $webdriver->get('https://google.com');
});
?>

Symfony Output

You will notice a better output formatting in your console. That’s because we migrated to Symfony Console output. As you may know, Symfony console has 3 levels of verbosity, that can set via -v option. Codeception now support them. If you want to get all available information about test, run with -vvv option. The --debug option is now equivalent for running -vv. That’s right, you can get even more information with vvv. The output will be improved during the development of 1.7 branch. We hope to get completely different output depending on level of verbosity set.

Cest Annotations

Cests are now much smarter then they were before. If you were using StepObject you might wondered how can you pass step object class into it. That was not really obvious, as by default you get Guy class defined in config. But now you can use guy annotation to specify which guy class to use.

<?php
/**
 * @guy WebGuy\AdminSteps
 */
class AdminCest {

	function banUser(WebGuy\AdminSteps $I)
	{
		// ...
	}

}
?>

Alternatively you can use guy annotation for the method itself.

Also you can now use before and after annotations to define whic methods of Cest class should be executed before the current one. Thus you can move similar actions into protected methods and invoke them via annotations.

<?php
class ModeratorCest {

	protected function login(WebGuy $I)
	{
		$I->amOnPage('/login');
		// ...
	}

	/**
	 * @before login
	 */
	function banUser(WebGuy $I)
	{
		$I->see('Logout');
		// ...
	}

}
?>

Just by using annotations you can control the invokations of methods of the Cest class. Sure, you should define your support methods with protected, so they won’t be executed as tests themselves. Another thing worth to mention, that callbacks defined in after annotation will be called even the main test has failed, thus it makes them useful for clean ups.

We still maintain and bugfix 1.6 branch and there will be 1.6 bugfix releases. The old release 1.6.9 (yep, 1.6.9 was released to with minor bugfixes) can be downloaded from https://codeception.com/releases/1.6.9/codecept.phar.

Removed Dependencies

Optional dependencies were removed from the core, so if you use Composer version and one of the following packages:

    "behat/mink-selenium-driver": "1.1.*",
    "facebook/php-sdk": "3.*",
    "behat/mink-zombie-driver": "1.1.*",
    "videlalvaro/php-amqplib": "*"

you should include them manually in your composer.json. If you use phar version - nothing is changed for you.

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

1.7: Bugfix release

0
0

The release of 1.7 added new WebDriver module as well as rewritten Output component. Some if changes where major and was not tested for all crucial cases. If you feel comfortable with 1.6 you can stay on 1.6 branch. But If you want to get more features with some instabilities - connect to 1.7

Bugfixes in 1.7.1

  • error and failures are now displayed correctly with improved stack traces.
  • fix for module before/after hooks in Codeception\TestCase\Test.
  • select option in WebDriver throws readable message
  • wait in WebDriver throws exception when receives 1000 seconds.

Bugfixes in 1.6.10 (also in 1.7)

  • Fix the problem when the method getPort() return 443 80 or null by thbourlove.
  • Notifies about CURL not installed.

Update

redownload your codeception.phar for update:

1.7.1

wget https://codeception.com/codecept.phar -O codecept.phar

1.6.10

wget https://codeception.com/releases/1.6.10/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Codeception 1.7.2

0
0

So here is November and a new release with a minor changes is here. The development on 1.8 branch is started and you will get some of a long awaited features soon as well as Phalcon. Btw, you can already try Phalcon module for functional testing (yes, its in master branch, and not inclued in phar yet).

Bugfixes:

  • Updated to be able to use seeOptionIsSelected and dontSeeOptionIsSelected by label in framework modules by piccagliani
  • Update to appendField function to handle checkbox value or label by allmyitjason
  • Removed default delay WebDriver allmyitjason
  • better error description whenever config file could not be found by tiger-seo.
  • Added appendField function into WebDriver by allmyitjason
  • switched to use getInternalResponse in frameworks modules. Fixes “PHP Fatal error: Call to undefined method **\Response::getStatus() “ in Symfony2, Laravel4.
  • Fixed PhpBrowser module persisting HTTP authentication between tests by elazar
  • Hardcoded ‘localhost’ removed for Yii1 by kop
  • CodeCoverage works for WebDriver module as well by allmyitjason
  • Improved output formatting for unit tests by davert

Some of this fixes were also added into 1.6.

Update

redownload your codeception.phar for update:

1.7.2

wget https://codeception.com/codecept.phar -O codecept.phar

1.6.11

wget https://codeception.com/releases/1.6.10/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Working with PHPUnit and Selenium Webdriver

0
0

In this post we will explore some basics of user acceptance testing with Selenium. We will do this with classical unit testing framework PHPUnit, web browser Firefox, and with new php-webdriver library recently developed by Facebook.

Selenium allows us to record user actions that we do inside a browser and then automate them. PHPUnit will be used to do various assertions and check them for fallacy. And php-webdriver is used to connect PHP with Selenium, in order to do browser manipulation in PHP.

Probably you know, that PHPUnit itself can do Selenium manipulations via PHPUnit. There is PHPUnit_Selenium extension you may use. We will use php-webdriver instead because this implementation is modern and its API is more clean and looks much the same way as original Java Selenium API. That’s why it is easier to learn, and much powerful, then the PHPUnit’s one. For example, it allows you use send native touch events, which is important in era of mobile-ready web applications.

Grab the tools

Let’s install all the required tools using Composer. For this we will need to have composer.json file created:

{
    "require-dev": {
        "phpunit/phpunit": "*",
        "facebook/webdriver": "dev-master"
    }
}

We won’t develop any application, thus we are ok, with require-dev section only. Let’s run

php composer.phar install

and grab the latest versions of both libraries. We also will need Selenium server executable as well. You need Java installed in order to run the Selenium server. You can launch it by running this:

java -jar selenium-server-standalone-2.37.0.jar

And when the tools are prepared, let’s write some tests.

PHPUnit Test

So let’s try to test something in the web. Let’s start with something simple and well known, like Github. So, let’s create a file and call it GitHubTest.

<?php
class GitHubTests extends PHPUnit_Framework_TestCase {
}
?>

As any PHPUnit test it should extend PHPUnit_Framework_TestCase (as it was mentioned, we are not using PHPUnit_Extensions_Selenium2TestCase here). For every test we will need to launch a browser, or in other words, we are starting a Selenium session. This is done by creating RemoteWebDriver instance:

<?php
class GitHubTests extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

}
?>

This initialization part is taken from php-webdriver README file. We don’t have any test yet, so let’s write something a very basic. Something like: “If I open https://github.com, page title should contain GitHub”.

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    protected $url = 'https://github.com';

    public function testGitHubHome()
    {
        $this->webDriver->get($this->url);
        // checking that page title contains word 'GitHub'
        $this->assertContains('GitHub', $this->webDriver->getTitle());
    }    

}
?>

Now we execute our first test with phpunit

vendor/bin/phpunit GitHubTest.php

and in a few seconds we should see a Firefox window with Github Page in it

Opening web page with PHPUnit and Selenium

please notice the WebDriver text in the status bar, this tells you that this browser window is controlled by WebDriver.

In a console we will see this output:

PHPUnit 3.7.28 by Sebastian Bergmann.


.

Time: 19.44 seconds, Memory: 1.75Mb

OK (1 test, 1 assertion)

We will see that test has finished, but the browser window stays opened. That is because we did not implement a tearDown method, that should be used to close the webdriver session:

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->quit();
    }
}
?>

Advanced Test

We didn’t touch any of page elements in a previous test. We just opened the page and checked its title. But the power of webdriver reveals when you want to click elements, fill forms, drag and drop elements, etc. That’s why we will write a test that demonstrates some of this features.

But how can control the browser? Should we move the mouse in order to click on element? Well, not exactly. WebDriver allows us to locate element on page by its ID, class name, element name, CSS, or XPath. Let’s list all possible locator types, taken from WebDriverBy class:

  • WebDriverBy::className() - searches for element by its CSS class.
  • WebDriverBy::cssSelector() - searches for element by its CSS selector (like jQuery).
  • WebDriverBy::id() - searches for element by its id.
  • WebDriverBy::linkText() - searches for a link whose visible text equals to the value provided.
  • WebDriverBy::partialLinkText() - same as above, but link partly contain the value.
  • WebDriverBy::tagName() - search for element by its tag name.
  • WebDriverBy::xpath() - search for element by xpath. The most complex, yet, most powerful way for element location.

To find element we should use webDriver->findElement method, with locator specified with WebDriverBy.

After the matched element is found we can click on it. Like this:

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->close();
    }

    public function testSearch()
    {
        $this->webDriver->get($this->url);
        // find search field by its id
        $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
        $search->click();
	}    
}
?>

Locating Web Element in Firefox

We are clicking the GitHub global search field, located in top menu bar, matched by its id. By the way, how did we get the element’s id? That’s a good question. Searching for element locators is the most important task in acceptance testing. For every test we need to get the elements that are involved in it. Let’s show some simple tricks that will definitely help you in writing complex acceptance tests.

Locating Elements: Tips & Tricks

The first thing we can do is to pause the test execution. While browser window is still open, we can use it to find the locator. To pause the test execution lets write this helper method inside a test class:

<?php
    protected function waitForUserInput()
    {
        if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
    }
?>

If we use it somewhere in our tests, PHPUnit will wait for Enter key pressed in console before going on.

<?php
    public function testSearch()
    {
        $this->webDriver->get($this->url);
		$this->waitForUserInput(); // paused       
    }    
?>

Now when the browser window is opened we are free to search for required element with no hurry. We are using Firfox Developer Tools for that. With the Element Inspector within we can point to element and get its unique CSS locator.

Using WebDriver with PHPUnit to test GitHub

That is how we got search field id: #js-command-bar-field. Doing the sample steps, let’s continue writing our test and find php-webdriver repository on GitHub.

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    protected $url = 'https://github.com';
    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->close();
    }

    public function testSearch()
    {
        $this->webDriver->get($this->url);
        
        // find search field by its id
        $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
        $search->click();
        
        // typing into field
        $this->webDriver->getKeyboard()->sendKeys('php-webdriver');
        
        // pressing "Enter"
        $this->webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
        
        $firstResult = $this->webDriver->findElement(
            // some CSS selectors can be very long:
            WebDriverBy::cssSelector('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)')
        );
        
        $firstResult->click();
        
        // we expect that facebook/php-webdriver was the first result
        $this->assertContains("php-webdriver",$this->webDriver->getTitle());
        
        // checking current url
        $this->assertEquals(
        	'https://github.org/facebook/php-webdriver', 
        	$this->webDriver->getCurrentURL()
        );
    }


    protected function waitForUserInput()
    {
        if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
    }

}
?>

If we run this test we will see that it is failing on the last step:

PHPUnit Test is Failing on Assertion

That’s because we forgot that GitHub uses https by default, and GitHub is a company and not a non-profit organization (as we used to think of it, he-he). Though let’s change the expected url to ‘https://github.com/facebook/php-webdriver’ and see the test is passing.

Element Not Found

Probably we will also want to check if element is located on a page. If we were using Selenium2TestCase of PHPUnit, we would have several nice assertion that we can use just for that. In case of php-webdriver library we will need to implement them on our own. But that’s pretty easy. Php-Webdriver throws various exceptions, which we can handle and transform into PHPUnit’s assertions:

<?php
    protected function assertElementNotFound($by)
    {
        $els = $this->webDriver->findElements($by);
        if (count($els)) {
            $this->fail("Unexpectedly element was found");
        }
        // increment assertion counter
        $this->assertTrue(true);        
    }

?>

You can create similar assertion just in the same manner.

We will use newly created assertElementNotFound method to check that there is no user avatar on “facebook/php-webdriver” page.

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    protected $url = 'https://github.com';
    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->close();
    }

    public function testGitHubHome()
    {
        $this->webDriver->get($this->url);
        // checking that page title contains word 'GitHub'
        $this->assertContains('GitHub', $this->webDriver->getTitle());
    }

    public function testSearch()
    {
        $this->webDriver->get($this->url);

        // find search field by its id
        $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
        $search->click();

        // typing into field
        $this->webDriver->getKeyboard()->sendKeys('php-webdriver');

        // pressing "Enter"
        $this->webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);

        $firstResult = $this->webDriver->findElement(
            // some CSS selectors can be very long:
            WebDriverBy::cssSelector('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)')
        );

        $firstResult->click();

        // we expect that facebook/php-webdriver was the first result
        $this->assertContains("php-webdriver",$this->webDriver->getTitle());

        $this->assertEquals('https://github.com/facebook/php-webdriver', $this->webDriver->getCurrentURL());

        $this->assertElementNotFound(WebDriverBy::className('avatar'));

    }

    protected function waitForUserInput()
    {
        if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
    }

    protected function assertElementNotFound($by)
    {
        $els = $this->webDriver->findElements($by);
        if (count($els)) {
            $this->fail("Unexpectedly element was found");
        }
        // increment assertion counter
        $this->assertTrue(true);
        
    }
}
?>

Refactoring

To clean up some things we will separate test methods and support methods. It is a good idea to move custom assertions into trait: WebDriverAssertions. And the pause switcher waitForUserInput can be moved into WebDriverDevelop trait. We can enable this trait in a test class, when we develop a test, and turn it off once we finished.

The complete demo project, after this basic refactoring, you can find on GitHub.

And what about Codeception?

So you did notice that this is Codeception blog. But we didn’t use Codeception framework at all in this tutorial. Sure, we need to mention, that Codeception includes php-webdriver library and WebDriver module out of the box starting from version 1.7. In Codeception you can perform all the web manipulations in a much simpler manner using the WebGuy APIs. If you use Codeception you don’t need to implement your own WebDriver assertions nor write boilerplate code.

Conclusions

No matter you are using Codeception or not it is a good idea to understand how to perform browser based acceptance testing using just the php-webdriver by itself. Php-webdriver library provides very clean and flexible APIs you will enjoy working with.

PHP is a language popular for web development, but not for web testing. Test automation engineers prefer Java and Ruby over it. And there is a serious reason for that. There is no such thing like “official Selenium bindings” for PHP, i.e. there is no Selenium client library for PHP created by Selenium team. Developers of php-webdriver get very close to the official Selenium client APIs. That’s why you should use php-webdriver - it really feels and works like native and official Selenium libraries. That is especially important if you have an experience writing acceptance tests in Java or Ruby. Moving to PHP is not that hard, when all the APIs are the same.

Codeception 1.8: Phalcon, Environments, DataProviders

0
0

This release brings lots of changes. Finally we got working DataProviders (the issue was opened for about a year), and @depends tag. But the details below. Let’s start with the most important new features:

Phalcon Framework Support

Phalcon

Phalcon is the fastest PHP framework in the world. And that’s not just a marketing slogan. Phalcon was developed as C extension, so it is already precompiled and loads into memory only once. Phalcon is modern framework with DI container, ORM (inspired by Doctrine), and templating engine (inspired by Twig). It is one of the most innovative projects in PHP world, so you should at least check it out.

If you already work with Phalcon, here is a good news for you. Codeception got Phalcon1 module, that allows you to write functional tests with minimum setup. Besides standard framework interface this module provides actions for session and database access. It also wraps all tests into nested transactions, and rollbacks them in the end. Thus, all your database tests run superfast.

Is there any Phalcon project tested with Codeception? Yes, it is the official Phalcon’s forum.

This module was developed by cujo and improved by davert. We hope you like it.

Environments

This is something you might not expect in the form it was produced, but, probably, that was long awaited. This is to run tests multiple times over different environments. The most common issue is running acceptance tests over different browsers: firefox, phantomjs, chrome. Now you can create 3 different configurations and you can get tests will be repeated 3 times: in firefox, in phantomjs, and in chrome. The second usecase is run tests over different databases.

Feature is pretty straightforward in use. You define the name of environment below the env key, and then you redefine any of configuration values you need.

``` yaml
class_name: WebGuy
modules:
    enabled:
        - WebDriver
        - WebHelper
    config:
        WebDriver:
            url: 'http://127.0.0.1:8000/'
            browser: 'firefox'

env:
    phantom:
         modules:
            config:
                WebDriver:
                    browser: 'phantomjs'

Advanced Usage chapter was updated.

DataProviders

You can use PHPUnit dataproviders in Codeception\TestCase\Test files. Yep. Finally.

Probably DataProviders are not really readable, as you need always to refer into data sets, which may be defined in the different part of a testcase. You can consider using examples of Codeception\Specify library, as for alternative for dataproviders.

Is there a way you can use data providers in scenario driven test? Not exactly, but you can emulate them with loops and conditional asserts:

<?php
foreach ($posts as $post) {
	$I->canSee($post->title,'.post h2');
}
?>

Depends

Declare depending tests in Cest and Test files. Works just as the original @depends of PHPUnit. In Cests you can combine this with @before annotation. More information in Advanced Usage.

Debug

Debug output was refactored, and moved out to Codeception\Util\Debug class. This class can be used globally, i.e in tests, helpers, - wherever you want. To print debug information you should call:

<?php
use Codeception\Util\Debug;
Debug::debug("This is working");
?>

This change dramatically improves debug output. You can also pause execution with pause static method of this class. Useful for debugging and tests development, implemented in WebDriver module as pauseExecution action.

Bugfixes and Minor Changes

  • WebDriver module got pauseExecution method which pauses running test in debug mode.
  • Generated PageObject file URL const was changed to static variable.
  • waitForElementChange() callback return value was not being used by wheelsandcogs also in 1.7
  • bugfix for making screenshots with WebDriver by Michael Wang. also in 1.7
  • Doctrine2: handle NULL value in seeInRepository param array by imanzuk also in 1.7
  • REST: fix adding parameters to url to non-GET HTTP methods by sheershoff
  • REST: Added sendOPTIONS() and sendHEAD() requests for CORS testing by elazar
  • Symfony2: fixed usage of profiler
  • Strict declaratin error fixes for framework constraints also in 1.6, 1.7

Update

Warning. Module Unit was deprecated in 1.6 and was removed in 1.8. Please disable it in unit.suite.yml if you were using it. If you see this error:

Codeception PHP Testing Framework v1.8.0
Powered by PHPUnit 3.7.28 by Sebastian Bergmann.



  [Codeception\Exception\Configuration]
  Unit could not be found and loaded

Just disable Unit module. Thanks

If you prefer stability over features you can stay on 1.7 or 1.6 releases. We’ve got them updated too.

redownload your codeception.phar for update:

1.8.0

wget https://codeception.com/codecept.phar -O codecept.phar

1.7.3

wget https://codeception.com/releases/1.7.3/codecept.phar -O codecept.phar

1.6.12

wget https://codeception.com/releases/1.6.12/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

What to expect from 2.0

It is almost 2 years of Codeception, and we are planning to release 2.0 version as a next major to celebrate that. It is a major change, thus we can add few BC breaks. We are planning to:

  • move to PHP 5.4. Not really necessary, yet it is getting hard to support 5.3.
  • remove Mink entirely in favor of WebDriver for browser-testing and Goutte for headless testing.
  • remove 2-times test execution (finally!).
  • ??? (proposed by you)

Selenium WebDriver tests with Codeception

0
0

Last time we discovered API of new WebDriver PHP bindings. We wrote a basic test in PHPUnit that uses Selenium and Firefox browser to find php-webdriver library on Github. Today we will reimplement the same test with Codeception.

Installation

Depending on your preferences you can install Codeception either by downloading codecept.phar archive from the site, or alternatively by using composer.

{
    "require-dev": {
        "codeception/codeception": "*",
    }
}

With composer you will need to execute:

php composer.phar install

In previous tutorial we did installation using Composer, so in current examples we will be using it as well.

Sure, we also need Selenium server executable as well. You need Java installed in order to run the Selenium server. You can launch it by running this:

java -jar selenium-server-standalone-2.37.0.jar

When all installation steps are done, we can continue with creating Codeception bootstrap.

Bootstrap

Unlike the phpunit Codeception requires a small bootstrap step. Codeception helps you to organize tests into 3 categories: acceptance, functional, and unit tests. To create all tests and support directories, you will need to run the bootstrap command.

vendor/bin/codecept bootstrap

Selenium tests are acceptance tests. So let’s create a skeleton for the basic acceptance test:

vendor/bin/codecept generate:cept acceptance GitHub

This will create new file in tests/acceptance/GitHubCept.php. But we need some additional configuration to be done before proceeding. Instead of creating webdriver session in tests manually, we delegate this to Codeception. Codeception will take care for creating session before each test, and closing it after. That’s why Selenium configuration should be written into tests/acceptance.suite.yml.

class_name: WebGuy
modules:
    enabled:
        - WebDriver
        - WebHelper
    config:
        WebDriver:
            url: 'https://github.com'
            browser: 'firefox'

Each time you change configuration in Codeception you should run the build command.

vendor/bin/codecept build

Writing a Test

Let’s start with something very basic. We will open the Github page and we will make sure that GitHub word is within the page title.

<?php
$I = new WebGuy($scenario);
$I->wantTo('see GitHub word in title ');
$I->amOnPage('/');
$I->seeInTitle('GitHub');
?>

We are using the wantTo command just to give a test clean description. amOnPage command opens browser on the github home page, and all the commands that start with see are assertions you can use. There are lots of commands in WebGuy class you can use writing the test. All them are taking from corresponging modules, in our case it is WebDriver module. If you use IDE you can check them all with autocompletion.

Codeception Autocomplete

But let’s execute a test with run command:

vendor/bin/codecept run

And you will see this output:

Codeception PHP Testing Framework v1.8.0
Powered by PHPUnit 3.7.28 by Sebastian Bergmann.

Functional Tests (0) ------------------------
---------------------------------------------

Acceptance Tests (1) -------------------------------------------
Trying to see github word in title  (GitHubCept.php)       Ok
----------------------------------------------------------------

Unit Tests (0) ------------------------------
---------------------------------------------


Time: 21.62 seconds, Memory: 5.00Mb

OK (1 test, 1 assertion)

You may have noticed that Codeception is itself powered by PHPUnit. Basically you can execute native PHPUnit tests inside Codeception, but the main idea of Codeception is scenario driven tests written from an eye of a tester. Each test should descibe user’s actions in simple words: I see, I click, etc. Let’s see how using just this simple terms with no OOP complexity we can write a bit more complex test.

Our test should open github in a browser and use the search form to get to “facebook/php-webdriver” library page. Before writing the test we did some research of GitHub page to find selectors we can use to match elements on page. This was described in previous tutorial.

<?php
$I = new WebGuy($scenario);
$I->wantTo('find facebook/php-webdriver on GitHub');
$I->amOnPage('/');
$I->fillField('#js-command-bar-field','php-webdriver');
$I->pressKey('#js-command-bar-field', WebDriverKeys::ENTER);
$I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
$I->seeInTitle("php-webdriver");
$I->seeCurrentUrlEquals('/facebook/php-webdriver');
?>

You can execute this test and see it passes. Unlike the classical PHPUnit test, this code does not require comments. It is very clean and easy to understand and edit by anyone a team with basic PHP/HTML/CSS skills.

Tips and Tricks

In this section we will share some ideas, that you can use in your tests. Sure that would not be the same tips we have in PHPUnit. But in PHPUnit test we wrote a helper function to pause the execution. This helped us to search for required locators. In Codeception (starting from 1.8) we can use $I->pauseExecution(); method for this. The test will stop the scenario in that place and wait for Enter to be pressed. Worth to mention, that pauseExecution works only in debug mode (sure, you don’t want to freeze when running tests on CI). So you should pass --debug option when using it.

But even Codeception provides us with dozen of predefined commands, we might want to access webdriver session by your own. For this we have a executeInSelenium command:

<?
$I->executeInSelenium(function(RemoteWebDriver $webDriver) {
	$webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
});
?>

But that doesn’t make our test more readable. What can we do to simplify it a bit? Well, let’s replace this double search for ‘#js-command-bar-field’ input field. We can use submitForm command that works for forms on any kind, and you can pass array of field values into it. Current form id is top_search_form, so we can rewrite our test like:

<?php
$I = new WebGuy($scenario);
$I->wantTo('find facebook/php-webdriver on GitHub');
$I->amOnPage('/');
$I->submitForm('#top_search_form', array('q' => 'php-webdriver'));
$I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
$I->seeInTitle("php-webdriver");
$I->seeCurrentUrlEquals('/facebook/php-webdriver');
?>

When running WebDriver tests with Codeception they may look a bit slower then the same tests run with PHPUnit. That’s because Codeception has default delay of 5 seconds for each test action on page. This delay can be set to 0 in order to run test faster. This is configured in acceptance.suite.yml

class_name: WebGuy
modules:
    enabled:
        - WebDriver
        - WebHelper
    config:
        WebDriver:
            url: 'https://github.com'
            browser: 'firefox'
            wait: 0

This should execute the same test 30% faster then before. But if your page use JavaScript and Ajax, you should increase the wait time or use various waitFor commands of WebDriver.

Conclusion

At first sight setting up Codeception Selenium test may look a bit harder then with PHPUnit. At first sight it’s hard to see real benefits in it. But the key idea of Codeception is to separate the test code and support code. So you should write only tests, and most of helper methods was already written for you. Thus, the test is kept clean and readable. It is easy to change and easy to manage. If you want to switch a browser, you should not edit a test, but change the configuration. You want to execute test in 2 browsers one by one? No problems, just change a configuration.

Codeception is very flexible framework that you can use to write your Selenium tests. After all it’s really simple and fun.

Source file of this tutorial are available on GitHub.

Codeception 1.8.1: Minor updates

0
0

Codeception 1.8.1 is out. Bugfixes and small useful features in it. The most interesting improvement was done by frqnck. Phar version now has self-update command which acts the same way as it is in composer. From now on you can easily upgrade codeception.phar file.

Small yet important change in WebDriver module. Default wait parameter is set to 0. This was done because Selenium implicit waits didn’t work as expected - this parameter slowed down test execution. Browser waited for element even if it was already on a page. Please notice this on upgrading.

Changes

  • upgraded to php-webdriver 0.3
  • added general sendAjaxRequest() method to all framework/phpbrowser modules to send ajax requests of any types by gchaincl.
  • fixed URI construction in Yii1 module by kop(also in 1.7)
  • Fixed Yii2 statusCode by cebe
  • fixed: Placeholder\Registry::unsetRegistry() should only be used with < 2.2.0 by Bittarman
  • waitForElementChange() was fixed in WebDriver module
  • fixed seeLink and dontSeeLink methods in framework modules by enumag.
  • added seeHttpHeaderOnce to REST module for checking if headers appear only once.
  • setUpBeforeClass/tearDownAfterClass method will work as they are expected in PHPUnit
  • Debug::debug can output any variable in console.
  • fixed: “WebDriver makeScreenshot doesn’t create directories” by joksnet
  • fixed grabValueFrom method in framework modules (also in 1.7)
  • fixed: “Disable remote coverage not work” by tiger-seo (also in 1.6, 1.7)
  • self-update command added to phar by frqnck

Dependencies were updated, thus Symfony components were updated to 2.4. No changes to composer.json was made, so this release is compatible with Symfony 2.3 as well.

Update

redownload your codeception.phar for update:

1.8.1

wget https://codeception.com/codecept.phar -O codecept.phar

1.7.4

wget https://codeception.com/releases/1.7.4/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Testing Emails in PHP. Part 1: PHPUnit

0
0

So how do you check that your applications sends email correctly? It looks like dealing with emails is always a challenge. How would you verify that an email message is formatted and delivered correctly, without actually sending them to your clients? That’s the first question. And the second question is: how can we automate the testing of emails?

For both questions we have an answer. There are two awesome services that have been developed to help developers in dealing with email hell. They are Mailtrap and Mailcatcher. Both services run an SMTP server that does not deliver emails, but stores them locally. They both have a web interface in which you can review all the outgoing emails. The difference between these services are: mailtrap runs as a web service, and mailcatcher is a ruby gem that can be installed locally.

mailcatcher

It’s up to you which one to use. Definitely they will simplify your life while developing a web application. Do they have something to offer for testing? Sure! We can access all handled emails via REST API and verify our assertions.

In this post we will marry Mailcatcher with the PHPUnit testing framework. We’ve chosen Mailcatcher so we do not have to rely on a 3rd-party web service and have all the tests run locally. We will write methods for both PHPUnit and Codeception in order to provide different solutions and compare them.

Before we start we need to make sure that Mailcatcher is installed and running. When done you can access it’s web interface on port 1080 and use port 1025 for the fake SMTP server. Configure your web application to use exactly that port when running in test environment.

Testing emails in PHPUnit

Mailcatcher has a really simple REST API that is used for email access. Here is a quote from their official site.

A fairly RESTful URL schema means you can download a list of messages in JSON from /messages, each message’s metadata with /messages/:id.json, and then the pertinent parts with /messages/:id.html and /messages/:id.plain for the default HTML and plain text version, /messages/:id/:cid for individual attachments by CID, or the whole message with /messages/:id.source.

What was not mentioned was that you can also clear all emails by sending DELETE request to /messages. The most complete documentation on API is its code. Even if you don’t know Ruby, it is quite easy.

Thus, we will need to send GET and DELETE requests and parse the json response. To send them we will use the Guzzle framework. PHPUnit and Guzzle can be easily installed via Composer:

{
    "require-dev": {
    	"phpunit/phpunit": "*",
    	"guzzle/guzzle": "~3.7"
    }
}

Let’s create EmailTestCase file and place MailCatcher API calls into it.

<?php
class EmailTestCase extends PHPUnit_Framework_TestCase {

    /**
     * @var \Guzzle\Http\Client
     */
    private $mailcatcher;

    public function setUp()
    {
        $this->mailcatcher = new \Guzzle\Http\Client('http://127.0.0.1:1080');

        // clean emails between tests
        $this->cleanMessages();
    }

    // api calls
    public function cleanMessages()
    {
        $this->mailcatcher->delete('/messages')->send();
    }

    public function getLastMessage()
    {
        $messages = $this->getMessages();
        if (empty($messages)) {
            $this->fail("No messages received");
        }
        // messages are in descending order
        return reset($messages);
    }

    public function getMessages()
    {
        $jsonResponse = $this->mailcatcher->get('/messages')->send();
        return json_decode($jsonResponse->getBody());
    }
?>    

That’s enough to fetch a list of all delivered emails. All the emails will be cleaned between tests, so each test will be executed in isolation. Let’s implement some assertion methods to check the sender, recipient, subject and body of the email.

<?php
    // assertions
    public function assertEmailIsSent($description = '')
    {
        $this->assertNotEmpty($this->getMessages(), $description);
    }
    
    public function assertEmailSubjectContains($needle, $email, $description = '')
    {
        $this->assertContains($needle, $email->subject, $description);
    }

    public function assertEmailSubjectEquals($expected, $email, $description = '')
    {
        $this->assertContains($expected, $email->subject, $description);
    }

    public function assertEmailHtmlContains($needle, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.html")->send();
        $this->assertContains($needle, (string)$response->getBody(), $description);
    }

    public function assertEmailTextContains($needle, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.plain")->send();
        $this->assertContains($needle, (string)$response->getBody(), $description);
    }

    public function assertEmailSenderEquals($expected, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.json")->send();
        $email = json_decode($response->getBody());
        $this->assertEquals($expected, $email->sender, $description);
    }

    public function assertEmailRecipientsContain($needle, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.json")->send();
        $email = json_decode($response->getBody());
        $this->assertContains($needle, $email->recipients, $description);
    }
?>

The complete code listing is published as gist.

Example

How might a test using this EmailTestCase look?

<?php		
	function testNotificationIsSent()
	{
		// ... trigger notifications

        $email = $this->getLastMessage();
        $this->assertEmailSenderEquals('<bugira@bugira.com>', $email);
        $this->assertEmailRecipientsContain('<davert@ukr.net>', $email);
        $this->assertEmailSubjectEquals('[Bugira] Ticket #2 has been closed', $email);
        $this->assertEmailSubjectContains('Ticket #2', $email);
        $this->assertEmailHtmlContains('#2 integer pede justo lacinia eget tincidunt', $email);
    }
?>    

We now have a really simple class for testing emails from your application. Ok, that’s not a unit test. For unit testing you should use a mocking framework to fake the delivery in your PHP code. But you can use this class in acceptance tests (with Selenium) or integration tests. It is much simpler to test emails this way, than to dig into the internals of your email sending library and define mocks. The drawbacks here are the usage of standalone daemon, and reconfiguring your application to use its SMTP server.


It looks like this post is long enough to be published. We will continue email testing next time with Codeception framework. We will develop EmailHelper class for scenario-driven tests of Codeception.

EmailTestCase Source


Codeception 1.8.2: Bugfixes

0
0

Time passed since the previous release. Since 1.8.1 we got a nice list of significant bugfixes and we are ready to publish new stable release. No new features added, actually, but they are coming in new 2.0 branch which is actively developed in this days.

Changes:

  • [REST] match similar elements in JSON arrays fixed in #837 by blacknoir.
  • generate:pageobject now takes -c option correctly #809
  • [REST] Fixed setting Content-Type header #827
  • [REST] Headers are uppercased according to BrowserKit standard.
  • [Db] records inserted with haveInDatabase now are cleaned in _after event by dirk-helbert #761.
  • [Laravel] Fixed usage of Redirect::back in tests
  • Fixed collecting CodeCoverage using WebDriver/Selenium2 modules.
  • [REST] Fixed “Call to undefined method Symfony\Component\HttpFoundation\Request::getCookies() in codeception/codeception/src/Codeception/Module/REST.php line 352” by casconed #814
  • Fixed: tests run twice if you use .dist.yml config by tomtomsen #582
  • Environments: Test classes was loaded only once in multi-environment mode. Fixed by ayastreb and his nice tokenizer solition #812
  • Excluding running abstract classes in addCest by filipgorny #792
  • [PhpBrowser] Fixed setting cookies from headers
  • [Framework] Form data on page was not cleaned after form submit. So when sending the same form twice, cached data was submitted. This is now fixed.
  • and others…

Also we’ve got a Mockery module for using powerful Mockery framework with Codeception. It is not included in Codeception itself but can be installed via Composer. (Thanks to Jáchym Toušek)

Thanks to all contributors, thanks for getting bug fixed and reported. We are trying to be better with each release.

Update

redownload your codeception.phar for update:

1.8.2

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

What’s Next

We are moving to Codeception 2.0. It’s first alpha is expected on next week. Stay tuned and get ready for new features!

While developing Codeception 2.0 we released a lightweight task runner Robo. We needed it to run routine tasks for Codeception - building phar archives, merging releases, etc. You may use it as lightweight alternative for Phing or PHP alternative for shell scripts. It’s not very documented, it doesn’t include list of all required tasks, but it will evolve. If you have ideas on improvement, or you want to add more tasks into it - please send Pull Requests.

And thanks to @pfaocle for the feedback on using Robo!

Codeception 1.8.3: Laravel and Yii2 DB actions

0
0

Here goes another minor release with some fixes and improvements. Codeception 1.8 now supports Laravel 4.1 and Yii2 and is tested for this frameworks on Travis. Also Laravel and Yii modules got some nice new actions for database interactions.

Laravel, Yii2, and Phalcon frameworks implement ActiveRecord pattern. That’s why all database actions in this modules look and work in a very similar manner.

Laravel 4.1

<?php
$user_id = $I->haveRecord('users', array('name' => 'Davert'));
$I->seeRecord('users', array('name' => 'davert'));
$I->dontSeeRecord('users', array('name' => 'davert'));
$user = $I->grabRecord('users', array('name' => 'davert'));
?>

This methods will work for Laravel 4 as well, but Laravel 4.1 supports nested transactions and allows us to wrap functional test into one database transaction. This is really useful, as we can rollback any database changes we do in functional tests. Tests also run really fast, as nothing is written in database. A new cleanup config option was introduced to Laravel4 module, and by default it is turned on.

Now it is really simple to use database in your functional tests. Don’t hesitate and try *Record methods in action!

Also nice seeSessionErrorMessage was added by elijan to perform validation error assertions.

Yii2

Yii2 is in very active development, and its official basic application is tested with Codeception, and uses Specify and Verify libraries. Yii2 module is tested on Travis as in official Yii2 repo and in Codeception repo as well.

<?php
$user_id = $I->haveRecord('app\model\users', array('name' => 'Davert'));
$I->seeRecord('app\model\users', array('name' => 'davert'));
$I->dontSeeRecord('app\model\users', array('name' => 'davert'));
$user = $I->grabRecord('app\model\users', array('name' => 'davert'));
?>

ActiveRecord methods work in very similar manner. We expect that Yii2 will have nested transactions support before the release.

Thanks to Ragazzo for Yii2 module contributions and bughunting.

Bugfixes

  • CodeCoverage was improved. Remote codecoverage with WebDriver or PhpBrowser opens page, sends authorization cookie before the test.
  • WebDriver cookies (and sessions) are destroyed after the test. If you have troubles when session is not restared in WebDriver - toggle restart option.

Update

redownload your codeception.phar for update:

1.8.3

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

P.S. Yeah, yeah, Codeception 2.0 is on its way.

Codeception 2.0 alpha

0
0

Finally we are ready to show you Codeception 2.0. We tried not to break everything (as it was supposed for major version change), but keep things work as they are, but maybe in a different way. Let’s review the most important changes:

Codeception: Not Just For Guys

Before 2.0 there were only Guys in it. That is not fair! We wanted to make Codeception a tool for everyone: for guys, girls, developers, test engineers, ninjas and even wookiees… (nope, wookiees should wait for 3.0).

That’s why you can now choose the desired actor from a list during the bootstrap process.

Before proceed you can choose default actor:

$I = new [ACTOR]
  Select an actor. Default: Guy  
  [0] Guy
  [1] Girl
  [2] Person
  [3] Engineer
  [4] Ninja
  [5] Dev

so you all your test actors (that’s how we call guy classes now) will be in form like TestGirl, WebNinja, CodeDev, etc. Pretty flexible.

Codeception: Not Alone

Before Codeception 2.0 guys (or should we say actors now) were left to themselves. You know, it is so sad to see that there is only $I in the test, forever alone, like on a desert island, or in space… But in 2.0 you can invite some friends into your tests. Let’s say…

<?php
$I = new WebGuy($scenario);
$nick = $I->haveFriend('nick');
?>

So we can write multi-session tests that can be executed in two browser windows. You may try to run this test on github.com to see how it works:

<?php
$I = new WebGuy($scenario);
$I->wantTo('surf Github with Nick');
$I->amOnPage('/');
$I->submitForm('#top_search_form', array('q' => 'php-webdriver'));
$nick = $I->haveFriend('nick');
$nick->does(function(WebGuy $I) {
    $I->amOnPage('/Codeception/Codeception');
    $I->click('Issues');
    $I->canSeeInTitle('Issues');
});
$I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
$I->seeInTitle("php-webdriver");
$nick->does(function(WebGuy $I) {
    $I->click('Milestones');
    $I->canSeeInTitle('Milestones');
});
$I->seeCurrentUrlEquals('/facebook/php-webdriver');
$I->click('Issues');
$I->canSeeInTitle('Issues');
?>

As you see, everything in does closure is executed in separate session. This way you can test user-to-user interactions on your site. For example Alice writes a private message to Bob and Bob responds back. Similarly you can have multiple REST sessions in a test.

Such scenario cases can be implemented. That’s what friends are for.

Notable Changes

As it was announced earlier, the main goals for Codeception 2.0 was internal refactoring.

  • Mink (and its drivers) was removed completely. Instead you can use WebDriver module to do Selenium testing, and PhpBrowser (which uses Goutte) for browser-emulation. PhpBrowser module is now more compatible with frameworks modules, they use the same methods and acts in the same manner. If you were using Selenium or Selenium2 module you should switch to WebDriver, for PHPBrowser everything should (crossing fingers) work smoothly.
  • 2-phases test execution with tricky magic including usage of Maybe class was removed. Tests are now executed one time, like any regular PHP file. So you can use any PHP code in your tests, and appearance of Maybe object would not confuse you anymore.
<?php
$card = $I->grabTextFrom('#credit_card')
var_dump($card); // was showing `Maybe` instead of real value
?>
  • Codeception 2.0 require PHP 5.4 and higher. Time changes, PHP 5.3 is getting harder and harder to support, thus we decided to move to 5.4 and keep our code base up to date. And yes, we wanted to use short array syntax. We are tired to keeping write all those nasty array() stuff.
  • Actor classes (initial Guy classes) are now rebuilt automatically. Thus, you want get exception when change suite configuration, or add methods to helper. Rebuilds are not required anymore.
  • Added Assert module that can be used to write common asserts in your tests. You can now use seeEquals, seeContains and other actions inside your Cepts or Cests.
  • Experimental: added Silex module. We need your Feedback on using it.

Minor Internal Changes

Refactoring, Refactoring, Refactoring. We use PSR-2 now. We rewrote CodeCoverage. We have better directory structure… More new files. What else? Oh, lets admit it, these are not the changes you would actually notice. But internals are now more clean and easy to understand (Except the parts which heavily rely on PHPUnit).

Upgrading

We’d like to ask you to try Codeception 2.0 on your projects. Before the release is ready we need to collect feedback and fix all encountered issues. You know where GitHub issues are.

Download:

wget https://codeception.com/releases/2.0.0-alpha/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-alpha" 

Development of 2.0 now happens in master branch, so keep track on changes and send your pull requests.

Some Upgrading Notes

  • Run build command
  • Replace Selenium2 to WebDriver module
  • Check you don’t use PHPBrowser->session property anywhere (it was Mink part)
  • CodeCoverage with c3 will require new version of c3.

What’s next?

We need your feedback, and meanwhile we will work on updating documentation parts. 1.8.x will be maintained, but new features will be added to 2.x branch.

Codeception 2.0 beta

0
0

It took about a month to get to the beta release. This release brings lots of changes even a few BC breaks and clearly shows what Codeception 2.0 is planning to be. Today we will announce more technical changes, and not conceptual ones. As it was stated previously: Codeception 2.0 will bring lots of small improvements, will be much easier to extend, and understand. Changes for beta release were not planned initially, but were added to help Codeception evolve during the 2.0 branch development. Let’s list briefly all major changes and see how to deal with them. Documentation for 2.0 branch can be found on GitHub and will be published on RC release.

Changes

  • Upgraded to PHPUnit 4.0
  • Upgraded to facebook/webdriver 0.4
  • RunFailed extension added. To rerun tests on fail include Codeception\Platform\RunFailed extension, and call codecept run -g failed.
  • Logger disabled by default and moved to Extension. You will need to install Monolog by yourself, and enable Codeception\Platform\Logger as extension.
  • Methods _before/_after of Cest can use $I object.
  • Destination for xml/html/reports can be customized. For instance
codecept run --xml myreport.xml
codecept run --xml /home/davert/report.xml
codecept run -g database --xml database.xml
```
  • --coverage + --xml or --html won’t produce xml or html codecoverage output. Use new options coverage-xml and coverage-html instead. They were added so you could specify custom destination for codecoverage reports as well.
  • you can get current environment in a test by accessing $this->env (even in Cept)
  • shortcut functions added: codecept_debug to print custom output in debug mode, codecept_root_dir, codecept_log_dir, codecept_data_dir for retrieving Codeception paths.
  • extensions can change global config before processing.

Breaking Changes

In Codeception 1.x bootstrap files were used differently for different types of tests. They were introduced to pass variables (and fixtures) into Cept scenarios. They were almost useless in Cests, and absolutely useless for Test files. They are not removed at all, but now they are loaded only once before the suite. If you were using bootstrap files for setting fixtures, you still can use them, but you will need to require them manually.

<?php
require __DIR__ . '/_bootstrap.php'

$I = new WebGuy($scenario);
?>

The same way you can load bootstrap files into Cests. But we recommend to create _fixtures.php file, and use bootstrap file for suite initialization.

Dynamic Groups

One significant internal feature is dynamic groups. You can save test names into file and run them inside a group. That’s how the RunFailed extension works: it saves names of failed tests into file tests/_log/failed, then execute codecept run -g failed to run these tests.

groups:
  # add 2 tests to db group
  db: [tests/unit/PersistTest.php, tests/unit/DataTest.php]

  # add list of tests to slow group
  slow: tests/_log/slow

For example, you can create the list of the most slow tests, and run them inside their own group. Pretty interesting and powerful feature that also can be used to run tests in parallel.

Parallel Test Execution

Not yet! Codeception does not support parallel test execution, nor will be provide it out of the box. That’s because there is no solution that will fit for everyone. You may want to run parallel tests locally in threads via pthreads extension, or run them on different hosts via SSH, AMQP, etc. But we already have everything to implement parallel testing.

Here is the algorithm:

  • split all tests into groups (with dynamic groups)
  • run each groups separately
  • merge results

And we will write guides on implementing this algorithm, as well as we plan to release a sample tool that will run tests in parallel.

Try it

As usual, we need your feedback to get 2.0 released as stable.

Download:

wget https://codeception.com/releases/2.0.0-beta/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-beta" 

P.S. Our sponsors 2Amigos updated their site and logo. Check it out!

Codeception 1.8.4 with minor fixes

0
0

Bugfix release on stable 1.8 branch. While you are waiting for 2.0 RC and final version, you can update your current Codeception to get the newest patches and updates. Here they are:

  • [WebDriver] Correctly return to main page when switchToIFrame() is called by ** n8whnp**.
  • [WebDriver] fixed screenshot capture by FnTm.
  • [Frameworks] More details in debug output by Ragazzo.
  • Fixed problem with encoding in $I->wantTo() by vgoodvin.
  • Add clone and unset capabilites to Maybe by brutuscat.
  • [WebDriver] Fixes to submitForm when typing password by pcairns.
  • [Phalcon1] haveRecord return id even when id is not public by xavier-rodet.
  • Modules to be loaded from global context by Ragazzo. Now you can pass long name of module into module configuration:
class_name: WebGuy
modules:
    enabled: [\My\Custom\Module, WebHelper]

Really helpful feature if you have multiple Codeception inits in one project and you want to have shared module.

All those features were included in 2.0-beta. Thanks to all contributors!

Update

redownload your codeception.phar for update:

1.8.4

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

Codeception 1.8.5

0
0

New improvements and bugfixes came during the last month into the 1.8 branch. We summarized them all and prepared a new release.

Ecosystem

But before we proceed you should check out the new Addons section on this site. We had only extensions listed there, but now you can find new modules and even applications there. And let’s name at least two projects you should definitely be aware of:

Thank you for all extensions and modules developers. If you developed a module, helper, or extension, share your work with others by adding your project to addons page (click Edit link in the bottom).

Bugfixes

  • [WebDriver] facebook/webdriver version was set strictly to 0.3. And this won’t be changed in future for 1.8 branch. We accidentaly commited ~0.3 into the 1.8.4 which made composer to install 0.4 version of facebook/webdriver. There was API break in 0.4, so WebDriver module didn’t work properly.
  • [Phalcon] Phalcon 1.3 compatibility added #944
  • [Laravel] Service providers are now correcly loaded by ShawnMcCool.
  • [Laravel] findRecord fixed by andkom.
  • [ZF2] Fixed handling of query parameters by piccagliani.
  • [Frameworks] Fixed problem with submitting form with input[type=image] by piccagliani.

Features

  • [Webdriver][Frameworks] input field can now be located by its name.
  • [Webdriver][Frameworks] element can be clicked by name too.

Update

redownload your codeception.phar for update:

1.8.5

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

Codeception 2.0 RC: Strict Locators

0
0

Codeception 2.0 is almost ready for final release. Thanks for everyone who already running alpha/beta version of Codeception and provides us with valuable feedback. If everything goes well the final release will be published in a week. The delay between releases is dictated by the need to allow parallel tests execution. As it was said before: Codeception won’t provide parallel test running out of the box, yet we will add a new chapter into guides where you will learn how to implement parallelism model that matches needs of your project.

Improvements

Strict Locators

In order to provide better element locations you can now specify locators explicitly like this: ['css' => 'input.name']. This will work for PHPBrowser, Frameworks modules, and WebDriver.

<?php
$I->fillField(['css' => 'input.name'], 'jon');
$I->click(['link' => 'Submit']);
$I->seeElement(['xpath' => './/div[@data-name="jon"]']);
?>

Allowed locator mechanisms are:

  • id
  • name
  • css
  • xpath
  • link
  • class

You can use strict locators in your Page Object classes as well.

See Element with Attributes

To improve checking element with expected attributes seeElement now takes second parameter:

<?php
$I->amOnPage('/form/field');
$I->seeElement('input[name=login]');
$I->seeElement('input', ['name' => 'login']); // same as above
?>

dontSeeElement works in corresponding manner.

grabAttributeFrom method added

To fetch value of attribute of element:

<?php
$I->amOnPage('/search');
$attr = $I->grabAttributeFrom('form', 'method');
$attr == 'GET'
?>

Also makeScreenshot method of WebDriver was simplified. Filename of saved screenshot does not include a test name.

Try it

Download:

wget https://codeception.com/releases/2.0.0-RC/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-RC" 

Codeception 2.0 RC2: Guzzle 4

0
0

Unfortunately this is not a final release of Codeception 2.0 you may have expected. Still we had to get another release candidate in order to implement one breaking and important change: upgrade to Guzzle version 4.

As you know PHPBroswer of Codeception used Mink, Goutte, and Guzzle to emulate visiting pages, filling forms, and other user-like web interaction. Recently Guzzle got a new major release, so we had to upgrade Codeception as well. We also removed Mink and Goutte and Codeception uses Guzzle directly. This will allow us faster to hunt down and fix the issues which may appear in future.

Guzzle changed their package name and namespace in latest release. If you were using Guzzle 3 and you want to upgrade to Codeception 2 you won’t get any conflicts in dependencies. But if you were using Guzzle\Http\Client class in your tests or helpers you will to change it to GuzzleHttp\Client.

PHPBrowser additional configuration

Guzzle4 got very flexible configuration. You may omit CURL_ constants and use request parameters to set headers, auth, cookies, and SSL verification. Codeception now accepts all request parameters of Guzzle in module configuration part:

 
modules:
   enabled: [PhpBrowser]
   config:
      PhpBrowser:
         url: 'http://localhost'
         auth: ['admin', '123345]
         headers:
            'User-Agent': 'Codeception'
            Accept: 'application/json'
         curl:
             CURLOPT_RETURNTRANSFER: true

And yes, curl configuration section is left for more advanced options and for backwards compatibility. BTW, SSL settings should be also defined via new configuration using verify, cert, ssl_key request options.

Try it

As usual we need your feedback. Guzzle upgrade is very dramatic change and we recommend to try it on your test suites. If you have issues report them and we will try to fix them before the final release, which is schenduled for next week.

Download:

wget https://codeception.com/releases/2.0.0-RC2/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-RC2" 

Codeception 2.0 Final

0
0

Codeception 2.0 is out. It was a long journey to get here but it was worth it. Main purpose of making Codeception 2.0 was not to break things up. Instead we focused on making Codeception simpler in use and reduce that awkward WTF moments you were experiencing rebuilding Guy classes and discovering dark magic of Maybe object.

That’s it. Codeception does not have 2-phases of execution and runs your tests as native PHP scenarios. It also does not require you to rebuild actor classes on configuration change - it is done automatically. And at last we don’t have Guys anymore. They were renamed in more neutral Tester objects. But it is up to you how will you call your actors: Testers, Guys, or even ninjas, those names can be configured!

Naming Conventions

We decided to change some naming conventions in Codeception.

  • _log directory was renamed to _output. It was logical decision, as Codeception does not print any logs by default, but constantly uses _log directory for printing debug information and reports.
  • _helpers was renamed to _support. Now it is more obvious that this directory may contain not only helpers but any kind of support code required for tests.

As it was said, Guys were renamed to Testers. Actor classes and helpers are named by suite. For instance, acceptance test will start with this line:

<?php
$I = new AcceptanceTester($scenario);
// and uses AcceptanceHelper
?>

In unit tests we simplified access to actor classes by just using $this->tester property (actor title). Looks a bit more readable.

<?php
function testSavingUser()
{
    $user = new User();
    $user->setName('Miles Davis');
    $user->save();
    $this->tester->seeInDatabase('users',array('name' => 'Miles', 'surname' => 'Davis'));
}
?>

If you are upgrading from 1.x you can still use codeGuy as it was left for compatibility.

Listed naming changes are recommended defaults. You don’t need to follow them if you plan to upgrade from 1.x.

Changes

Let’s briefly name some important changes added in Codeception 2.0

  • PHP >= 5.4
  • Upgraded to PHPUnit 4.x
  • Upgraded to Guzzle 4.x
  • Removed Mink (and Selenium, Selenium2, ZombieJS modules)
  • Removed Goutte
  • Logger moved to extension and disabled by default
  • Naming conventions changed
  • One-time execution for all tests
  • Autorebuild of actor classes
  • MultiSession testing with Friends
  • Strict Locators introduced
  • Fail fast option -f added to run command
  • Coverage options changed behavior in run command
  • Bootstrap test are loaded suite, not before each test
  • Parallel Execution (!!!)

Guides

Guides were reviewed and updated for 2.0 release. Unit Testing guide was completely rewritten, and new chapter Parallel Execution was added. Please note that parallel running tests is advanced feature and will require some time for you for set everything up correctly. But If your tests run longer then 30 minutes, you will get a real benefit out of it.

The Most Important Bug Fixed

The Most Important Bug was reported by many of our users:

We decided to fix it for 2.0 release. Now the date in footer is displayed correctly according to PHP rules (using <?=).

Upgrading

Codeception 2.0 was publishing pre-releases since February. Lots of developers already gave it a try and reported their issues. We hope the upgrade will go smoothly for you. Basically what you need is to:

  • run build command
  • remove any internal usage of Mink or Goutte
  • remove any usage of $scenario->prepare() as it always return false now
  • optionally remove $scenario->running() calls and $I->execute calls. They are not required anymore.
  • take a look into PhpBrowser module which is uses Guzzle4 directly. You may have issues with using it (as it was completely rewritten). Please report those issues to GitHub.
  • Selenium2 module should be replaced with WebDriver (as you already didn’t do so).
  • if you use variables in _bootstrap.php, rename it to _fixtures.php and load it manually into your tests.

If you have issues during upgrade send issues to GitHub and we will make upgrade more smoother.

wget https://codeception.com/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0" 

P.S. Codeception 1.8.6 was released as well. List of its changes will go tomorrow.

Codeception 2.0.1 and Changelog page

0
0

First bugfix release from the 2.0 version. This release introduces very important fix for running Codeception with PHP 5.4.x. Due to bug in PHP you would see strange errors while trying to execute Cept files. We can’t say which exect PHP 5.4 versions were affected, but if you had issues running Codeception 2.0 earlier, please upgrade.

The next notable change is see method of WebDriver module. Now it checks for a visible text on a page. If you need to check if text is present inside HTML source of a page you can use newly added seeInPageSource method for that. Thanks artyfarty for this bugfix.

Update

redownload your codeception.phar for update:

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

Changelog Page

From now on there will be no blogposts for minor releases. There is new changelog page which displays changes in latest releases. You should refer to it in order to follow the updates. Important changes, features, bugfixes will be mentioned there.

Unit Testing With Database

0
0

He-hey! You just opened this page to say that there is no such thing as unit testing with database. Unit tests are supposed to test isolated pieces, and sure unit tests should not touch the database. And we can agree with that. But it happened that “unit test” term is far more popular then “integration test”. Unit test have one very strong requirement. Isolation. Complete isolation from other code units and services. In real world we can spoil our lives trying to achieve complete isolation. And even when with pain and sweat we finally got out models isolated from database, we receive a very strange class of tests. They do not provide a valuable feedback. Model test that does not connect to real database is useless, as it provide false positive results, and the code behind may crash connecting to real storage. So what do we want from unit test after all? To be written in complete isolation (what for?), or provide real feedback?

So let’s not to concentrate on terms. Let’s concentrate on testing.

Testing Faster With Transactions

What drawbacks we receive if unit tests are connecting to database? Sure it will be dramatic slowdown. We will need to connect to database, insert data, and perform a cleanup afterwards. The most simple way of cleaning things up is to repopulate database completely. Yet, this is the slowest way. But there is a much better way of doing things. Especially for unit (and functional) tests. We can use transactions.

That’s right. We can cover a test into transaction, and rollback it after the test is finished. This is the fastest and the most stable way of working with database in unit tests. The only issue with it is that not all databases support nested transactions. For instance, in MySQL you can’t begin transaction and begin next transaction as there can be only one at one point. Yet, those nested transactions can be emulated. And here are good news: if you use one of popular PHP Frameworks: Laravel, Yii2, Phalcon, or Doctrine ORM, your framework already can do that! Thus, in unit test you can begin transaction in setup, and finish it in teardown of a test. Sure, you should use the framework’s database layer for that.

In Codeception we already use this feature for speeding up functional tests. But it was not that obvious that you can use the same practice for unit tests.

Let’s connect our framework to the unit.suite.yml. In current example we will use Laravel, yet very similar steps can be done for Yii2, Phalcon, and Symfony+Doctrine.

# Codeception Test Suite Configuration

# suite for unit (internal) tests.
class_name: UnitTester
modules:
    enabled: [Laravel4, UnitHelper, Asserts]

That’s all. This line of Laravel module will start the transaction before each test in suite.

Using ActiveRecord Helpers

Laravel, Phalcon, and Yii are frameworks that use ActiveRecord pattern for working with database. Codeception Framework modules have built-in helpers, which share the standard interface to work with database records.

Let’s write a sample Laravel test which demonstrates usage of Codeception helpers. In this example we will test activate method of User model. It activates user when provided activation key matches predefined one.

<?php
class UserTest extends \Codeception\TestCase\Test
{
    /**
     * @var UnitTester
     */
    protected $tester;

    protected $user_id;

    function _before()
    {
        // preparing a user, inserting user record to database
        $this->user_id = $this->tester->haveRecord('users', [
            'username' => 'John',
            'email' => 'john@snow.com',
            'activation_key' => '123456',
            'is_active' => 0

        ]);
    }

    function testUserCanBeActivatedWithValidKey()
    {
        // lookup for user with Eloquent API
        $user = User::find($this->user_id);
        // executing action
        $isActivated = $user->activate('123456'));
        // performing assertion
        $this->assertTrue($isActivated);
        // checking that data was actually saved into database
        $this->tester->seeRecord('users', [
            'id' => $this->user_id,
            'is_active' => 1
         ]);
    }

    function testUserNotActivatedWithInvalidKey()
    {
        $user = User::find($this->user_id);
        $this->assertFalse($user->activate('00000'));
        $this->tester->dontSeeRecord('users', [
            'id' => $this->user_id,
            'is_active' => 1
         ]);
    }
}
?>

As it was mentioned, similar test can be written for other mentioned frameworks. Depending on framework API ActiveRecord helper methods

  • grabRecord
  • seeRecord
  • haveRecord
  • doneSeeRecord

may take table name (Laravel) or model name (Phalcon, Yii2).

Who is that Tester?

$this->tester object used here is instance of Actor class and contain all methods of modules and helpers used in unit suite. It would be a good idea to put any shared code into helpers, and reuse across test cases by accessing $this->tester. This object is injected into any testcase that extends Codeception\TestCase\Test.

Conclusion

Sure, we could implement similar test using just Eloquent API without using Codeception helpers. Still, they are quite useful. Especially assertion methods like seeRecord which queries database for a record with given attributes.

Codeception Updates

0
0

Codeception is improving from day to day with the help of our contributors. Recently we had new 2.0.6 release with lots of bugfixes and new features. Unfortunately some not obvious regressions were introduced, and we had to release 2.0.7 as fast as we could.

In order to maintain a predictable release cycle we plan to release a new bugfix version each month and deliver a new major version each 6 months. We tend to follow this plan for future, yet we are not ready to follow it strictly.

Today we have a few more announcements to sum up all recent work on Codeception and its support projects.

CodeCoverage

Latest updates to c3 brought full compatibility with Codeception 2.0. Also c3 can be installed and updated via Composer.

"require-dev": {
    "codeception/c3": "2.*"
},
"scripts": {
    "post-install-cmd": [
        "Codeception\\c3\\Installer::copyC3ToRoot"
    ],
    "post-update-cmd": [
        "Codeception\\c3\\Installer::copyC3ToRoot"
    ]
}

This allows to follow updates of code coverage collector (c3) easily. Also please note, that c3 issues were moved to main Codeception repo.

AspectMock

AspectMock was recently updated to version 0.5.0. Latest update is powered by Go Aop framework version 0.5. In new AspectMock fixed passing parameters by reference, and added ability to mock PHP functions.

AspectMock allows to replace PHP function in desired namespace with user-defined closure or a return single return value. For instance, you can stub file operations by mocking file_get_contents or file_put_contents. You can also mock time functions, in order to make tests stable to time changes.

<?php
namespace App\Cache;

$func = test::func('App\Cache', 'file_put_contents');
$cache = new FileCache;
$cache->store('hello'); // calls file_put_contents
$func->verifyInvoked(); // verifies file_put_content was called
test::cleanup(); // restore original function
?>

Same can be done for time functions:

<?php
namespace demo;
test::func('demo', 'time', 'now');
$this->assertEquals('now', time());
?>

You can install latest AspectMock via Composer:

composer require "codeception/aspect-mock:~0.5"

Codeception 2.1

Next major release of Codeception will include new features, breaking changes, with ability to keep backwards compatibility. Here are the most interesting features you should wait for:

  • All support classes like actors, steps, pages, etc, will be moved to tests/_support directory.
  • Support classes will use PSR-4 standard.
  • Dependency Injection. All support and test classes can be included one into another: Pages can be injected into Helpers, Helpers can be injected into Cest classes, etc.
  • Module Conflicts. Lots of issues are happening when users try ot include modules with same apis into one suite. Like PhpBrowser + Laravel4 + WebDriver. Those module is possible to be used together, but possibly that was not really intended to be used this way. Call to $I->amOnPage may be executed by any provided modules depending on the order of their declaration. This leads to unpredictable errors, and is the source of confusion. In 2.1 Codeception will allow to define module conflicts, and throw exceptions if modules are not expected to be used together.
  • …and more

Global Installation

One of recently discussed topics for Codeception, was global installation of Codeception. From now on you can install Codeception globally using Composer:

composer global require 'codeception/codeception=2.0.*'

This will be recommended installation method since Codeception 2.1.

RoboCI

This project that desires its own post in this blog. It is open-source reproduction of Travis CI using Docker and Travis Cookbooks. RoboCI can be use to execute Travis CI tests locally or on remote server. It does not require VirtualBox nor Vagrant installed. It was executed with Codeception internal tests, with Selenium + Firefox, MySQL, PostgreSQL, MongoDb, and other services. You can read more about RoboCI and try it in your projects. In future we plan to provide consulting services on using RoboCI with Codeception or any other PHP testing framework. Stay in touch for updates!

Managing Data with FactoryMuffin

0
0

One of the greatest things of testing Ruby on Rails applications was usage of Factories, managed by FactoryGirl. It changed the way the test data was managed in test. Instead of defining fixtures one by one, in a single .yml configuration, it allowed to generate required models per test. This concept is very convenient, however it is not widespread in PHP world. Probably, this was because, there is no single ORM in PHP, like ActiveRecord is for Ruby. We have Doctrine, Eloquent, and each major framework has its own ORM layer, so that we it is pretty hard to deal with models of those frameworks in one manner.

But it looks like factories finally came to PHP! The League of Extraordinary Packages produced a great package, self-claimed as factory_girl for PHP. Meet Factory Muffin. It allows simple generation of database records for integration and functional testing. Let’s see how it can be used with Codeception.

Setup in Codeception

At first lets add league/factory-muffin": "~2.0 to composer.json:

"require-dev": {
    "codeception/codeception": "~2.0",
    "league/factory-muffin": "~2.0"
}

Then we execute composer install.

Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing fzaninotto/faker (v1.4.0)
    Downloading: 100%         

  - Installing league/factory-muffin (v2.1.1)
    Downloading: 100%         

Writing lock file
Generating autoload files
Generating optimized class loader

As you may noticed, FactoryMuffin uses awesome Faker library for generating random data for models.

For using factories inside tests we will create FactoryHelper:

php vendor/bin/codecept g:helper Factory
Helper /home/davert/project/tests/_support/FactoryHelper.php created

We will define factories in the _initialize method of FactoryHelper. Let’s say we have two models, Post and User in application. This is how we specify rules for generating new objects of these models.

<?php
class FactoryHelper extends \Codeception\Module
{
    /**
     * @var \League\FactoryMuffin\Factory
     */
    protected $factory;
    
    public function _initialize()
    {
        $this->factory = new \League\FactoryMuffin\Factory;
        $this->factory->define('Post', array(
            'title'    => 'sentence|5', // title with random 5 words
            'body'   => 'text', // random text
        ));

        $this->factory->define('User', array(
            'email' => 'unique:email', // random unique email
        ));
    }
}
?>

This is how generation of Post and User is defined. From the box, Muffin is designed to work with ActiveRecord models, and with Eloquent in particular. So you can use it in Yii, Phalcon, yet we will demonstrate its usage in Laravel application. FactoryMuffin can also be customized to work with Doctrine as well.

Using Factories in Laravel

To use FactoryMuffin in Laravel functional tests we need to create additional methods in FactoryHelper for generating and saving records: havePosts and haveUsers methods. They will populate database with number of records specified. We will need to clean up those records at the end of a test.

<?php
class FactoryHelper extends \Codeception\Module
{
    /**
     * @var \League\FactoryMuffin\Factory
     */
    protected $factory;
    
    public function _initialize()
    {
        $this->factory = new \League\FactoryMuffin\Factory;
        $this->factory->define('Post', array(
            'title'    => 'sentence|5', // title with random 5 words
            'body'   => 'text', // random text
        ));

        $this->factory->define('User', array(
            'email' => 'unique:email', // random unique email
        ));
    }

    public function havePosts($num)
    {
        $this->factory->seed($num, 'Post');
    }

    public function haveUsers($num)
    {
        $this->factory->seed($num, 'Post');
    }

    public function _after(\Codeception\TestCase $test)
    {
        // actually this is not needed
        // if you use cleanup: true option 
        // in Laravel4 module
        $this->factory->deleteSaved();
    }
}
?>

By including FactoryHelper to the functional suite config, we can use it inside the actor ($I) object.

This allows us to test features like pagination. For instance, we can check that only 20 posts are listed on a page:

<?php
$I = new FunctionalTester($scenario);
$I->wantTo('check that only 20 posts are listed');
$I->havePosts(40);
$I->amOnPage('/posts');
$I->seeNumberOfElements('.post', 20);
?>

Source code of this example is on GitHub.

Factories in Unit Tests

Factories can also be used in unit testing. Let’s say user can create posts, and in order to optimize queries we are saving number of user posts in num_posts column of users table. We are going to test that this column is updated each time a new post by user is created.

We will need one more method added into FactoryHelper class:

<?php
public function produce($model, $attributes = array())
{
    return $this->factory->create($model, $attributes);
}
?>

After we include FactoryHelper into unit suite we can use its methods in tests:

<?php
class UserTest extends \Codeception\TestCase\Test
{
    /**
     * @var UnitTester
     */
    protected $tester;

    function testCounterCache()
    {
        $user = $this->tester->produce('User');
        $this->assertEquals(0, $user->num_posts);

        $this->tester->produce('Post', ['user_id' => $user->id]);
        $this->assertEquals(1, User::find($user->id)->num_posts);

        $this->tester->produce('Post', ['user_id' => $user->id]);
        $this->assertEquals(2, User::find($user->id)->num_posts);
    }
}
?>

Conclusion

As you see, factories make your tests clean and expressive. Forget about managing test data manually, forget about fixtures. Use FactoryMuffin.

Update FactoryHelper for Symfony2 and Doctrine modules

Acceptance Testing With No Selenium or PhantomJS Installed

0
0

Selenium PhantomJS

This post summarizes some recent work that was done for creating portable testing environments. The idea was to reduce dependencies required by acceptance testing. As you know, testing of web pages require Selenium or PhanromJS servers, which allow to interact with web page, and execute user scripts on page. However, installing those dependencies, can be a pain. Especially Selenium. If you are not running it inside desktop environment you probably need to have Firefox or Chromium, and they can’t be run without virtual framebuffer, as they have some display server to run on. Setting up those packages can take liters of sweat and bunch of nerves.

That’s why we decided to pack Selenium and PhantomJS into portable Docker containers. Did you hear about Docker? If not - you probably should take some time to look at it. Docker allows you to create virtual environments on Linux hosts without real virtualization. Docker creates an isolated environment inside the host machine without recreating operating system.

Docker containers are lightweight and easy to use. If you have Linux. Probably all your servers use Linux, so you might find docker images of Selenium and PhantomJS pretty useful!

SeleniumEnv: Selenium, Xvfb, Firefox, and Chromium Inside Container

OK, lets assume you are using Linux and Docker >=1.2. From now on running Selenium won’t take from you anything more then running those commands:

docker run -i -t -p 4444:4444 davert/selenium-env

This will start virtual display server (Xvfb) and Selenium inside a container. Please note, you don’t need anything of those installed on your host. Not even Java, required to run Selenium. Only Docker is needed and you can run Selenium in one line!

By running this command you will see this output:

Running Selenium Env: Selenium Server, and Xvfb with Firefox and Chromium
Host IP: 172.17.42.1
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
[dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!
02:02:30.641 INFO - Launching a standalone server
02:02:30.670 INFO - Java: Oracle Corporation 24.60-b09
02:02:30.670 INFO - OS: Linux 3.16.0-24-generic amd64
02:02:30.684 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
02:02:30.735 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: LINUX
02:02:30.790 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
02:02:30.791 INFO - Version Jetty/5.1.x
02:02:30.792 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
02:02:30.792 INFO - Started HttpContext[/selenium-server,/selenium-server]
02:02:30.793 INFO - Started HttpContext[/,/]
02:02:30.806 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@1a966bcb
02:02:30.807 INFO - Started HttpContext[/wd,/wd]
02:02:30.809 INFO - Started SocketListener on 0.0.0.0:4444
02:02:30.809 INFO - Started org.openqa.jetty.jetty.Server@2b6ea258

By specifying -p 4444:4444 option in docker run we mapped container’s port 4444, which is default for Selenium to the same port on our host machine. So, any process can connect to Selenium on its standard port, even Selenium is completely isolated from other processes.

And here comes a tricky question. If Selenium is completely isolated, how can we test applications running on a local web server? We have a solution for that. If your application is served by nginx or any other web server, you can probably access it by hostname (other than localhost). What we can do is to pass hostname inside a container.

docker run -i -t -p 4444:4444 -e APP_HOST=myhost davert/selenium-env

This will bind myhost to your host machine IP. So, container, and Firefox, Chromium, can get access to your local web site from inside container.

Similar idea is used for application executed on specific port. If you use PHP built-in server, you can start it at specific port, and pass it into container as well. Please note, that to make server accessible from container, it should be started on 0.0.0.0 IP:

php -S 0.0.0.0:8000
# in new terminal window
docker run -i -t -p 4444:4444 -e APP_PORT=8000 davert/selenium-env 
# in new terminal window
codecept run acceptance

PhantomJsEnv: PhantomJS Inside Container

Instructions are the same for running PhantomJS. It can be started with

docker run -i -t -p 4444:4444 davert/phantomjs-env

Everything else is pretty similar. Also you can easily switch from Selenium to PhantomJS by starting or stopping those two containers. Isn’t that useful?

Images and Source Code

SeleniumEnv and PhantomJsEnv are created from Travis CI Cookbooks and packed into containers. They are free to use and free to modify for custom needs.

If you need to have Selenium Server from version other than 2.44.0 (which is latest for today), you should update Docker file and build an image from it.

SeleniumEnv and PhantomJsEnv may simplify testing for all Linux users. They will definitely simplify setting up Continuous Integration servers. You can even recreate a complete testing environment inside a container. Take a look into RoboCI project for this. It allows you to build Travis CI-like service on your own host!

Use them and enjoy :)

P.S. Feedback, Pull Requests, and Issues, are welcome )


Setting up Jenkins with Codeception

0
0

Jenkins

A frequent question that appear from time to time is How to set up Codeception with Jenkins. Despite the growth of popularity of various Continuous Integration solutions Jenkins is still the most popular open-source solution on market. It is easy to setup and is easy to customize by applying various 3rd-party plugins. In this post we will take a look on how to setup testing process of PHP project with Jenkins CI and Codeception.

Create new job in Jenkins

Preparing Jenkins

We will start from an empty Jenkins Server executed on localhost. Let’s create a new Job for project tested with Codeception and name it PHPCodeceptionProject. Before we proceed to configuring it, lets install required plugins:

  • Git Plugin - for building tests for Git repo
  • Green Balls - to display success results in green.
  • xUnit Plugin, jUnit Plugin - to process and display Codeception XML reports
  • HTML Publisher Plugin - to process Codeception HTML reports
  • AnsiColor - to show colorized console output.

Jenkins Plugins

Basic Setup

Once everything is installed lets get back to configuring our newly created PHPCodeceptionProject. At first we will set up Git to checkout code and build. Depending on your needs you can set up periodical build or trigger build once the change is pushed to GitHub (you will need GitHub plugin for that). You may also set up Jenkins as alternative to Travis CI to build your pull requests with GitHub pull request builder plugin.

We will enable colors in console using Ansi Color

Jenkins ANSI color

Next and the most important part is to define build step. We won’t create any sofisticated environment settings like setting up database and services. We assume that all we need to execute tests is to inctall composer dependencies and run codeception tests:

composer install
./vendor/bin/codecept run

Jenkins Codeception Build Step

And that’s the absolute minimum we need to execute tests in Jenkins. We can save project and execute the job. Jenkins will clone repository from Git, install composer dependencies and run Codeception tests. If tests fail you can review the console output to discover what went wrong.

Jenkins Console Output

XML Reports

But we don’t want to analyze console output for each failing build. Especially If Jenkins can collect and display the results inside its web UI. Codeception can export its results using JUnit XML format. To generate XML report on each build we will need to append --xml option to Codeception execution command. Codeception will print result.xml file containing information about test status with steps and stack traces for failing tests. Unfortunately Jenkins can’t process such special like Codeception steps. So we will need to configure to print data only which strictly applies to xUnit xml format. This should be done in codeception.yml

settings:
    strict_xml: true

Now let’s update our build step to generate xml:

composer install
./vendor/bin/codecept run --xml

and ask Jenkins to collect resulted XML. This can be done as part of Post-build actions. Let’s add Publish xUnit test result report action and configure it to use with PHPUnit reports.

Use PHPUnit xUnit builder for Jenkins

Now we should specify path to PHPUnit style XML reports. In case of standard Codeception setup we should specify tests/_output/*.xml as a pattern for matching resulted XMLs. Now we save the project and rebuild it.

Jenkins Result Trend

Now for all builds we will see results trend graph that shows us percentage of passing and failing tests. We also will see a Latest Test Result link which will lead to to the page where all executed tests and their stats listed in a table.

HTML Reports

To get more details on steps executed you can ask Codeception to generate HTML report. Jenkins can display them as well.

composer install
./vendor/bin/codecept run --html

Now we need HTML Publisher plugin configured to display generated HTML files. It should be added as post-build action similar way we did it for XML reports.

Jenkins Codeception HTML Setup

Jenkins should locate report.html located at tests/_output/. Now Jenkins will display HTML reports for each build.

Jenkins HTML Report Jenkins Codeception HTML Results

That’s pretty much everything you need to get a simple Jenkins setup with Codeception.

Codeception 2.1 Beta

0
0

We are glad to announce the first release of Codeception 2.1. It was a long time of developing the next major version of our testing framework. We delayed the release many times trying to bring as many cool features as we could. Today we can say that master branch is stabilized and we are prepared to share results of our work with you. Codeception 2.1 tries to do its best in test codebase organizing, improving ux, and customization.

Take a look what is landed in Codeception 2.1.

Full list of features is available in changelog. Documentation was updated accordingly.

You can install Codeception 2.1 via composer by requiring 2.1.0-beta version, or by downloading phar.

Give new Codeception a try and send us feedback. As always use GitHub issues for that. Thanks for being with us all that time! We hope you will love this release.

Stable 2.1.0 is expected by the middle-end of May

Upgrade Notes

  • If you are upgrading from Codeception 2.0 delete actor classes in tests/acceptance, tests/functional, etc, and rebuild actor classes. They will be recreated in tests/_support.
  • Try to execute new tests. If there are unmet dependencies, or module conflicts, you will be notified by exception with an advice how to resolve it.
  • It is recommended to recreate Helpers per suite with g:helper generator, like codecept g:helper Acceptance. New helpers will be store in tests/_support/Helper.
  • Same for PageObjects and StepObjects, they should be moved to _support.

Codeception 2.1 RC

0
0

Today we are announcing Codeception 2.1 RC release. We are almost ready to release a stable version, however we’d like to receive more feedback before the actual launch. We encourage you to install the latest 2.1 release and try it on your projects. Migrations should be painless if you follow the upgrade instructions. Here are the things you should keep in mind while upgrading:

  • REST, SOAP and Doctrine2 modules rely on another module which should be explicitly set via depends config param. For instance, you should pass Laravel5 module if you do API testing with REST module
modules:
    enabled: [REST, Laravel5]
    config:
        REST:
            depends: Laravel5
  • Actor classes are moved to _support directory. Please delete all previously generated actor classes which reside in suites directories. New AcceptanceTester, FunctionalTester, UnitTester classes are expected to be extended with custom methods.
  • Modules that share similar interfaces like WebDriver, PhpBrowser, and framework modules won’t run together. You should avoid enabling more than one of those modules in suite config to avoid confusion. If you enable Laravel5 and WebDriver and execute $I->amOnPage('/') you can’t be sure how this command is exected - will it open a browser window using WebDriver protocol, or it will be handled by Laravel framework directly.
  • Cept files should avoid setting their metadata via $scenario methods. Instead of calling $scenario->skip(), $scenario->group('firefox'), etc, it is recommended to set scenario settings with annotations // @skip, // @group firefox. With this change you can now skip tests based on a condition: if (substr(PHP_OS, 0, 3) == 'Win') $scenario->skip().
  • Kohana, Symfony1, Doctrine1 modules were removed and they reside in separate repositories in Codeception organization on GitHub.

That should be enough to have everything upgraded. We already prepared updated guides (they didn’t change from 2.1.0-beta). Please try new Codeception and tell us your opinion in comments or on GitHub.

Codeception 2.1 can be used with latest Guzzle 6 (as well as previous Guzzle releases). That’s the most important change since the beta version. See the changelog for more info. We are preparing site updates and demo apps to reflect new changes. Stay tuned!

Download 2.1.0-rc phar

via composer.json:

"codeception/codeception": "2.1.0-rc1"

Codeception 2.1 Is Here

0
0

We are finally ready to show you the Codeception 2.1. Since RC we stabilized its codebase, and we encourage you to start all your new projects with Codeception 2.1. As well as migrate old ones. Codeception 2.1 is aimed for consistency and provide you the better experience for testing your web applications. This new release makes tests even more simple to read, write, and maintain.

If you didn’t track for the changes in master we will list all the new features here:

  • We added Recorder extension, which is probably the most fancy feature you may try. Using it you can record test execution history by saving a screenshot of each step. This is handy for running tests on CI, debugging tests executed via PhantomJS or showing nice reports to your boss.

recorder

  • Updated to Guzzle 6. Codeception can now work both with Guzzle v5 and Guzzle v6. PhpBrowser chooses right connector depending on Guzzle version installed.
  • PSR-4: all support classes moved to tests/_support by default. Actors, Helpers, PageObjects, StepObjects, GroupObjects to follow PSR-4 naming style. New AcceptanceTester, FunctionalTester, UnitTester classes are expected to be extended with methods of common behaviors there. For instance, it is a good idea to place login method into the actor class:
<?php
class AcceptanceTester extends \Codeception\Actor
{
    use _generated\AcceptanceTesterActions;

    public function login()
    {
       $this->amOnPage('/login');
       $this->fillField('name', 'jon');
       $this->fillField('password', '123345');
       $this->click('Login');
    }
}
?>
  • Dependency Injection: support classes can be injected into tests. Support classes can be injected into each other too.
<?php
class UserCest 
{
  // inject page objects into Cests
  function updatePassword(\Page\User $page, AcceptanceTester $I)
  {
      $page->openProfile();
      $page->editProfile();
      $I->fillField($page->oldPasswordField, '123456');
      $I->fillField($page->newPasswordField, '654321');
      $I->fillField($page->passwordFieldRepeat, '654321');
      $I->click($page->saveBtn);
      $I->see('Password has been updated');
  }

  // inject step object into Cest
  function adminUpdatePassword(\Step\Admin $I)
  {
      $I->enterAdminArea();
      $I->changePassword('654321');
      $I->see('Password has been updated');
  }
}
?>
  • Module config simplified: Modules can be configured in enabled section of suite config. Take a look of this sample declaration of Api suite, there is no config section inside modules.
modules:
    enabled:
        - WebDriver:
            url: https://codeception.com
            browser: firefox
        - \Helper\Acceptance
  • Dependencies: module can explicitly define dependencies and expect their injection. REST, SOAP and Doctrine2 modules rely on another module which should be explicitly set via depends config param.
class_name: ApiTester
modules:
    enabled:
        - REST:
            url: https://api.github.com/
            depends: PhpBrowser           
        - \Helper\Api

As you can see, you don’t need to specify PhpBrowser in enabled section, you can set it only via depends.

  • Conflicts: module can define conflicts with each other. Modules that share similar interfaces like WebDriver, PhpBrowser, and framework modules won’t run together. You should avoid enabling more than one of those modules in suite config to avoid confusion. If you enable Laravel5 and WebDriver and execute $I->amOnPage('/') you can’t be sure how this command is exected - will it open a browser window using WebDriver protocol, or it will be handled by Laravel framework directly.

  • Current modules, environment, and test name can be received in scenario.

<?php
$scenario->current('name'); // returns current test name
$scenario->current('modules'); // returns current modules
$scenario->current('env'); // returns environment

// can be used like this
if ($scenario->current('env') == 'firefox') {
  // do something for firefox only
}
// naming screenshot
$I->makeScreenshot($scenario->current('name').'_screenshot.png');
?>
  • Environment Matrix: You can run tests combining several environments by separating their names by comma:
codecept run --env dev,phantom --env dev,chrome --env dev,firefox

Environments can now be stored in separate configuration files in tests/_envs dir created with generate:environment command:

tests/_envs
|── chrome.yml
|── phantom.yml
|── firefox.yml
|── dev.yml
  • Cept files should avoid setting their metadata via $scenario methods. Instead of calling $scenario->skip(), $scenario->group('firefox'), etc, it is recommended to set scenario settings with annotations // @skip, // @group firefox. However, you can use $scenario->skip whenever you need to do it on some condition, like
<?php
if (substr(PHP_OS, 0, 3) == 'Win') $scenario->skip()
?>
  • Improved HTML reports html report

  • Modules can be partially loaded. If you need only some actions to be included into tester objects. For instance, you want to have REST API tests with Laravel5 module. In this case you probably don’t want methods from Laravel module like amOnPage or see to be included into the ApiTester as they interact with HTML pages, which we are not supposed to use. But you still need Laravel ORM methods like seeRecord to verify that changes were saved to database. In this case you can enable only ORM methods of Laravel5 module.

modules:
    enabled: 
        - Laravel5:
            part: ORM
        - REST:
            part: Json
            depends: Laravel5

As for REST module you can load methods only for API format you are using. You can choose either XML or Json, so only methods for Json or XML will be loaded.

  • Whenever you have steps grouped inside actor class, pageobject, or stepobject, you can the step executed with its substeps in console or HTML report. html report
  • RunFailed extension is enabled by default for all new projects. It means that you can rerun your failed tests by executing codecept run -g failed when this extension is enabled. 1

And lots of other notable improvements which you can see in changelog.

Starting from Codeception 2.1 we recommend using Cest as a default format for all scenario-driven acceptance and functional, or api tests. Guides were rewritten to reflect new improved approaches to testing that you should practice by using Codeception.

Upgrading

  • It is pretty easy to upgrade from 2.0. The only thing you should start with is to rebuild your actor classes by running codecept build command. The old actor classes (*Tester or *Guy) in suite directories should be deleted manually.
  • REST, SOAP, Doctrine2 module will need to be configured to use a dependent module as it was shown above. You will get a detailed exception with configuration example once you execute tests with those modules.
  • Helpers, PageObjects, StepObjects are expected to follow PSR-4 standard and use namespaces. It is recommended to rename classes to replace suffixes with namespaces, like UserPage => Page\User. However, those changes are not required, so you can keep your support objects without changes.

If you are using 2.0 and you won’t plan to upgrade in nearest future, you can still use releases from 2.0 branch. Minor fixes and improvements will still be accepted there. Also the site will contain documentation for 2.0 and 2.1 versions.

Try Codeception 2.1 today by installing it via Composer:

composer require "codeception/codeception:*" --dev

or by downloading it as Phar archive

And provide a feedback!

Using Codeception for Symfony Projects

0
0

Codeception Testing Framework from its roots was a plugin of symfony 1 framework. Today Codeception is powered by Symfony components and can be used to run functional tests for practically any popular PHP framework.

Why would you someone ever choose Codeception if Symfony already has mature testing infrastructure. Well, here are our reasons. Codeception tests are:

  • fast, as each functional/integration test is wrapped into transaction using Doctrine ORM
  • scenario-driven, it means that tests are linear, described in easy to get PHP DSL
  • can be used for testing complex interactions inside functional tests.
  • easy to write, as Codeception already provides bundled actions and assertions for most popular use cases.
  • combine all testing levels (acceptance, functional, unit) in one tool.

Today we will write how Codeception can be installed into a Symfony project and fit with its structure: we will put functional and unit tests to corresponding bundles, write acceptance tests for complete application, and use one runner to execute them all at once.

symfony-demo

We will use official symfony-demo application in this example. Once you get it installed you should add Codeception testing framework to dev dependencies list in composer.json:

composer require --dev "codeception/codeception:~2.1"

As you may know Codeception has bootstrap command which create common file structure for acceptance, functional, and unit tests. However, as we decided we will follow Symfony way and skip creating global functional and unit tests. So we start with bootstrapping empty project without predefined suites.

php bin/codecept bootstrap --empty

As you may see, we got new tests directory and codeception.yml file created. Let’s have acceptance tests there:

php bin/codecept g:suite acceptance

Acceptance tests are expected to test the site from an end-user’s perspective. No matter how many unit tests you have in your projects you can’t get without acceptance testing. What if you see a blank page even all unit tests passed. How could this happen? Maybe you rendered wrong template, maybe some scripts or styles were not loaded. Those things can’t be handled with internal: unit or functional testing, however with acceptance tests you may be confident that UI is available for customers.

That’s why we recommend to have tests with real browser interaction. You can learn more about acceptance testing from our guides.

But what about unit and functional tests? As we decided we will put them into bundles. Symfony-demo has only AppBundle included, so we will create new Codeception setup in src/AppBundle. Take a note that we want those tests to be placed in their own namespace:

php bin/codecept bootstrap --empty src/AppBundle --namespace AppBundle

We will also create unit and functional suites there:

php bin/codecept g:suite functional -c src/AppBundle
php bin/codecept g:suite unit -c src/AppBundle

As you noticed we specify path to different Codeception setup with -c or --config option.

Unit tests of Codeception are not quite different from regular PHPUnit tests. You can even copy your old PHPUnit tests to src/AppBundle/tests/unit and have Codeception run them. It is much more interesting to use Codeception to have functional tests replacing ones extending Symfony\Bundle\FrameworkBundle\Test\WebTestCase class.

Let’s have a test that will check that there is specific number of posts on a page. Symfony-demo app has the similar test included:

 
<?php
namespace AppBundle\Tests\Controller;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use AppBundle\Entity\Post;

class BlogControllerTest extends WebTestCase
{
  public function testIndex()
  {
      $client = static::createClient();
      $crawler = $client->request('GET', '/en/blog/');
      $this->assertCount(
          Post::NUM_ITEMS,
          $crawler->filter('article.post'),
          'The homepage displays the right number of posts.'
      );
  }
}

We will rewrite it in Codeception manner. At first we are generating new empty test case for it. We use scenario-driven test format called Cest:

php bin/codecept g:cest functional BlogCest -c src/AppBundle

And here goes the test:

 
<?php
namespace AppBundle;
use AppBundle\Entity\Post;

class BlogCest 
{
    public function postsOnIndexPage(FunctionalTester $I)
    {
        $I->amOnPage('/en/blog/');
        $I->seeNumberOfElements('article.post', Post::NUM_ITEMS);
    }
}
?>

As you see, Codeception test is shorter. It is simple, clean, and can be easily extended for more complex interactions. However, we are not ready to run it yet. We need to prepare Codeception to run functional tests inside Symfony context. For this we need to edit src/AppBundle/tests/functional.yml configuration file to enable modules: Symfony2 and Doctrine2 to use:

class_name: FunctionalTester
modules:
    enabled:
        - Symfony2:
            app_path: '../../app'
            var_path: '../../app'
        - Doctrine2:
            depends: Symfony2
        - \AcmeBundle\Helper\Functional

The most important thing here is to provide valid app and var paths for Symfony. Also we are specifying that Doctrine’s EntityManager should be taken from Symfony DIC. Let’s run functional tests of AppBundle:

php bin/codecept run functional -c src/AppBundle

In this case you will see following output:

Codeception PHP Testing Framework v2.1.2
Powered by PHPUnit 4.8.2 by Sebastian Bergmann and contributors.

AcmeBundle.functional Tests (1) ------------------------
Posts on index page (BlogCest::postsOnIndexPage)     Ok
--------------------------------------------------------


Time: 274 ms, Memory: 36.00Mb

OK (1 test, 1 assertion)

But you can do much more with functional testing. You can insert/assert data with Doctrine by using prepared methods like haveInRepository, seeInRepository of Doctrine2 module. You can perform complex web interactions like filling forms, clicking links, following redirects and much more with methods of Symfony2 module. Those modules are combined together and their methods are available in FunctionalTester class you are supposed to use for writing functional tests. If you are interested to see more complex Codeception tests, we’ve got them for you.

Btw, you can use Symfony2 and Doctrine2 module for writing your unit and integration tests as well.


But how can we run acceptance tests of a project with tests from AppBundle together? We need to edit codeception.yml configuration file in project root to make it. Let’s add those lines there:

include:
    - src/*Bundle

That’s it. For now Codeception will include all installations stored in Bundles on run. If you execute:

php bin/codecept run

you will probably see that BlogCest of AppBundle was executed as it was expected to.


tests

The most complex thing in starting using Codeception with Symfony is have it configured. Despite Codeception is auto-connecting to Symfony framework and Doctrine you still have to do some changes to follow Symfony structure. Please take a detailed look into our forked version of symfony-demo project which we configured in the manner we described in this post. Please use similar configuration for all your Symfony projects.

Start using Codeception and discover how complex things can be tested in really simple manner. And once again, even functional and integration tests are really fast, as we start transaction before each test and rollback it afterwards. Write them as many as you need to, do not rely on unit tests only!

P.S. Symfony2 and Doctrine2 module is seeking for an active maintainer. If you work with Symfony and Codeception please contact us to join Codeception team!

On The Threshold of Codeception 2.2: Upcoming Features

0
0

Codeception 2.2 is going to land in nearest weeks. We planned to release it much earlier, but as we are not bound to specific date, we decided to work more on improvements and include the most tasty ones in next release. In this and following blogposts we will review the most significant of upcoming features. So take a look what we prepared for you.

We don’t plan any major breaking changes in this release. However, we will remove deprecations, as you may know we display deprecation warnings in output, so this break should be at least predictable.

PHPUnit5 support

Yes, Codeception is finally will unlock it’s dependency on PHPUnit 4.8 so you could receive the latest stable versions of PHPUnit. As you may know PHPUnit 5.x has much better support of PHP7, so anyone who develop using the latest version of PHP should use it. But wait, it’s not the feature of Codeception 2.2 itself! Codeception 2.1.7 will be released with PHPUnit 5.x support as well.

Test Dependencies

Finally @depends annotation works as you might expected it to. The hardest thing about dependencies is that PHPUnit does nothing to organize dependent tests in a right order:

PHPUnit supports the declaration of explicit dependencies between test methods. Such dependencies do not define the order in which the test methods are to be executed but they allow the returning of an instance of the test fixture by a producer and passing it to the dependent consumers.

So their usage in Codeception was useless, especially if you were executing tests in random order with shuffle: true configuration. Some test frameworks, like minitest from Ruby run tests in shuffle by default. On the contrary PHPUnit restricts that, test with @depends annotation must be be executed after the test it relies on. It actually requires you to declare dependent tests in the same file and use the exact order of test execution.

Codeception takes the exact @depends annotation, reorders tests so dependent tests always will executed after the tests they rely on. This will work for Cest and Test formats both. And you can actually declare dependencies for tests in other files. Be sure, they will be taken, executed in the right order, and if a test fails its dependencies will be skipped. This is likely improve acceptance testing. If login test doesn’t pass, you shouldn’t even try to launch all other tests with authorization required.

Here is the annotation format to use to declare dependencies. It should be {fullClassName}:{testName} or {testName} for the test in the same class.

@depends login
@depends UserCest:login
@depends App\Service\UserTest:create

Unlike PHPUnit, you can’t pass data between dependent tests. Alternatively you can use Helpers or Codeception\Util\Fixtures class to store shared data.

Params

This lines are written just in the same time as corresponding pull request is merged:

Codeception 2.2 will allow you to use parameter in configuration files populated from environment variables, yaml configs or .env files. You can use parametrized vars in suite configs, by wrapping them in % placeholder, and adding one or several parameter providers using the params key in global config:

params: [env] # get prameters from environment vars

You can use Symfony-style .yml files, Laravel-style .env, or .ini files to receive params.

This feature would be useful if you want to share database configuration with application and tests. This will especially be useful if you use some sort of credentials passed via environment vars. For instance, if you use cloud testing services, you can set login and password with params and get the real values from environment:

    modules:
       enabled:
          - WebDriver:
             url: http://mysite.com
             host: '%SAUCE_USER%:%SAUCE_KEY%@ondemand.saucelabs.com'
             port: 80
             browser: chrome
             capabilities:
                 platform: 'Windows 10'

Conflicts

This is possibly a breaking feature which was announced in Codeception 2.1 but never worked the way it supposed to be. So let’s talk about it once again. A friendly reminder, if you use config like this:

modules:
    enabled: [PhpBrowser, Laravel5, WebDriver]

you will have problems in Codeception 2.2, if you didn’t have them earlier. Basically the issue here, that those modules provide pretty the same API but different backends for test executions. So it’s hard to tell from config, will this tests be executed as Laravel functional tests, as acceptance tests in real browser, or HTML-scrapper called PhpBrowser. We have lots of issues, when developers misuse those modules, trying to include everything in one config, and we can’t help them besides the good advice to choose one module of one kind.

To avoid this confusion we introduced Conflicts API. Module can declare conflict for modules of the same API, and ConfigurationException will be thrown if you try to use them in group.

But what if you really need to use Laravel5 module along with, let’s say, WebDriver? Don’t worry, partially loaded modules won’t get into conflict. The most common use case is using Laravel ORM methods to create-delete data for acceptance test. In this case you don’t need to load all actions of Laravel5 module but only those related to ORM:

modules:
    enabled:
        - WebDriver:
            url: http://localhost
            browser: firefox
        - Laravel5:
            part: orm

this way you receive haveRecord, seeRecord, grabRecord, have, haveMultiple methods running though Eloquent but amOnPage, see, etc will be executed through WebDriver.

If you were mixing WebDriver with PhpBrowser in order to use REST API inside acceptance tests, you can still have it. Don’t enable PhpBrowser but set it as dependent module for REST:

modules:
    enabled:
        - WebDriver:
            url: http://localhost/
            browser: firefox
        - REST:
            url: http://localhost/api
            depends: PhpBrowser

This way you can use API to create test data for application:

<?php
$I = new AcceptanceTester($scenario);
// let's create a post
$I->sendPOST('/posts', ['title' => 'Hello, Codeception 2.2', 'body' => 'Almost here']);

// check it is actually there
$I->amOnPage('/posts');
$I->see('Hello, Codeception', 'h2');

…and much more to come

Sure, that was not everything what is coming in 2.2! Stay tuned for new cool announcements, which will be posted soon. We will describe new modules, new test formats, and new features! Stay tuned.

Even More Features of Codeception 2.2

0
0

Today we continue to show you new features of Codeception 2.2, planned to release this March. In previous post we’ve got you acquainted with Test Dependencies, Params, and Conflicts. This post will continue this overview, and we will start with some nice modules, which might be useful to you.

As you know, in Codeception we try to help developers and testers share their experience by providing set of shared pieces, and not to reinvent the wheel. Modules play exactly that role: you just include modules to fit your project, and write your tests, concentrating on its business logic, and not on implementation. 90% test steps in functional and acceptance tests are covered with out modules. So what do we have prepared for 2.2?

DataFactory

This module should solve problem of generating and populating data for tests. Right now we have Db module, MongoDb with really limited functionality, methods like haveRecord, seeRecord in various frameworks, and Sequence module for generating unique keys for data. However, they didn’t provide any way to generate data needed exactly for a particular test.

DataFactory module should solve this with fixture generators, called factories. The original idea take from popular Ruby gem FactoryGirl and implemented in php in FactoryMuffin library, which we use. Laravel 5 users are already aware of using factories. Now you can use them with all PHP frameworks with ActiveRecord pattern and in Doctrine.

This is how it works. You define rules to generate models:

<?php
use League\FactoryMuffin\Faker\Facade as Faker;

$fm->define(User::class)->setDefinitions([
 'name'   => Faker::name(),
    // generate email
 'email'  => Faker::email(),
 'body'   => Faker::text(),

 // generate a profile and return its Id
 'profile_id' => 'factory|Profile'
 ]);

and DataFactory creates them using underlying ORM, inserts them to database once you call

<?php
$I->have(User::class);

and deletes them after the test. As we said that will work for Doctrine as well, if you are familiar with Nelmio’s Alice, you might find the same idea but with easier syntax to use.

AngularJS

AngularJS is probably the most popular framework for building single page web applications. It provides its own tool for acceptance testing - Protractor but what if you already use Codeception and you are not so passionate to switch to NodeJS? Well if you actually do, please check out our side-project CodeceptJS which brings Codeception concepts to JavaScript world. However, in case of Angular testing there is less reasons to switch, as we brought you Protractor experience in new AngularJS module.

It simply wraps WebDriver module, and adds an asynchronous script between steps to ensure that all client-side operations finished before proceed. This way no new actions are taken before Angular finishes rendering. Also you’ve got new strict locator to use:

<?php
$I->see('davert', ['model' => 'user']);

to match elements by their ng-model.

But enough with modules, we have something more than that!

Examples

In PHPUnit you could have one test to be executed several times with different data, using the data provider mechanism. You can do the same in Codeception inside unit tests. But what about functional and acceptance testing? What if you need to run same scenario but passing different values into it? To be honest, the data provider didn’t looked like an elegant way to define a test data. Data was stored in additional method, often few pages below the original test, so it was hard to see the picture in a whole. We introduce a concept of Example, similar to what you might have seen in BDD frameworks. Using the @example annotation you can define data in test annotation and receive from Codeception\Example instance, injected into test:

<?php
  /**
   * @example(path=".", file="scenario.suite.yml")
   * @example(path=".", file="dummy.suite.yml")
   * @example(path=".", file="unit.suite.yml")
   */
  public function filesExistsAnnotation(FileTester $I, \Codeception\Example $example)
  {
      $I->amInPath($example['path']);
      $I->seeFileFound($example['file']);
  }
?>

For REST API testing this might look like:

<?php
 /**
  * @example ['/api/', 200]
  * @example ['/api/protected', 401]
  * @example ['/api/not-found-url', 404]
  * @example ['/api/faulty', 500]
  */
  public function checkEndpoints(ApiTester $I, \Codeception\Example $example)
  {
    $I->sendGET($example[0]);
    $I->seeResponseCodeIs($example[1]);
  }
?>

Data in @example annotation can be defined using JSON objects, JSON-arrays, or Symfony-style annotation. And yes, examples work only in Cests.

Custom Commands

Long requested feature that finally was implemented by Tobias Matthaiou allows you to register custom commands to Codeception runner.

If you ever created Symfony Console commands you will be familiar in creating custom commands for Codeception. You migth probably use to have your own template generators, perform data migrations, etc. You can register one as simple as you do it for extension:

extensions:
    commands: [MyApp\MyCustomCommand]

Getting current browser and capabilities in tests

The last one, simple yet useful thing that might improve your acceptance testing experience. If you want to have different behavior of tests for different browsers, you can get current browser name from a $scenario->current value:

<?php
if ($scenario->current('browser') == 'phantomjs') {
  // emulate popups for PhantomJS
  $I->executeScript('window.alert = function(){return true;}'); 
}
?>

in a similar manner you have access to all browser capabilities:

<?php
$capabilities = $scenario->current('capabilities');
if (isset($capabilities['platform'])) {
  if ($capabilities['platform'] == 'Windows') {
    // windows specific hooks
  }
}
?>

That’s all for today, but not for Codeception 2.2 The most important and most impressive feature is waiting for you in next post. Subscribe to our Twitter to not miss it. Stay tuned!


Codeception 2.2 Beta

0
0

Happy Easter, everyone! Ok, not actually everyone, but Happy Easter to those who celebrate it this weekend. If you do (or even not), you probably will have free time to play with something new. And yes, by something we mean a beta version of Codeception 2.2!

We already announced lots of changes in Codeception:

but we forgot the most important one. It is Gherkin Support, which allows you to combine business requirements with functional testing. Yes, *.feature files are now part of a family with Cest, Cept and Tests. Codeception is a tool for running all kind of tests and in this release we significantly improved test internal architecture and test formats.

But back to Gherkin.

Feature: ToDo items
  In order to get my tasks done
  As a motivated person
  I need to be able to organize my todo list

  Scenario: show current todos
    Given there are todo items:
      | Task             | State   |
      | make Gherkin PR  | opened  |
      | update docs      | opened  |
      | create examples  | closed  |
    When I open my todos
    Then I should see 2 todo items

Complete Guide on BDD with Codeception is not ready yest, but you can start with generating first feature file:

codecept g:feature <suiteName> <featureName>

We recommend to have a special features folder in acceptance or functional suite, so it could be symlinked to the features dir in root of your project. This way non-technical users can esaily access feature files, without need to examine actual tests.

Next thing to do is to describe feature with scenarios. When you are done, prepare scenario steps for implementation be running

codecept gherkin:snippets <suiteName>

You will get a list of methods which should be included into your actor class (let’s say AcceptanceTester). Then you should have it implemented. In theory, you can use any method of any class annotated with @Given, @When, @Then to be the step definition. So don’t worry you will end up with everything to put in one context, you will have option to use multiple contexts depending on role, tags, etc. More about it in BDD guide coming in next weeks.

For those of you, who set your dependencies as “codeception/codecepton:*” and (with no settings of "minimum-stability": "stable" you will probably have some wonderful time once you get back from holidays. This release is considered minor, so it won’t break everything, but still has some breaking changes. We notified of breaking changes in earlier versions by “deprecation warnings”, and we actually removed lots of deprecated stuff. The most important breaking change is proper implementation of Conflicts API. Please make sure you are not using modules of the same kinds in your configs.


Codeception 2.2-beta is available for installation only via Composer:

composer require --dev "codeception/codeception:2.2.0-beta"

Next steps to do:

Stable release will come in next week(s). The release date depends on reported issues and on progress of updating documentation.

Have nice weekends and testing time! We, Codeception team, hope you will love our product.

P.S. Reported issues should include a version. Pull Requests should be sent to master branch.

Codeception 2.2 is here

0
0

Today we are glad to announce that Codeception 2.2 is finally released. The first beta came in March and after there were 2 RC versions. We assume that everyone could already try the new branch and check its new sweet features like Gherkin support, Examples and others. We hope that you will like new version as it brings true multi format support to your testing experience! What other tool can execute Gherkin features and PHPUnit tests? We already support 4 testing formats for code and scenario testing and it is quite easy to introduce more.

Despite its growth and feature richness Codeception is still the most simple tool for test automation. We updated documentation to keep up to date with latest changes and improved the learning experiences. Also we prepared some Case Study pages which will help you to setup Codeception for your application. Now it is easier to start with Laravel and Symfony apps.

If you have questions about Codeception you can ask them on our community forum which is PhpTestClub. We plan to build a sustainable community, so If you use Codeception for a long please share your experience there, help others to get into the world of PHP testing.

How To Upgrade

We encourage you to UPGRADE to 2.2. Please take a look into changes marked as breaking.

  • The most important change is introduction of module conflicts. If modules share the same interface they probably should not be used together except the cases when the modules are loaded partially. This way you can’t use Laravel5 module with WebDriver but you can use Laravel5 with part: ORM so only ORM actions to be loaded. This change is important in order to avoid confusion in your functional or acceptance tests. If you use 2 similar modules you can’t be sure which one is executed when you call $I->amOnPage. Learn more from previous blogpost.
  • Please also note that Dbh and Laravel4 module were removed from core package and moved into separate packages.
  • Codeception\TestCase was replaced with Codeception\TestInterface to it is a good idea to replace its usages in your Helper files.
  • Cept and Cest formats are no longer extend PHPUnit_Framework_TestCase, so they don’t have expectException, getMock, etc.

Please read changelog to learn more.

But wait, you may ask, why there are breaking changes in minor release? Does Codeception follow semver? Not right now but future releases will do. Next major version is going to be Codeception 3.0 which sure will introduce breaking changes for good reason. Also we plan to decouple most of modules from core package to make faster release cycles for them. Codeception 2.2 is also marked as minor that its still support PHP 5.4. However, we encourage you to upgrade for future versions.

New Features Described


If you are using 2.1 and you won’t plan to upgrade in nearest future, you can still use releases from 2.2 branch. Minor fixes and improvements will still be accepted there. Also the site will contain documentation for 2.2 and 2.2 versions.

Try Codeception 2.2 today by installing it via Composer:

composer require "codeception/codeception: ~2.2" --dev

Writing Better Tests: Expectation vs Implementation

0
0

What makes a meaningful test? This question should always be asked. No matter we write your tests first or tests after, they may stay in a project for years and it is pretty important them to test the things that really matters. Badly written tests can slow us down by constantly failing on each implementation change, and such tests can be written no matter you follow TDD or not. The idea of a test is to ensure that software works and not to freeze it at specific point.

Such situation happens when a test is bound to implementation details. We are choosing to rely on parts which we may consider to be stable but they are not. Whenever we come to unit testing and we are writing a mock we should understand that we change the reality, and how that artificial reality will survive the test of time. Consider using such test written in XIX century as an example:

Given I am drunk 
And I am in a pub
And I want to get home
When I order a cab
Then I expect to see 2 horses and carriage
And they bring me home

By ordering a cab nowadays you probably won’t expect to see 2 horses but you probably will get home even faster. In XXI century we still have pubs and we still need cabs to get home, that’s something stable in our world. The point is: you should not be worried of HOW you will be brought home: by horses, by car, by flying dragons or by teleport. That’s implementation details.

But how to understand what is stable and what is not? We need to use interfaces. Not that one which is written as interface keyword in PHP but a general term: User Interface, API. And that’s what makes unit testing and browser testing similar: we always need to rely on public interfaces for a test. If break this rule and we start to test private methods or raw values in database we are producing a synthetic tests which will lead us the very tricky path.

An opposite to follow public APIs is to depend on implementation.

But back to software development. Here is an example of magento generators written in PhpSpec. Not saying it is bad or not, but we use it to illustrate the point.

<?php
class ModuleSpec extends ObjectBehavior
{
  function let(\Symfony\Component\Filesystem\Filesystem $filesystem)
  {
      $this->beConstructedWith($filesystem);
  }

  function it_should_create_a_folder_given_a_valid_path($filesystem)
  {
      $path = 'app/code/test/test';
      $filesystem->exists($path)->willReturn(false);
      $filesystem->mkdir($path, 0700)->willReturn(true);
      $this->make($path)->shouldReturn(true);
  }
}

The test is very descriptive: there is a class that depends on Filesystem and can be used to create a given folder. However, one line is questionable:

$filesystem->mkdir($path, 0700)->willReturn(true);

This is the mock that expects mkdir to be called with a specific parameters and to return an expected result. But is that stable to changes? What if at some point mkdir gets deprecated in favor of mkdirPlus, what if a method signature changes?

Well, in this current case we can be sure that this method won’t be changed. The reason is simple: it’s Symfony, it’s LTS, and its API is stable. But can you say that about internal classes of your application? That they are 100% documented and they won’t change their behavior in future.

When we change the implementation of Module->make() it still fits expected specification (‘it should create a folder given a valid path’), but the test fails. This happens because the test pretend to know too much. In similar manner a strict master doesn’t just ask apprentice to do the job. He thinks he knows better how to do it and provides him with a detailed instructions. He doesn’t care of the outcome but apprentice should understand the basics: disobedience will be prosecuted.

But what if the actual result is not important to us? What if we want to ensure that tested method took the right path. That what happens in a test above: the implementation is verified. Indeed, If you are master (senior developer) and you have an apprentice (junior), this works pretty well: you ask them to implement the method just the way you see it.

This makes a difference between testing behavior or testing result. Probably in most cases you want to test the real outcome, a result, but in some important areas you may need to test the behavior as well.

To test a result we should rely on public interfaces only. We can validate class only by calling public method of this class and its collaborators In this case we need to call $filesystem->exists('app/code/test/test'), and make a class with in-memory filesystem. We can even make this class without using any mocking framework, just with anonymous classes.

By making a test bound to implementation we are copying parts of that implementation into a test. This breaks the DRY principle. At some point test and code can be unsynchronized and this leads to fragility. If you change class, the test using a mock of that class may still pass, but actual code using it will fail, and this false positiveness is dangerous.

Overusing mocks also makes tests longer. Lines of test code you write doesn’t always convert to software quality. Actually, from the software developer experience the true is quite the contrary: the more code you write the more bugs you may introduce. And your tests are just the same as production code: they will evolve with your software and they needs to be maintained.

So the actionable advice here is to think what is important to you: a behavior or a result (no matter how it is achieved). Try to discover what is stable and what is not, rely on stable parts, and use public interfaces. Thanks for reading.

Written by Michael Bodnarchuk @davert.

Thanks to Nick Palamarchuk for the review and ideas.

Writing Better Tests: Obtaining Specification

0
0

When you start implement testing in your development process you should always ask: which tests are important for this project. And there is no general answer to this question. “Test everything, everywhere” is not a good solution as well, by writing more and more lines of code you won’t stable software. Tests should be driven by specification, and specification comes from your business. Depending on that you should choose which areas are more important to cover with tests and which are less.

Part 1: Expectation vs Implementation

It is always thought that you should have several acceptance tests, a dozen of functional, and a lot of unit or integration tests. However, If you develop a media portal it is more important to you, and for your business to have a good-looking UI. Same story if you develop new To-Do application focused on perfect UX. It is not much business logic for a To-Do app, but you need to ensure that everything user sees is slick. Even you have some internal bugs it is more important to you to assure that all buttons are visible, they can be easily clicked and so on.

Like in this app:

That’s right for this type of applications you should revert your testing pyramid. You should have lots of acceptance tests and few unit/integration.

If UI/UX is a part of your business, concentrate on acceptance (browser) tests. But if you run banking application, ecommerce solution, ERP or CRM, UI is not that important. Those applications include complex domain logic and internal quality is much more important than visual part.

What if user by a bug creates 3 extra todos in To-Do app? Well, it doesn’t look nice but not a big deal. But what if user by a bug withdraws from ATM 3 times more money they own? That’s a real problem. In case, you deal with money or real things it is critically important to test all the business processes. Test all possible scenarios, all edge cases, make the domain to be 100% covered with tests.

That brings us to key idea of this post. There are two kinds of IT products:

  • software that automates business
  • software that is a business by itself

And depending on the kind of software you work on you have different priorities. This also answers a question: “do I need BDD for my project”. In case, you are automating traditional business, you need to translate all business processes into a code, and you need ubiquitous languages for that. In case, you are building a new business, a startup, you probably don’t need that layer of complexity as presumably founders already speak the IT language.

So learn what is important in your business. What brings you money. What are the risks. Cover that part at first and think from business side and not from technical. This is how a business would understand the real value of automated tests.

Written by Michael Bodnarchuk

We provide consulting services on Codeception and automated testing in general.

New Addons Page

0
0

As you know Codeception has a really nice ecosystem with lots of modules and extensions built by the community. Thanks for everyone who invest their time into building better testing tools. Sometimes we accept 3rd-party modules into Codeception core, sometimes we ask contributors to publish them as standalone packages.

Today we’d love to announce a minor but very nice change on our website. We updated the Addons page so community-built modules and extensions would look nice there. The new Addons page looks as slick as a marketplace but everything there is for free!

If you have a helper or an extension that might be useful to others, don’t hesitate to share it! New “cards” format for extensions and modules will make them more visible to others. You can also attach your pictures and set colors for cards to be even more visible (If you think your extensions worth it).

The philosophy of Codeception is: don’t reinvent the wheel, share testing solutions. Spend more time writing tests and not test support code. Get more powerful tools and make testing fun!

There are some tools you definitely should not miss:

and many many others! If you don’t have a module or extension but you still have some experience to share, write a few lines to the Cookbook.

Thamks for reading. Stay tuned for next cool updates!

Codeception 2.3

0
0

Today the Codeception 2.3 sees the world. This is a stable release with almost no breaking changes but with new features you will probably like.

At first, we need to say “thank you” to Luis Montealegre for bringing PHPUnit 6.0 support to Codeception. This is done by using class aliases so this doesn’t break old PHPUnit versions, so as usual, Codeception can be used with PHPUnit 4.8 and higher. However, If you run PHP7+ and you experience PHPUnit issues in this release we recommend you to set phpunit/phpunit package to ^5.7.0 in your composer.json.

Supporting 3 major versions of PHPUnit may have its issues, so in next release, which is going to be 3.0 we will probably drop support for old PHPUnits and PHP < 7.

Ok, let’s talk about other changes in this release!

Installation Templates

Codeception is a wide-range testing framework and sometimes it is hard to get it configured for a specific case. Nevertheless, you need only acceptance testing Codeception will still install all 3 test suites for you. In 2.3 we prepared installation templates: customized setup wizards that can configure tests for your needs.

To setup Codeception for acceptance testing with browser only you can run this command

codecept init acceptance

After answering a few questions, you will be able to execute first browser test

The setup wizard will prepare only acceptance test suite with tests located in tests directory. Suite configuration will be stored in codeception.yml.

For QA engineers codecept init acceptance is a new convenient way to start end-to-end testing in PHP.

We also included API and Unit templates for faster setup of REST API or unit tests:

codecept init api
codecept init unit

But the best thing about installation templates that they can be created by anyone! They can be placed inside a separate package and loaded by init command. A template class should be placed into Codeception\Template namespace and then it can be autoloaded. Installation templates are pretty simple, learn how to build your own by taking a look at Acceptance template as an example.

We have added Installation templates to Customization guide and added a reference to Codeception\InitTemplate class.

Configuration Improvements

Suites Inside Main Config

Suites can be defined inside main config:

actor_suffix: Tester
paths:
    tests: .
    log: _output
    data: _data
    support: _support
suites:
    unit:
        path: .
        actor: UnitTester
        modules:
            enabled:
                - Asserts

A good option if you have a single suite.

Suite Paths

The suite can have its custom path (specified by path). From config above expects unit tests to be placed into the root directory, where codeception.yml is located.

selection_201

Suites With No Actor

Suite can be defined without an actor, which is useful for unit testing

paths:
    tests: tests
    log: tests/_output
    data: tests/_data
    support: tests/_support
    envs: tests/_envs
suites:
    unit:
        path: .
        modules:
            enabled:
                - Asserts

In this case, UnitTester won’t be created, as well as _generated/UnitActions. However, such suites won’t be able to use Cest and Cept files.

Naming Changes

class_name suite in suite config replaced with actor:

class_name: UnitTester => actor: UnitTester

actor from global config is replaced with actor_suffix config option (which makes more sense).

All these changes are backward compatible, so old values in config will work.

Extensions

Dynamical loading of extensions was already with --override option but was not very usable. Now extensions can be loaded with --ext option:

codecept run --ext Recorder

or by providing a full class name

codecept run --ext "Codeception\Extension\Recorder"

This can be used to enable a custom reporter. For this reason, the new DotReporter has been added:

codecept run --ext DotReporter

selection_205

Db Populator

From the early days of Codeception we had the Db module which was trying to do its best to populate database and clean up it between tests. However, parsing all possible SQL dialects and running them through PHP was not very effective. What if you could use native Database tools to import data instead of doing it from PHP? Why not!

In Codeception 2.3 we recommend to specify a command to load a database in populator option of Db module:

 modules:
    enabled:
       - Db:
          dsn: 'mysql:host=localhost;dbname=testdb'
          user: 'root'
          password: ''
          cleanup: true # run populator before each test
          populate: true # run populator before all test
          populator: 'mysql -u $user $dbname < tests/_data/dump.sql'

This approach is system-dependent, you can’t use the same config on Windows and Nix systems, but is much faster. Thanks to Mauro Asprea @brutuscat for this feature.

Db module defaults

Important notice: we changed defaults for Db module, so cleanup and populate options are disabled by default. They were quite dangerous in use, so we decided that you need to set them explicitly in Db module config.


Codeception 2.2.12 has been released as well. See complete changelog for all notable changes.


P.S. Codeception is seeking for Symfony module maintainer. If you use Codeception with Symfony and you’d like to improve it, please contact us at team@codeception.com. Maintainer responsibilities are: review issues, pull requests, update symfony demo app and so on. Take a part in project development and make open source brighter!

Drive your Browser Tests with Codeception

0
0

In last few weeks Codeception received updates aimed to empower acceptance testing. We try to make PHP a better place for browser tests. As you know, QA engineers often prefer other languages like Java or Python for test automation. But PHP is not that bad by itself, it is simpler, it is faster in most cases, it has great IDEs. And for sure, we have Codeception. With it you can write the most complex acceptance tests in a simple scenario-driven manner.

So what was added in the last release?

RunProcess

RunProcess extension was introduced. Use it to easily start/stop Selenium Server, ChromeDriver, or other required services for a test suite.

Enable it in suite config or in environment:

# acceptance.suite.yml
extensions:
    enabled:
        - Codeception\Extension\RunProcess:
            - php -S 127.0.0.1:8000
            - java -jar selenium-server.jar

# or inside environment config
# in this case, run tests as
#
# codecept run --env selenium
env:
  selenium:
    extensions:
        enabled:
            - Codeception\Extension\RunProcess:
                - php -S 127.0.0.1:8000
                - java -jar selenium-server.jar           

Depending on the environment (dev host, CI server) you can easily switch setups if you use environment configs.

SmartWaits

This is the new unique feature of Codeception which incorporates implicit waits. By itself, the implicit wait was available in Codeception with wait: option in WebDriver config. However, it was not usable, because by design it slowed down test execution. In this release, we introduce the SmartWait concept. Implicit waits are used only when they are really needed and disabled for all other cases. This makes tests extremely stable and fast.

Thus, a test won’t fail if expected element didn’t yet appear on a page but waits for it a few seconds more. Just set wait: 5 to WebDriver config to try it and read the documentation.

Customization

Codeception took to much of browser control. Let give it back to you. With start: false option you can disable autostart browser before a test, and create a browser session manually. Codeception doesn’t provide actions to start browser session inside a test (because it is supposed you already have one in a test), but you can write a custom method in a helper:

<?php
// support/Helper/Acceptance.php:
// 
public function startBrowser()
{
    $this->getModule('WebDriver')->_initializeSession();
}

public function changeBrowser($browserName)
{
    $this->getModule('WebDriver')->_restart(['browser' => $browser]);
}

public function closeBrowser()
{
    $this->getModule('WebDriver')->_closeSession();
}

Then these methods can be used inside a test.

<?php
// I prepare something for a test
$I->startBrowser();
// I do something inside a browser
$I->closeBrowser();

If you use BrowserStack you can use this features to set a test name dynamically to capabilities. Here is how can you do it in Acceptance Helper:

<?php
// inside Helper\Acceptance
// 
public function _before(TestInterface $test)
{
    $webDriver = $this->getModule('WebDriver');
    $name = $test->getMetadata()->getName();
    $webDriver->_capabilities(function($currentCapabilities) use ($name) {
        $currentCapabilities['name'] = $name;
        return new DesiredCapabilities($currentCapabilities);
    });    
    $webDriver->_initializeSession();
}

Please don’t forget to set start: false in config option, so browser wouldn’t be started twice!

@prepare

What if you need to change the configuration for a specific test? Let’s say you want to run all tests in Firefox but this specific one in Chrome? Or something like that.

We added new annotation @prepare which can be used in Cest and Test formats. It will execute a method which can change the configuration before the module is called.

<?php
/**
 * @prepare switchToFirefox
 */
public function testSomethingInFirefox(AcceptanceTester $I)
{
    $I->amOnPage('/');
    $I->click('Detect Browser');
    $I->see('Firefox');
}

protected function switchToFirefox(\Codeception\Module\WebDriver $webDriver)
{
    $webDriver->_reconfigure(['browser' => 'firefox']);
}

This @prepare can be used not only for browser tests but everywhere. Use it wisely!


Is that all? Almost.

We also updated WebDriver docs to include more options for Local Testing, like ChromeDriver. We also published a reference on running Chrome in Headless mode.

Update to the latest 2.3.4 version to try all the new features.

We worked hard to bring all this stuff to you. Now is your turn: please help to spread the word and encourage more of your colleagues to use PHP and Codeception for web testing. Setting up Codeception for web tests nowadays is as simple as running:

codecept init acceptance

Happy testing!


Writing Better Tests: Riding The Command Bus

0
0

Before writing any line of test code we should think of how meaningful this test would be. The best code is the code you never write, you know. So if you ever begin to think of which parts of the application you should cover with tests.

Let’s start with a heart of your application, the Business Logic.

Layered Architecture

Depending on architecture Business logic could be in Controller (ough), in Model (ough-ough), or in a special Service layer, which is the most preferred way. Business Logic responsible for taking decisions: which entities are valid and which are not, who can create entities, what happens when the entity is stored. Service layer should delegate the database operations to the infrastructure level.

Infrastructure level is purely technical. It just takes orders from Service layer and executes them. It doesn’t make any decisions on its own. In most cases, ORM should be at the infrastructure level.

While models or repositories do not contain any decision making logic it is not that important what ORM type to use: ActiveRecord or DataMapper.

CommandBus and Domain Events

One of the solution to separate business logic from everything would be the CommandBus architecture.

CommandBus is well-known in PHP community. Learn more about it from

If the application implements CommandBus pattern it is easy to find the business logic. It is located in command handlers. So let’s test them!

Testing CommandHandler

CommandHandler takes a command, calls the infrastructure services, and sends the domain events. In theory, CommandHandlers can be tested in isolation, with unit tests. Infrastructure services can be mocked and domain events can be caught. However, those tests may have a little sense. And here is why/

“Trust me, I will save this to database” - may say the test with mocked infrastructure. But there is no trust to it. To make the command handler reliable and reusable we need to ensure it does what it is expected to do.

Also, mocks comes with their price. While integration test is really similar to the actual business code, unit test is bloated with mock definitions. It will get hard to maintain and support it really soon:

<?php
// is that a business logic test?
// but no business inside of it
$eventDispatcher = $this->createMock(EventDispatcher::class);
$busDispatcher = $this->getMockBuilder(BusDispatcher::class)
   ->setMethods(['dispatch'])
   ->getMock();

$busDispatcher->expects($this->exactly(2))
   ->method('dispatch')
   ->withConsecutive(
       [$this->isInstanceOf(CreateAccount::class)],
       [$this->isInstanceOf(CreateCompany::class)]
   );

$handler = new RegisterUserHandler($eventDispatcher, $busDispatcher);
// ... 

Here we also mix implementation with a specification, which is a pure sin. What if dispatch method will be renamed? What if we fire more than 2 commands in a call? How is this related to business?

Even you can mock services you shouldn’t always do it.

In most cases, business logic should be tested with an integration test. Because the database is an essential part of your application. You can’t deliver an app without a database. The same way you can’t make sure domain logic works as expected when nothing is stored in the database.

Testing Flarum Forum

As an example let’s pick Flarum project which is a forum built on top of Illuminate components (Laravel) and Symfony Components. What is most important that it has the Commands and CommandHandlers. By looking at one directory we can learn what Flarum does. That’s awesome!

Looks like the #1 priority is to get all those Command Handlers tested. We can use StartDiscussionHandler to start.

For integration tests, we need to initialize Application with its Dependency Injection Container. Then, we fetch StartDiscussionHandler out of it:

<?php
protected function _before()
{
    // initialize Flarum app
    $server = new \Flarum\Forum\Server(codecept_root_dir());
    $app = $server->getApp();
    // get StartDiscussionHandler from container
    $this->handler = $app->make(StartDiscussionHandler::class);
}

When the handler is prepared we can write the first basic test:

<?php
public function testAdminCanStartADiscussion()
{
    // ARRANGE: create command object with all required params
    $admin = User::find(1); // User #1 is admin
    $command = new StartDiscussion($admin, [ // create command
        'attributes' => [
            'title' => 'How about some tests?',
            'content' => 'We discuss testing in this thread'
        ]
    ], '127.0.0.1');
    // ACT: proceed with handler
    $discussion = $this->handler->handle($command);
    // ASSERT: check response
    $this->assertNotNull($discussion);
    $this->assertEquals('How about some tests?', $discussion->title);
}

How do you like this test? This test so tiny and so easy to read. Sure, we should add more assertions to find out that all the business rules are applied, but for now it’s ok to try the very basic scenario. Maintaining and extending this test will be a pleasure.

A bit of Stubs and Mocks

Integration test represents a part of a system, a working component. Once it triggers the outer service, it should be mocked. In layered architecture, a class from a layer should have access to its neighbors, and to classes of lower levels. So in our case, CommandHandler of business logic can access other command handlers but should be banned from accessing other services or other command buses.

For sure, mailers, queues, and other asynchronous services should be replaced with stubs.

Conclusion

Your application has a heart. Don’t make it die from a heart attack. Write tests. Write meaningful stable tests that will last.

Written by Michael Bodnarchuk

We provide consulting services and trainings on Codeception and automated testing in general.

Codeception 2.4 released

0
0

Hello everyone! We’d like to announce the immediate availability of Codeception 2.4.0.

This follows up the PHPUnit release and mostly contains compatibility fixes for PHPUnit 7. Internal usage of PHPUnit in Codeception was switched to new namespaced class names. This doesn’t break any public APIs but can break some setups. That’s why this is a minor release with a proper announcement. Please upgrade and unless you have some PHPUnit version conflict you should not experience any issues.

Upgrade Notice: If you have tests extending PHPUnit_Framework_TestCase (old-style names) you have two options:

  • Lock version for PHPUnit in composer.json: "phpunit/phpunit":"^5.0.0"
  • Update your codebase to extend tests from Codeception\Test\Unit (which won’t be changed in future).

Codeception 2.4.0 is a stability release. We dropped unmaintained PHP 5.4 and PHP 5.5. But we keep compatibility with PHP 5.6 so if your company still uses PHP 5 (and we understand business reasons) you can install this update too. Codeception is still going to be maintained with PHP 5.6 compatibility in mind. We don’t break API just for the sake of clean code. We try to make things reliable.

In the same Codeception 2.3.9 is released, so if you want to get the latest changes and you still use old PHP, upgrade to this version.

In Codeception 2.4 you will also find new hooks for Cest files:

  • _passed is called when a test inside Cest is successful
  • _failed is called for unsuccessful tests
  • _after is called at the end of a test for both cases.

Previously _after hook worked as _passed and that behavior was confusing. This was changed so it can be potentially a breaking change as well.

See changelog for the complete list of fixes and improvements.

Thanks to all our contributors who made this release happen!


We also launched Codeception Slack to talk about testing and Codeception.

Join Slack

We’d also love to see more contributors there. We need more talented developers to help to bring new features and fix the issues. It’s opensource, after all. We face constant challenges: new APIs, new libraries, new approaches, etc. And Codeception as a really big project would benefit from any help. We are open to accepting new maintainers, to discuss trends, to build a better product! As you know, we are merging 90% of submitted Pull Requests so if you are not contributing, please go ahead!


We try to keep Codeception going and bring more releases to you. If your company uses this framework and you’d like to give back please consider sponsoring Codeception. That’s right. We are asking to invest into open source, to get the features you expect and to give back to open source.

Sponsor Codeception

Please show this form to your company bosses. If you want to have stable reliable product it’s ok to pay for that. Be our first official sponsors!

Yes, we also provide enterprise support and trainings. This is another way you can support the development. Thank you!

Codeception 2.5: Snapshots

0
0

The Autumn is a good time to start something new. How about new Codeception? As you know, each minor release indicates some important change, new features we wanted to share. This time we want to introduce you to

Snapshots

That’s the new feature, which might be useful if you are tired of harcoding your data into fixtures. For instance, when you test the search engine you don’t know the exact list of items to check, however, you are interested that the list would be the same every time for a search term. What about API responses? You don’t want to hardcode it fully but you may want to check that response is the same as it was before.

So now Codeception can do what php-vcr was doing all the time. Saving data into the file (called snapshot), and comparing with it on next runs.

This is nice feature for testing REST APIs.

Let’s assume you have such JSON response.

{
  "firstName": "John",
  "lastName" : "doe",
  "phoneNumbers": [
    {
      "type"  : "iPhone",
      "number": "0123-4567-8888"
    },
    {
      "type"  : "home",
      "number": "0123-4567-8910"
    }
  ]
}

And you want to check that phone numbers are the same on each run. For this we can use a snapshot.

namespace Snapshot;

class PhoneNumberSnapshot extends Codeception\Snapshot {
  
    /** @var \ApiTester */
    protected $i;

    public function __construct(\ApiTester $I)
    {
        $this->i = $I;
    }

    protected function fetchData()
    {
        // return an array of phone numbers
        return $this->i->grabDataFromResponseByJsonPath('$.phoneNumbers[*].number');
    }  

}

Then in test we can check if data matches the snapshot by calling the snapshot class:

public function testUsers(ApiTester $I, \Snapshot\PhoneNumberSnapshot $shapsnot)
{  
  $I->sendGET('/api/users');
  $snapshot->assert();
}

If the data changes, snapshot is easy to update. Just run the test in --debug mode. The snapshot will be overwritten with a new data.

So, the good thing about snapshots:

  • you don’t keep flaky data in your code
  • you don’t need to hardcode data values
  • data can be easily updated on change

Use them!


There are also some other minor changes:

  • Db module now supports multiple database connections. If you use few databases we got you covered! That was a long awaited feature and finally we have a very nice implementation from @eXorus.
  • --seed parameter added to run so you could rerun the tests in same order. This feature works only if you enabled shuffle mode.
  • Possible breaking: seeLink behavior changed.
    • Previous: $I->seeLink('Delete','/post/1'); matches <a href="/post/199">Delete</a>
    • Now: $I->seeLink('Delete','/post/1'); does NOT match <a href="/post/199">Delete</a>

See changelog for the complete list of fixes and improvements.

Thanks to all our contributors who made this release happen!


Call to Sponsors!

We try to keep Codeception going and bring more releases to you. If your company uses this framework and you’d like to give back please consider sponsoring Codeception. That’s right. We are asking to invest into open source, to get the features you expect and to give back to open source.

For instance, how would you like improving stability of WebDriver tests? It would be cool to automatically retry failed steps and rerun failed tests. These could be a very cool features but they can’t be made without your help.

If you are interested consider sponsoring us:

Sponsor Codeception

Yes, we also provide enterprise support and trainings. This is another way you can support the development. Thank you!

Codeception 3.0

0
0

It’s finally time to bring a new major version of Codeception to life! Through years of evolution and constant improvements we learned a lot and today we think we are bringing to you the best Codeception version so far. We still belive that simplicity of testing is important, that test code should be easy to read, write, and debug. And if you are reading this post you probably already discovered that.

Maintaining such project that tries to embrace all kind of frameworks and CMSes, from Symfony to WordPress, from Laravel to Magento, is challenging, but what we see that people from those communities send us patches regularly. Probably, our philosophy - separate tests from the framework, share similar solutions via modules, shows the way. And our awesome community continue to improve this project. Hey, just look into our changelog. Even patch releases contain HUGE list of improvements!

Ok, so what about Codeception 3.0?

Breaking Changes

PHPUnit 8 Support

In 3.0 release we didn’t break a lot of stuff. We tried, but probably we just leave that for next major release. So we do not bump supported PHP version. We are still PHP 5.6+ compatible, just because testing should be available for everyone. We still support all major frameworks and we keep PHPUnit 6, and PHPUnit 7 compatibility.

However, keeping all that parts together is hard. So we assume, in 3.0 you can get possible breaking change, as we bring PHPUnit 8 support. For everything else, the upgrade should be smooth for you. If you face issues upgrading, go and change phpunit version to 6 or 7 in your composer.json:

"phpunit/phpunit": "^7.0"

We say thank you to our core contributor @Naktibalda to bring PHPUnit 8 support without breaking compatibilitiy. That was huge job, and if you look into our codeception/phpunit-wrapper project you will understand why.

Modules Removed

We also decided to drop some of old modules, which we hope no one uses. Here they are

  • AngularJS - it worked only for Angular 1
  • ZF1 - framework outdated
  • Yii1 - framework outdated
  • Silex - framework outdated
  • Facebook - module not maintained
  • XMLRPC - module not maintained

If you need them, just copy their code from the 2.5 branch and create a custom helper. However, we won’t support them any longer.

Changed Defaults

If you use multi-session testing and $I->haveFriend commands, you will see your tests fail. Friend methods no longer included into Codeception\Actor class, so you should add them manually. In your AcceptanceTester class (or other class which uses multi session testing) include Codeception\Lib\Actor\Shared trait:

<?php
class AcceptanceTester extends \Codeception\Actor
{
    use _generated\AcceptanceTesterActions;
    use \Codeception\Lib\Actor\Shared\Friend;
}    

We also disabled conditional assertions for new setups (as people regularly misuse them), so if you want to use canSee methods you will need to enable them. We will take a look on that in next sections of this post.

Features

M-m-m… Now the most tasty thing. Yes, we wouldn’t do the major release without new features. Thanks to sponsorship from Seravo we could finish some features we had in our roadmap.

Improved Interactive Shell

This feature was backported from our friendly project CodeceptJS. It brings a new full featured REPL interface into a test, so you could pause test execution and fire different commands into console.

This feature absolutely changes the way you write your functional and acceptance tests. Instead of blindly trying different commands and restart tests all the time, you could write:

$I->amOnPage('/');
$I->pause();

And start typing commands one by one, writing a test step by step. You copy successful commands into your tests, and in the end you get a fully working test. If you use WebDriver, you could write a complete acceptance test using one browser session.

Unlike, previous interactive shell implementation, this one based on hoa/console, so you can use Left, Right keys to edit your input. Better to show it.

Learn more about using interactive shell in our updated Getting Started guide

Try & Retry

Those features were introduced to make browser testing less painful. In the world full of JavaScript and Single Page Applications, you can no longer rely on single click, or fillField commands. Sometimes you need to retry action few times to make perform it.

Now you can do that via retry* actions which can help to stabilize flaky steps in your tests:

<?php
// use
$I->retryClick('Create');

// instead of just
$I->click('Create');

This feature was also ported from CodeceptJS but implemented a bit differently. Learn how to use it in updated Acceptance Testing guide.

What if your site behaves differently in different environments? Like showing the “Cookie” badge depending on region? And you need to accept cookies notifications if the badge is shown? You can do it via tryTo* actions:

<?php
if ($I->tryToSeeElement('.alert.cookie')) {
  $I->click('Accept', '.cookie');
}

Thanks to Iain Poulson from Delicious Brains for showing this use case

Learn how to use it in updated Acceptance Testing guide.

Try and Retry were implemented as Step Decorators. As you know, actor classes contain code generated parts, and step decorators allow to generate additional actions by wrapping methods of modules. Conditional Assertions (canSee* methods) were refactored as step decorators too. So you need to enable them explicitly on new setups.

Artifacts in Output

Failed tests now contain list of all available artifcats. For REST API test each test will contain a body of last response to simplify debugging on CI servers.

Install or Upgrade

Codeception 3.0 is as easy to upgrade as bumping version to ^3.0 in composer.json:

"codeception/codeception": "^3.0"

Also please read the changelog for more details.

Final Notes

Thanks for everyone who have been with us all this years and who helps making Codeception better every day. Again, thanks to Naktibalda for taking care of maintaining minor releases and building compatibility bridge.

Thanks to Otto Kekäläinen from Seravo for generous donation.

And a few words from Michael Bodnarchuk, who started Codeception 8 years ago:

As a person who is interested in quality and testing I was interested in answering the question: what needs to be tested at first place. And the best answer I found so far is to get that information from the user feedback. Sometimes, a bug can be found only by a regular user, without causing a stack trace. And what I’d love to see, that users of web site or web application could submit such bugreports instantly. That’s why I built Bugira Bugtracker, my new SaaS product to get bugreports from your users. If you like my projects like Codeception or CodeceptJS, please try Bugira as well. It will help you to establish better communication with you users and build a better products!

Codeception 4.0 and beyond

0
0

Today we would like to share our plans for the future of Codeception. We are going to release two major versions soon and we are ready to do so! If you want to stay on board and get the latest updates, please read this post carefully. We will explain the reasoning behind the new release process and the upgrade path, which should be pretty simple to you!

As you know, we are very proud of our policy of keeping backward compatible releases. We know that tests should be stable and reliable at first. If a business is still powered by PHP 5.6, this is a business issue, and we can’t force a company to upgrade all their servers and processes just to support some more testing.

Codeception 4.0 is coming, 5.0 is on the way. Be prepared!

However, keeping backward compatibility is harder and harder because of our dependencies - PHPUnit & Symfony. They all go forward, break API and require newer versions of PHP and to follow them we need to do the same.

But can’t leave you without upgrades even if you don’t plan to switch to the latest Symfony or PHPUnit. That’s why we announce decoupling of Codeception.

That’s right! Since Codeception 4.0 the core will be separated from Codeception modules and extensions. This allows you to get the latest updates for the modules you use while keeping running Codeception on PHP 5.6+ with your set of dependencies.

See all modules decoupled at Codeception organization

After Symfony 5 is released we will release Codeception 5.0, with a direct dependency on Symfony 5 components. If you never plan to upgrade to Symfony 5 you can keep using Codeception 4, as (again) all modules are independent of the core. Codeception 5 (and upcoming versions) will support only the latest major versions of Symfony & PHPUnit.

Transforming into a new release takes time and requires hard work from the Codeception team. Many thanks go to Gintautas Miselis @Naktibalda for doing the hardest work on splitting modules into repositories. Maintaining many repositories instead of one also requires more involvement from us. It would be easier for us just to release major versions every year and ask you to upgrade. But we didn’t do that. We care about you.

That’s why we ask you to sponsor development of Codeception. Please talk to your boss, to your CTO, to your marketing team, to your CEO. Tell that reliable software comes with a cost and if tests save your life, it’s a good time to give back!

Now you donate in a few clicks on OpenCollective!

Sponsor Codeception

Sponsors receive priority support from our team, can ask for consulting, and add their logo on our website! Over 15K PHP developers visit our website monthly. If your brand needs to be recognizable in the PHP community - sponsoring Codeception is a very good opportunity for you!


From your side, we will prepare an upgrade script from version 3.x to 4.x. So once Codeception 4.0 is released you will be able to run a smooth upgrade.

We won’t accept patches to 3.1 branch, so please send your pull requests to the corresponding module repositories or to 4.0 branch of the core.

A few notes on a new release process:

  • Each module will be released independently. You won’t need to wait for a month for a patch.
  • Documentation for modules on codeception.com will be updated each month.
  • Phar will contain only framework-agnostic modules (PhpBrowser, WebDriver, REST, Db) and will have a rolling weekly release.

Official 4.0 release announcement coming soon as we need to prepare our website site and update documentation. Stay tuned and consider donating Codeception. We require your support these days!

Codeception 4.0 Released

0
0

We are finally here!

Codeception 4.0 is released and this means a lot for us!

After long months on splitting core into modules, publishing new packages and setting up build pipelines, we are proud to show you the result.

As you know - Codeception 4 changes the way it is installed. From now on, Codeception package won’t include any modules by default and you will need to install them manually. This can be a bit annoying at first but in the end, this will allow you to upgrade each package individually and not relying on the core.

In Codeception 4 we also added support for Symfony 5, so if you plan to use the latest Symfony components - you need to upgrade today.

Upgrade to Codeception 4

To start upgrading to Codeception 4 bump a version in composer.json and run composer update.

"codeception/codeception": "^4.0"

To simplify the upgrading process we introduced an upgrade script which will go through your config file and scan for all included modules. Corresponding packages will be added to composer.json, so you could install all Codeception modules you need.

Run upgrade script on current project:

php vendor/bin/codecept init upgrade4

The script will upgrade your composer.json file and it is recommended to run composer update immediately, so new modules will be installed.

If you use several Codeception configs for multiple bundles you will need to execute upgrade script for each of them:

php vendor/bin/codecept init upgrade4 -c src/app/frontend/

Congratulations! An upgrade is done. Now you will be able to use each module individually.

Thanks to Gintautas Miselis @Naktibalda for splitting the core and adding Symfony 5 support

Phar Distribution

Codeception 4.0 will provide two phar files with selected modules included. One file will target PHP 7.2+ and include Symfony 4.4, PHPUnit 8 and newer versions of other libraries. Another file will target PHP 5.6 - 7.1 and include Symfony 3.4 and PHPUnit 5.7.

Target audience of phar files is acceptance testers, so framework modules won’t be included. Here is the full list of bundled modules:

  • Amqp
  • Apc
  • Asserts
  • Cli
  • Db
  • FileSystem
  • FTP
  • Memcache
  • Mongodb
  • PhpBrowser
  • Queue
  • Redis
  • Rest
  • Sequence
  • Soap
  • Webdriver

Phar files will no longer be built on release of codeception/codeception, but on more ad-hoc basis either monthly or after important changes to core or some bundled module.

What’s new in 4.0

Since 4.0 our longread changelogs are gone. So to track changes in modules you will need to go to corresponding repositories and read changelogs. For instance, here is the changelog for Db module.

However, we will still publish changelogs for core changes here. And one of the most interesting features coming to Codeception 4 is command stashing in interactive pause.

Codeception 3.0 introduced interactive pause for better writing and debugging tests. By adding $I->pause() to your tests you could try all Codeception commands while a test will be interrupted. This helps for browser testing and framework testing, so you could try commands in action before leaving a test.

In Codeception 4.0 this instrument was improved so you could automatically stash all successful commands:

Use hotkeys to save successful commands into a file. So you no longer need to keep in mind which commands you need to take into your test. Just copy all successful commands and paste them into a test.

Stashing was brought to you by Poh Nean


Thank you for using Codeception, and thank you for staying with us these years. It is December 2019, so it means that Codeception turns 8. Through these years we reached 15Mln installations on Packagist, we are used in some well-known companies and we became a valuable part of PHP ecosystem.

We accept congratulations in Twitter or on OpenCollective. If your business gets some value from Codeception it is ok to support its maintainers. Especially, today, when we have to maintain not one repository but 32! So, if possible, talk to your boss and ask for sponsoring Codeception. This means a lot to us, Codeception Team, and it will motivate us to work more on Codeception.

We say “thank you” to all our current and previous sponsors:

  • Thanks to ThemeIsle for becoming our first regular financial contributor
  • Thanks to Sami Greenbury, Lars Moelleken, Dan Untenzu for investing their own money into open-source
  • Thanks to Rebilly and 2Amigos who fully supported the development since 2013 till 2016
  • Thanks to Seravo for sponsoring Codeception 3.0 release.

That’s all. Upgrade to Codeception 4 and improve your testing!

Codeception 5

0
0

Codeception 5.0 is out!

This release is PHP 8+ only, so we are back on track with modern PHP. We are dropping support for PHPUnit < 9, and are technically ready for PHPUnit 10. And we also support Symfony 6 without dropping support of previous Symfony versions. As always, we did our best to keep backward compatibility so if you can update your dependencies, all tests should be working for you.

So let’s take a look at some new features:

New Directory Structure

Codeception 5 will follow the PSR-12 coding standard. So all tests and classes will have their own namespace Tests. The directory structure was updated accordingly:

tests/
  _output
  Acceptance
  Functional
  Support/
    _generated/
    Data/
    Helper/
  Unit/

All suite names have their own namespace, as well as actor and helper classes:

<?php

namespace Tests\Acceptance;

use \Tests\Support\AcceptanceTester;

class LoginCest
{
    public function tryToTest(AcceptanceTester $I)
    {
        $I->amOnPage('/');
    }
}

This new directory structure will be generated by running codecept bootstrap.

Upgrading from Codeception 4

Codeception 5 is compatible with the Codeception 4 directory structure. So if you don’t want to change your file locations now, just keep your existing codeception.yml, and everything’s fine.

However, if you want to upgrade to the new Codeception 5 directory structure (recommended), here’s the new default codeception.yml:

namespace: Tests
support_namespace: Support
paths:
    tests: tests
    output: tests/_output
    data: tests/Support/Data
    support: tests/Support
    envs: tests/_envs
actor_suffix: Tester
extensions:
    enabled:
        - Codeception\Extension\RunFailed

Next steps:

  1. Rename your suite configuration files: acceptance.suite.yml => Acceptance.suite.yml functional.suite.yml => Functional.suite.yml unit.suite.yml => Unit.suite.yml
  2. Inside those configuration files, update to the new namespace:
     modules:
         enabled:
             - Tests\Support\Helper\Unit
    
  3. In your composer.json, update to the new namespace:
     "autoload-dev": {
         "psr-4": {
             "Tests\\": "tests/"
         }
     },
    
  4. In your tests/Support/Acceptance|Functional|UnitTester.php files, update to the new namespace.
  5. Run vendor/bin/codecept build to create the files in tests/Support/_generated.
  6. Modify the namespaces in all your test/cest files.
  7. Run the tests with capitalized suite names: vendor/bin/codecept run Unit

Attributes

Annotations were an essential part of the Codeception testing framework. Even though they were not native language constructs, they proved to be quite good to separate a test from its metadata. We believe that test should not include code that doesn’t belong to the test scenario.

So we were glad that native attributes have landed in the PHP world. In this release we encourage you to start using them:

#[Group('important')]
#[Group('api')]
#[Examples('GET', '/users')]
#[Examples('GET', '/posts')]
#[Env('staging-alpha')]
#[Env('staging-beta')]
#[Env('production')]
#[Prepare('startServices')]
public function testApiRequests(ApiTester $I, Example $e)
{
  $I->send($e[0], $e[1]);
  $I->seeResponseCodeIsSuccessful();
  $I->seeResponseIsJson();
}

As you see, attributes decouple all preparation steps, keeping the test scenario minimal. We also keep supporting annotations, so an urgent upgrade is not needed. Attributes can’t do anything that traditional annotations can’t, they are just a modern alternative.

List of available attributes (all under Codeception\Attribute namespace):

  • Before - specifies a method that should be executed before each test
  • After - specifies a method that should be executed after each test
  • Group - sets the group for the test
  • Skip - skips the current test
  • Incomplete - marks test as incomplete
  • Depends - sets the test that must be executed before the current one
  • Prepare - sets a method to execute to initialize the environment (launch server, browser, etc)
  • DataProvider - specifies a method that provides data for data-driven tests
  • Examples - sets data for data-driven tests inside the annotation
  • Env - sets environment value for the current test
  • Given, When, Then - marks a method as BDD step

Debugging

Do you remember, Hoa\Console? Unfortunately, this library was deprecated and we were looking for a modern alternative that could power codecept console and $I->pause();. We switched to PsySH, a PHP Read-Eval-Print Loop.

An interactive console is used to pause a test in the given state. While in pause you can try different Codeception commands, and check variable values. Instead of fixing tests blindly, you can start an interactive session. This is quite a similar effect you can get with a real debugger like XDebug but focused on Codeception commands. This is especially helpful for acceptance tests as the test scenario can be planned while executing a test. So the basic scenario can be written as:

$I->amOnPage('/');
$I->pause();

After opening a page you will be able to try commands in a browser. If a command succeeds you can use it in your tests.

Also new functions were added:

  • codecept_pause() - starts interactive pause anywhere in debug mode
  • codecept_debug() - prints a variable to console using Symfony VarDumper

Upgrading from Codeception 4

Just remove hoa/console from your composer.json.

Sharding

The Parallel Execution guide has been rewritten and focused on a new feature: sharding. It is the simplest way to run slow tests (think of acceptance tests first) in parallel on multiple agents.

In this case, you specify the batch of tests that should be executed independently and each job picks up its own not intersecting group of tests to run them.

# first job
./vendor/bin/codecept run --shard 1/3

# second job
./vendor/bin/codecept run --shard 2/3

# third job
./vendor/bin/codecept run --shard 3/3

This feature reduces the need for complex configuration and usage of robo task runner to split tests.

It is recommended to use sharding to parallelize tests between multiple jobs as the simplest approach. Unfortunately, PHP doesn’t have native multi-threading for test parallelization, and even if it had, it doesn’t solve the problem of running slow browser tests that are interacting with the entire application. So only horizontal scaling by jobs can be suggested as a long-running approach. The more build agents you add to your Continuous Integration server, the faster tests will run. That’s it!

Grep and Filter

New options --grep and --filter were introduced to select tests by part of their name. Actually, it is the same option and an alias. --grep is a common way to select tests to execute in NodeJS test runners, so we ported it to Codeception. But as usual, specific tests can also be executed by group or specifying a test signature.

./vendor/bin/codecept run --grep "user"

Other Changes

Please go through the list of changes introduced to see if they don’t affect your codebase:

  • Requires PHP 8.0 or higher
  • Compatible with PHPUnit 9 and ready for PHPUnit 10
  • Compatible with Symfony 4.4 - 6.0
  • Stricter check for phpdotenv v5 (older versions are not supported)
  • Throw exception if actor setting is missing in suite configuration
  • Removed generate:cept command (Cept format is deprecated)
  • Removed settings disallow_test_output and log_incomplete_skipped.
  • Removed setting paths.log (replaced by paths.output in Codeception 2.3)
  • Removed suite setting class_name (replaced by actor in Codeception 2.3)
  • Removed global setting actor (replaced by actor_prefix in Codeception 2.3)
  • Removed Configuration::logDir method (replaced by Configuration::outputDir in 2.0)
  • Moved XmlBuilder class to SOAP module
  • Decoupled test execution and reporting from PHPUnit
  • Custom reporters implementing TestListener are no longer supported and must be converted to Extensions
  • Added optional value to fail-fast option (#6275) by #Verest
  • Removed JSON and TAP loggers
  • Removed code coverage blacklist functionality
  • Removed deprecated class aliases
    • Codeception\TestCase\Test
    • Codeception\Platform\Group
    • Codeception\Platform\Group
    • Codeception\TestCase
  • Introduced strict types in the code base.

Complete Changelog


We are really happy that we are finally here with Codeception 5. This release was crafted during the war that happens in Ukraine. It is mentally and morally hard to work on tech products knowing that this peaceful virtual life can end at any moment by a random missile. Codeception was created in 2011 by Michael Bodnarchuk in Kyiv, and today in 2022 he also stays there writing this post. If you want to support Codeception, all the Ukrainian PHP community, and all our brave nation who stands for democracy against this barbaric Russian invasion, consider donating to Ukrainian charities. Not a single time. Every month until the war ends. Every time you travel or enjoy tasty food in a restaurant think of people who are forced to defend their land, or who fled their homes. Glory to Ukraine!

This release wouldn’t be possible without the hard work of Gintautas Miselis who keeps constant work on modernizing internals and keeping Codeception up to date. Also we are really thankful to Gustavo Nieves who did a lot of work transitioning Codeception to Symfony 6 and more! Thanks to our maintainers! If you want to support our work we have OpenCollective!






Latest Images