Codeception 5

Codeception 5.0 is out!

This release is PHP 8+ only, so we are back on track with modern PHP. We are dropping support for PHPUnit < 9, and are technically ready for PHPUnit 10. And we also support Symfony 6 without dropping support of previous Symfony versions. As always, we did our best to keep backward compatibility so if you can update your dependencies, all tests should be working for you.

So let’s take a look at some new features:

New Directory Structure

Codeception 5 will follow the PSR-12 coding standard. So all tests and classes will have their own namespace Tests. The directory structure was updated accordingly:

tests/
  _output
  Acceptance
  Functional
  Support/
    _generated/
    Data/
    Helper/
  Unit/

All suite names have their own namespace, as well as actor and helper classes:

<?php

namespace Tests\Acceptance;

use \Tests\Support\AcceptanceTester;

class LoginCest
{
    public function tryToTest(AcceptanceTester $I)
    {
        $I->amOnPage('/');
    }
}

This new directory structure will be generated by running codecept bootstrap.

Upgrading from Codeception 4

Codeception 5 is compatible with the Codeception 4 directory structure. So if you don’t want to change your file locations now, just keep your existing codeception.yml, and everything’s fine.

However, if you want to upgrade to the new Codeception 5 directory structure (recommended), here’s the new default codeception.yml:

namespace: Tests
support_namespace: Support
paths:
    tests: tests
    output: tests/_output
    data: tests/Support/Data
    support: tests/Support
    envs: tests/_envs
actor_suffix: Tester
extensions:
    enabled:
        - Codeception\Extension\RunFailed

Next steps:

1. Rename your suite configuration files: acceptance.suite.yml => Acceptance.suite.yml functional.suite.yml => Functional.suite.yml unit.suite.yml => Unit.suite.yml
2. Inside those configuration files, update to the new namespace:

    modules:
        enabled:
            - Tests\Support\Helper\Unit
   

3. In your composer.json, update to the new namespace:

    "autoload-dev": {
        "psr-4": {
            "Tests\\": "tests/"
        }
    },
   

4. In your tests/Support/Acceptance|Functional|UnitTester.php files, update to the new namespace.
5. Run vendor/bin/codecept build to create the files in tests/Support/_generated.
6. Modify the namespaces in all your test/cest files.
7. Run the tests with capitalized suite names: vendor/bin/codecept run Unit

Attributes

Annotations were an essential part of the Codeception testing framework. Even though they were not native language constructs, they proved to be quite good to separate a test from its metadata. We believe that test should not include code that doesn’t belong to the test scenario.

So we were glad that native attributes have landed in the PHP world. In this release we encourage you to start using them:

#[Group('important')]
#[Group('api')]
#[Examples('GET', '/users')]
#[Examples('GET', '/posts')]
#[Env('staging-alpha')]
#[Env('staging-beta')]
#[Env('production')]
#[Prepare('startServices')]
public function testApiRequests(ApiTester $I, Example $e)
{
  $I->send($e[0], $e[1]);
  $I->seeResponseCodeIsSuccessful();
  $I->seeResponseIsJson();
}

As you see, attributes decouple all preparation steps, keeping the test scenario minimal. We also keep supporting annotations, so an urgent upgrade is not needed. Attributes can’t do anything that traditional annotations can’t, they are just a modern alternative.

List of available attributes (all under Codeception\Attribute namespace):

• Before - specifies a method that should be executed before each test
• After - specifies a method that should be executed after each test
• Group - sets the group for the test
• Skip - skips the current test
• Incomplete - marks test as incomplete
• Depends - sets the test that must be executed before the current one
• Prepare - sets a method to execute to initialize the environment (launch server, browser, etc)
• DataProvider - specifies a method that provides data for data-driven tests
• Examples - sets data for data-driven tests inside the annotation
• Env - sets environment value for the current test
• Given, When, Then - marks a method as BDD step

Debugging

Do you remember, Hoa\Console? Unfortunately, this library was deprecated and we were looking for a modern alternative that could power codecept console and $I->pause();. We switched to PsySH, a PHP Read-Eval-Print Loop.

An interactive console is used to pause a test in the given state. While in pause you can try different Codeception commands, and check variable values. Instead of fixing tests blindly, you can start an interactive session. This is quite a similar effect you can get with a real debugger like XDebug but focused on Codeception commands. This is especially helpful for acceptance tests as the test scenario can be planned while executing a test. So the basic scenario can be written as:

$I->amOnPage('/');
$I->pause();

After opening a page you will be able to try commands in a browser. If a command succeeds you can use it in your tests.

Also new functions were added:

• codecept_pause() - starts interactive pause anywhere in debug mode
• codecept_debug() - prints a variable to console using Symfony VarDumper

Upgrading from Codeception 4

Just remove hoa/console from your composer.json.

Sharding

The Parallel Execution guide has been rewritten and focused on a new feature: sharding. It is the simplest way to run slow tests (think of acceptance tests first) in parallel on multiple agents.

In this case, you specify the batch of tests that should be executed independently and each job picks up its own not intersecting group of tests to run them.

# first job
./vendor/bin/codecept run --shard 1/3

# second job
./vendor/bin/codecept run --shard 2/3

# third job
./vendor/bin/codecept run --shard 3/3

This feature reduces the need for complex configuration and usage of robo task runner to split tests.

It is recommended to use sharding to parallelize tests between multiple jobs as the simplest approach. Unfortunately, PHP doesn’t have native multi-threading for test parallelization, and even if it had, it doesn’t solve the problem of running slow browser tests that are interacting with the entire application. So only horizontal scaling by jobs can be suggested as a long-running approach. The more build agents you add to your Continuous Integration server, the faster tests will run. That’s it!

Grep and Filter

New options --grep and --filter were introduced to select tests by part of their name. Actually, it is the same option and an alias. --grep is a common way to select tests to execute in NodeJS test runners, so we ported it to Codeception. But as usual, specific tests can also be executed by group or specifying a test signature.

./vendor/bin/codecept run --grep "user"

Other Changes

Please go through the list of changes introduced to see if they don’t affect your codebase:

• Requires PHP 8.0 or higher
• Compatible with PHPUnit 9 and ready for PHPUnit 10
• Compatible with Symfony 4.4 - 6.0
• Stricter check for phpdotenv v5 (older versions are not supported)
• Throw exception if actor setting is missing in suite configuration
• Removed generate:cept command (Cept format is deprecated)
• Removed settings disallow_test_output and log_incomplete_skipped.
• Removed setting paths.log (replaced by paths.output in Codeception 2.3)
• Removed suite setting class_name (replaced by actor in Codeception 2.3)
• Removed global setting actor (replaced by actor_prefix in Codeception 2.3)
• Removed Configuration::logDir method (replaced by Configuration::outputDir in 2.0)
• Moved XmlBuilder class to SOAP module
• Decoupled test execution and reporting from PHPUnit
• Custom reporters implementing TestListener are no longer supported and must be converted to Extensions
• Added optional value to fail-fast option (#6275) by #Verest
• Removed JSON and TAP loggers
• Removed code coverage blacklist functionality
• Removed deprecated class aliases
  • Codeception\TestCase\Test
  • Codeception\Platform\Group
  • Codeception\Platform\Group
  • Codeception\TestCase
• Introduced strict types in the code base.

Complete Changelog

We are really happy that we are finally here with Codeception 5. This release was crafted during the war that happens in Ukraine. It is mentally and morally hard to work on tech products knowing that this peaceful virtual life can end at any moment by a random missile. Codeception was created in 2011 by Michael Bodnarchuk in Kyiv, and today in 2022 he also stays there writing this post. If you want to support Codeception, all the Ukrainian PHP community, and all our brave nation who stands for democracy against this barbaric Russian invasion, consider donating to Ukrainian charities. Not a single time. Every month until the war ends. Every time you travel or enjoy tasty food in a restaurant think of people who are forced to defend their land, or who fled their homes. Glory to Ukraine!

This release wouldn’t be possible without the hard work of Gintautas Miselis who keeps constant work on modernizing internals and keeping Codeception up to date. Also we are really thankful to Gustavo Nieves who did a lot of work transitioning Codeception to Symfony 6 and more! Thanks to our maintainers! If you want to support our work we have OpenCollective!