Reliable API for freeing resources used by the test

[morozov]

Take this test for example:

<?php

declare(strict_types=1);

namespace Doctrine\DBAL\Tests;

use Exception;
use PHPUnit\Framework\TestCase;

abstract class IntegrationTest extends TestCase
{
    protected $connection;

    /**
     * @before
     */
    protected function connect(): void
    {
        // connect to the database
        $this->connection = ...;
    }

    /**
     * @after
     */
    protected function disconnect(): void
    {
        // disconnect from the database
        $this->connection = null;
    }
}

class TearDownFailureTest extends IntegrationTest
{
    protected function tearDown(): void
    {
        throw new Exception();
    }

    public function testSkipped(): void
    {
        self::markTestSkipped();
    }
}

The code above does the following:

1. The base class defines the @before and @after methods that will prepare and clean up the test database and the connection.
2. The specific test cases may define their own setup and teardown procedures.

There is a problem in the TearDownFailureTest that it may get skipped but it doesn't check (and probably can't) if it was skipped, it attempts to perform the teardown logic and throws the exception that gets suppressed by the test runner. Because of that, the disconnect() method in the parent class doesn't get called and the test instance remains connected to the database leaking resources.

It wouldn't be a problem if the test object got destroyed after the run but it doesn't (there are a few reported issues about that, e.g. #3039).

If the teardown methods are meant for the cleanup, should they be called independently on the results of each other? Currently, an exception stops the chain:

phpunit/src/Framework/TestCase.php

Lines 919 to 924 in 8053bc6

	foreach ($hookMethods['after'] as $method) {
	if ($this->methodDoesNotExistOrIsDeclaredInTestCase($method)) {
	continue;
	}
	$this->{$method}();

Currently, the desired logic could be implemented in a TestListener class which has access to the Test object but it's deprecated:

phpunit/src/Framework/TestListener.php

Line 37 in 5a95748

public function endTest(Test $test, float $time): void;

Neither the deprecated TestHook API, nor the event-based API seem to provide access to the Test instance for reliable cleanup.

[mvorisek]

duplicate of #4705