LazyLoadingViolationException are not reported in console when running tests (unless $this->withoutExceptionHandling() is called)

[khalyomede]

Laravel Version

10.47.1

PHP Version

8.3.3

Database Driver & Version

mysql Ver 8.0.28 for Linux on x86_64 (MySQL Community Server - GPL)

Description

I noticed a lazy violation exception in production, and I had created a test to reproduce the issue but the test passed. After I added the following code to my TestCase ($this->withoutExceptionHandling();), the test started to fail, and a bunch of other tests failed. I think "LazyLoadingViolationException" should be reported during tests (this is the perfect moment to catch these errors).

However they get correctly reported, but silently in the logs.

That may be a confusion/misunderstanding over what is the purpose of handling exceptions during feature tests (I did not find any usable information in the documentation about the why).

I was also surprised to have detected other "hidden" exceptions since I disabled exception handling (but that is outside the scope of this issue).

Steps To Reproduce

With a model with a one to many relationship (example shopping list -> items), controller loops through the items, and the tests should ensure 2 or more items are attached to the tested model (shopping list).

// app/app/Providers/AppServiceProvider.php

declare(strict_types=1);

namespace App\Providers;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\App;
use Illuminate\Support\ServiceProvider;
use Sentry\Laravel\Integration;

final class AppServiceProvider extends ServiceProvider
{
  public function boot(): void
  {
    Model::preventLazyLoading();

    if (App::isProduction()) {
      Model::handleLazyLoadingViolationUsing(Integration::lazyLoadingViolationReporter());
    }
  }
}

// app/tests/TestCase.php

declare(strict_types=1);

namespace Tests;

use Illuminate\Foundation\Testing\LazilyRefreshDatabase;
use Illuminate\Foundation\Testing\TestCase as BaseTestCase;
use Illuminate\Support\Facades\Http;

abstract class TestCase extends BaseTestCase
{
  use CreatesApplication;
  use LazilyRefreshDatabase;

  protected bool $seed = true;

  protected function setUp(): void
  {
    parent::setUp();

    $this->withoutVite();
    $this->withoutMix();
    $this->withoutExceptionHandling(); // When this line is commented, the test below does not fail (not wanted)

    Http::preventStrayRequests();
  }
}

// app/tests/Feature/Http/Controllers/ShoppingListCopyControllerTest.php

declare(strict_types=1);

namespace Tests\Feature\Http\Controllers;

use App\Models\Aisle;
use App\Models\Item;
use App\Models\ShoppingList;
use App\Models\Subscription;
use App\Models\User;
use Illuminate\Foundation\Testing\WithFaker;
use Tests\TestCase;

final class ShoppingListCopyControllerTest extends TestCase
{
  use WithFaker;

  public function testMemberDoesntGetItemsOwnerAisleWhenDuplicatingShoppingList(): void
  {
    $user = User::factory()
      ->has(Subscription::factory()->standard())
      ->create();

    $shoppingList = ShoppingList::factory()->for($user, "author")
      ->has(Item::factory()->for(Aisle::factory()->for($user)))
      ->has(Item::factory()->for(Aisle::factory()->for($user)))
      ->has(User::factory()->has(Subscription::factory()->standard()), "members")
      ->create();

    $member = $shoppingList->members->first();

    assert($member instanceof User);

    $this->actingAs($member)
      ->post(route("shopping-list.copy.store", $shoppingList), [
        "name" => $this->faker->name(),
      ])
      ->assertValid();
  }
}

// app/app/Http/Controllers/ShoppingListCopyController.php

declare(strict_types=1);

namespace App\Http\Controllers;

use App\Http\Requests\StoreShoppingListCopyRequest;
use App\Models\Item;
use App\Models\ShoppingList;
use Illuminate\Http\RedirectResponse;
use Illuminate\Support\Facades\DB;

final class ShoppingListCopyController extends Controller
{
  public function store(StoreShoppingListCopyRequest $request, ShoppingList $shoppingList): RedirectResponse
  {
    $user = $request->user();
    $duplicatedShoppingList = $shoppingList->replicate();

    \assert($duplicatedShoppingList instanceof ShoppingList);

    DB::beginTransaction();

    $duplicatedShoppingList->name = $request->string("name")->toString();
    $duplicatedShoppingList->author()->associate($user);
    $duplicatedShoppingList->save();

    $duplicatedItems = $shoppingList
      ->items
      ->map(function (Item $item) use ($user): Item {
        $duplicatedItem = $item->replicate();

        if (!$item->aisle?->user?->is($user)) { // lazy loading violation here
          $duplicatedItem->aisle()->dissociate();
        }

        return $duplicatedItem;
      });

    $duplicatedShoppingList
      ->items()
      ->saveMany($duplicatedItems);

    DB::commit();

    return redirect()->route("shopping-list.show", $duplicatedShoppingList);
  }
}

// app/app/Exceptions/Handler.php

declare(strict_types=1);

namespace App\Exceptions;

use Illuminate\Foundation\Exceptions\Handler as ExceptionHandler;
use Illuminate\Support\Facades\App;
use Sentry\Laravel\Integration;
use Throwable;

final class Handler extends ExceptionHandler
{
  /**
   * A list of the exception types that are not reported.>
   */
  protected $dontReport = [];

  protected $dontFlash = [
    "current_password",
    "password",
    "password_confirmation",
  ];

  public function register(): void
  {
    $this->reportable(function (Throwable $e): void {
      /**
       * @see https://docs.sentry.io/platforms/php/guides/laravel/#install
       */
      if (!App::environment("local")) {
        Integration::captureUnhandledException($e);
      }
    });
  }
}

Edit: I had to explicitly list which exceptions the framework should handle so that all new exceptions are not handled and I can decide which one should be. In my opinion this should be done behind the scene so that everything that can be asserted (validation, authorization, ...) is handled by default and everything that is not assertable (lazy loading violations etc...) keep popping up in the CLI.

This code below produces the results I expect, so that new missing attributes or violation exceptions gets detected.

// app/tests/TestCase.php

declare(strict_types=1);

namespace Tests;

use Illuminate\Auth\Access\AuthorizationException;
use Illuminate\Database\Eloquent\ModelNotFoundException;
use Illuminate\Foundation\Testing\LazilyRefreshDatabase;
use Illuminate\Foundation\Testing\TestCase as BaseTestCase;
use Illuminate\Support\Facades\Http;
use Illuminate\Validation\ValidationException;
use Symfony\Component\HttpKernel\Exception\HttpException;

abstract class TestCase extends BaseTestCase
{
  use CreatesApplication;
  use LazilyRefreshDatabase;

  protected bool $seed = true;

  protected function setUp(): void
  {
    parent::setUp();

    $this->withoutVite();
    $this->withoutMix();
    $this->handleExceptions([
      AuthorizationException::class,
      HttpException::class,
      ValidationException::class,
      ModelNotFoundException::class,
    ]);

    Http::preventStrayRequests();
  }
}

[github-actions]

Thank you for reporting this issue!

As Laravel is an open source project, we rely on the community to help us diagnose and fix issues as it is not possible to research and fix every issue reported to us via GitHub.

If possible, please make a pull request fixing the issue you have described, along with corresponding tests. All pull requests are promptly reviewed by the Laravel team.

Thank you!