EventEngine Test Suite: Timers

[drfloob]

A reusable test suite for EventEngine implementations.

To exercise a custom EventEngine, simply link against :event_engine_test_suite
and provide a testing main function that sets a custom EventEngine factory:

#include "path/to/my_custom_event_engine.h"
#include "src/core/event_engine/test_suite/event_engine_test.h"

int main(int argc, char** argv) {
  ::testing::InitGoogleTest(&argc, argv);
  SetEventEngineFactory(
      []() { return absl::make_unique<MyCustomEventEngine>(); });
  auto result = RUN_ALL_TESTS();
  return result;
}

See the :simple_event_engine_test blaze target for an example.

@Vignesh2208 @dennycd

[ctiller]

Watch out for backwards references here... there are build systems that will likely choke on this.

We may be better off with the following layout:

eventengine-tests:

  function<EventEngine*()> NewEventEngine();
  void SetEventEngineFactory(function<EventEngine*()>);

specific-event-engine-test:

  void main() {
    SetEventEngineFactory(MyEventEngineFactory);
    gtest::RunTheStuff();
  }

[drfloob]

Thanks for the heads up about linker backreference issues. I don't think it's an issue in this case, since implementation-specific test suite targets will always have a one-way dependency on the :event_engine_test_suite target, and nothing else should depend on that :event_engine_test_suite target, leaving that method undefined. But it is a "clever" thing, which is debatably a bad idea. I've made the changes you suggested. It's a bit more code now (I didn't want an unmanaged global ptr that never gets deleted, for example), but it does the job fine.

[ctiller]

s/timer_count/thread_count through here...

We probably want 100 threads creating 100 timers, for 10000 timers scheduled total. (i.e. 100 threads each run this loop).

This will help catch any threading issues with scheduling new timers too.

[drfloob]

I've added the 10k RunAt calls (split across 100 threads). As a result, I decided to delete the SimpleEventEngine example implementation rather than improve it, because TSAN was not able to deal with 10k threads in that naive implementation. The LibuvEventEngine PR will exercise this suite, and at the very least, the SimpleEventEngine implementation was helpful to verify the suite up to this point.

[ctiller]

This could be a source of flakiness: if we get pre-empted on line 88 for 1 second or more then this test will fail.

[drfloob]

Noted on line 76. I considered removing this test, since it does require some timeout hackery which is inherently brittle, and could take a lot of time if it's too conservatively configured. We're already testing the timer's scheduling accuracy in StressTestTimersNotCalledBeforeScheduled, and ordering is not guaranteed if both timers come due at the same time (presuming some delay in clock evaluation). I'm thinking again maybe we remove this. WDYT?

[ctiller]

This would make a very good stress test too: have 100 threads do this for 100 different vectors (and 100 different mutexes), assert all vectors are ordered at the end.

(this could be later work)

[ctiller]

This would also make an excellent stress test

[ctiller]

100 threads schedule infinite future callbacks, and then in phase 2 100 different threads cancel them.

[ctiller]

(again, future work suggestion)

[ctiller]

some future work suggestions... I expect we should do something about TimersRespectScheduleOrdering potential flakiness, but I'm not sure it should block submission either.

[drfloob]

(import notes) All tests had previously passed except for the ruby tests, which are failing due to an RVM bug. #27538