UE4 Code: Fixing Hot Reload of AutomationTests

The problem

Au­toma­tion­Tests are reg­is­tered from the con­struc­tor of your FAu­toma­tion­Base sub­class (that you cre­ate via IMPLEMENT_SIMPLE_AUTOMATION_TEST). The test will on­ly be reg­is­tered, if it hasn’t been be­fore. There­fore, when a test with the same name is reg­is­tered, it would skip the reg­is­tra­tion, be­cause FAu­toma­tion­Test­Frame­work con­tains a map of test name to test in­stance, count­ing same names (keys) as the same test, even if it’s a dif­fer­ent class.

They would be un­reg­is­tered in the de­struc­tor, which would be called when the DLL is un­load­ed, but for my case, the DLL is nev­er un­load­ed, be­cause I have a sep­a­rate mod­ule for my unittests. It doesn’t con­tain any oth­er class­es, i.e. UCLASS-es, in which case the mod­ule is treat­ed as not hotreload­able and is “aban­doned” dur­ing hot reload in­stead, leav­ing the DLL load­ed.

Af­ter that the new­ly com­piled DLL is load­ed, which does try to reg­is­ter your changed test, but there is still the one with the same name hang­ing around that you reg­is­tered pre­vi­ous­ly, so it won’t ac­tu­al­ly reg­is­ter that test!

Solution Attempts

Sounds sim­ple, just de­fine a UCLASS and prob­lem solved? For some rea­son if you add a UCLASS, your mod­ule is then reload­ed as a “UP­ack­age” which goes a dif­fer­ent path and doesn’t even reach the afore­men­tioned code, but is in­stead just aban­doned di­rect­ly.

In­stead, we could de­fine a sub­class of FAu­toma­tion­Test­Base, which in its con­struc­tor binds a “Pre­Un­load­Call­back” on the mod­ule, which would un­reg­is­ter the test. That doesn’t work though, be­cause the con­struc­tor is called be­fore the mod­ule is ful­ly reg­is­tered in the Mod­ule­M­an­ag­er. This leaves us no way to get a ref­er­ence to the mod­ule.

A de­cent fix would prob­a­bly be to have Un­re­al En­gine pre­fix the test names with the mod­ule name and un­reg­is­ter­ing them on shut­down of the mod­ule. If some­body wants to pay me to fix UE-25350 prop­er­ly, DM @squareys on twit­ter or some­thing, I’d be hap­py to do it :P , oth­er­wise try the fol­low­ing work­around:

TL;DR – The Workaround

If you have this prob­lem, you like­ly al­so don’t have any UCLASS-es in your test mod­ule. Work­around would there­fore be to just cre­ate an emp­ty UCLASS in the mod­ule that con­tains the tests.

I add this to my the main mod­ule source file <Your Game>Tests.cpp to make the test "F<Your Test Type Name>Test" hot reload­able:

class FTestModuleImpl : public FDefaultGameModuleImpl {

    void ShutdownModule() override {
        /* Workaround for UE-25350 */
        FAutomationTestFramework::Get().UnregisterAutomationTest("F<Your Test Type Name>Test");
        // ... for every test you defined.
    }
};

IMPLEMENT_GAME_MODULE(FTestModuleImpl, "<Your Game>Tests");

That’s it! Hap­py Au­toma­tion­Test­ing/Ga­me­Test­ing/UnitTest­ing ev­ery­body! :)

See you in the next blog.