Unreal Engine – Neural Network Symbol Recognizer UE6387

The plugin is no longer supported.

Please, choose Symbol Recognizer VR instead.

Demo: LINK Tutorial: LINK

Plugin enables you to draw collections of Symbols/Patterns which can be recognized during gameplay by a Neural Network system.

It can be used in different gameplay scenarios:

• Cast a spell according to a mouse gesture
• Paint a symbol on doors and unlock them if correct
• Make a minigame that requires speed and precise drawing to pass

Basic functionality:

• Draw different patterns (Symbols) in a plugin’s window. It can be letters, digits or any abstract shape. Launch machine learning and save the result. Test drawing accuracy inside the plugin window. Then do a simple setup in Blueprints or code to be able to draw and test the drawing accuracy in a game.

Workflow for in game drawing and testing symbols accuracy:

• Initialize drawing by calling a method BeginDrawing() on input PRESSED
• Pass brush (cursor) location into Draw() method when moving your brush or in Tick
• Call EndDrawing() to break current drawing line on input RELEASED
• When a drawing is finished Test its accuracy by calling TestDrawing or GetMostAccurateSymbol
• React somehow to the test result (either kill a player if failed of throw a blizzard of fireballs when succeeded)
• Call ClearCanvas when current drawing is not needed and want to start new drawing.

Plugin contains 3 examples of In Game usage (SRTestLevel)