Unreal Engine – Neural Network Symbol Recognizer UE6387

The plugin is no longer supported.

Please, choose Symbol Recognizer VR instead.

Demo: LINK Tutorial: LINK

Plugin enables you to draw collections of Symbols/Patterns which can be recognized during gameplay by a Neural Network system.

It can be used in different gameplay scenarios:

  • Cast a spell according to a mouse gesture
  • Paint a symbol on doors and unlock them if correct
  • Make a minigame that requires speed and precise drawing to pass

Basic functionality:

  • Draw different patterns (Symbols) in a plugin’s window. It can be letters, digits or any abstract shape. Launch machine learning and save the result. Test drawing accuracy inside the plugin window. Then do a simple setup in Blueprints or code to be able to draw and test the drawing accuracy in a game.

Workflow for in game drawing and testing symbols accuracy:

  • Initialize drawing by calling a method BeginDrawing() on input PRESSED
  • Pass brush (cursor) location into Draw() method when moving your brush or in Tick
  • Call EndDrawing() to break current drawing line on input RELEASED
  • When a drawing is finished Test its accuracy by calling TestDrawing or GetMostAccurateSymbol
  • React somehow to the test result (either kill a player if failed of throw a blizzard of fireballs when succeeded)
  • Call ClearCanvas when current drawing is not needed and want to start new drawing.

Plugin contains 3 examples of In Game usage (SRTestLevel)

技术细节

Features:

  • Built-In tool for drawing and saving symbols.
  • Customizable parameters to modify Neural Network sensitivity.
  • Simple setup for in game use.
  • Example level with 3 mechanics (casting spells, drawing on mesh, minigame).
  • Step by step tutorial inside the plugin.
  • Drawing symbols is independent from size and position which makes it much easier to draw a correct symbol.
  • Two debug widgets to test drawing accuracy in game.
  • Possibility to create multiple symbol collections that can be set for recognition in gameplay.

Code Modules:

  • SymbolRecognizerPlugin (Runtime)
  • SymbolRecognizerEditor (Editor)

Number of Blueprints: 8

Number of C++ Classes: 20

Network Replicated: No

Supported Development Platforms: Win64/Win32

Supported Target Build Platforms: Win64/Win32

Documentation: ‘ShowTutorial’ inside plugin or partly in video tutorials

Example Project: Inside plugin SRTestLevel (SymbolRecognizer/Resources/TestContent)

Important/Additional Notes: Do not change default root plugin folder name. It must be ‘SymbolRecognizerPlugin’.