Lucky Robots Blog Open Roles

LuckyEngine

A C++20 game-and-robotics engine: deferred Vulkan renderer, entt ECS, Jolt + MuJoCo + Box2D physics, C# scripting on .NET 9, ONNX policy inference, parquet/LeRobot recording, an MCP bridge for tooling, and a built-in editor with an LLM agent.

18
Subsystems
~38
Vendor libs
3
Physics backends
C++20
Core language
.NET 9
Scripting
gRPC
External API

Architecture at a glance

The engine boots through EntryPoint.h into a singleton Application that owns the window, the render thread, the event queue, and the layer stack. The editor is itself a layer. Layers create Scenes, which own everything that simulates: ECS registry, physics worlds, scripts, observers. Rendering reads scenes; physics writes them; scripts can do either.

Entry Application Core/Application.h Layers EditorLayer LuckyEditor/ RuntimeLayer LuckyRuntime/ World Scene entt::registry + TimeManager PrefabManager · Observer Physics Jolt JoltScene MuJoCo SceneInstance Box2D world Physics2D/ Scripting ScriptEngine + Coral .NET 9 InternalCalls Rendering SceneRenderer deferred · GTAO · SSR · bloom Renderer (facade) nvrhi → Vulkan Renderer2D · DebugRenderer batched · MSDF text Assets AssetManager Editor / Runtime backends Robots / ML MotionGraph Robots/Nodes/ PolicyApplier (ONNX) per-slot PD · Learn/ Data Observer + Recorders Parquet · LeRobot · LuckyHub
Boot path and core ownership. Solid lines = "owns / drives". Bold-bordered boxes are the canonical entry points.

Subsystems

Core
2.1

Application

Singleton root: window, layer stack, render-thread context, main loop. Threading policy chooses real render thread (runtime) vs same-thread queue drain (editor).

Core/Layer stack
2.2

Window / Platform

Abstract Window with GLFWWindow (desktop, Vulkan) and NullWindow (headless). Watch for GLFW-only methods missing on the base.

GLFWHeadless
World
2.4

Scene / ECS

entt-backed registry plus TimeManager, ScriptStorage, PrefabManager. Entities identified by stable UUIDs; transform hierarchy with shear-safe local↔world.

Scene/enttUUID
2.9

Asset System

Editor (file-based) and Runtime (packed) backends behind one AssetManager facade. Sync + async loads, dependency notifications, ~22 asset types.

Editor / RuntimeAssetPack
Physics
2.5

Physics (Jolt)

3D rigid bodies behind a PhysicsAPI abstraction. Phased stepping: Acquisition → Control → Physics → Validation → Export. Layered collision filtering.

Jolt5700 bodies
2.6

Physics (MuJoCo)

Robotics-grade simulator. Z-up to engine Y-up conversion via fixed axis matrices. CoACD convex decomposition. Substeps run per-phase callbacks.

Coord swapMJCF / URDF
2.7

Physics (Box2D)

2D mechanics directly via b2World — no abstraction layer. TimeManager callbacks for stepping and transform sync.

Box2D2D
Rendering
2.3

Renderer

NVRHI-backed Vulkan. SceneRenderer drives the deferred pipeline: pre-depth, cascaded shadows, GTAO, SSR, bloom, DOF, SMAA. HLSL shaders, cached pipelines.

Renderer/HLSLDeferred
2.14

Animation

Skeletal animation driven by AnimationGraph assets and bone entity hierarchies. TimeManager-driven update.

Skeletal
Scripting & Editor
2.8

Scripting

C# on .NET 9 via Coral interop. InternalCalls bridge into the engine; per-entity field storage; hot reload invalidates managed refs — never cache them.

.NET 9Coral
2.10

Editor

EditorLayer orchestrator plus a panel registry. EditorLayer is protected — new features go in their own panel. Strict ImGuiEx styling.

ImGuiEditorLayer protected
2.11

Audio

miniaudio engine with spatial audio, sound graphs, DSP, and runtime sound bank packaging. Singleton MiniAudioEngine; ECS components for source / listener.

miniaudioSpatial
2.12

Input

All-static frame-accurate keyboard, mouse, and controller polling. Update() per frame transitions key states.

Static
Robots / ML
2.13

Robot / MotionGraph

Graph-authored behaviour + per-entity ONNX policies. Policies override the graph on claimed joints; per-substep merge rules govern routing to actuators or qfrc.

ONNXIKPolicyApplier
2.15

Data / Observation

Records joint states, sensor data, camera frames → Parquet, LeRobot-compatible. Recording gated on LuckyHub approval; never stalls on hub latency.

Recording-criticalParquet
Tooling
2.16

Project / Build

Premake5, Conan, C++20. Make on Linux/Mac, VS2022 on Windows. ~38 vendor libs. CI in .github/workflows/.

Premake5Conan
2.17

MCP Server (Hazel-Bridge)

JSON-RPC 2.0 server. TCP + streamable HTTP. Localhost-only. Tool handlers marshal to the main thread via MainThreadDispatcher.

JSON-RPCcpp-httplib
2.18

LuckyEditor-Agent

In-editor LLM agent. Static lib with plugin-shaped public surface. Uses Hazel-Bridge to expose harness/* and tool/* tools to clients.

LLMPimpl

Cross-cutting topics

Source of truth

This site mirrors .claude/docs/Architecture-LuckyEngine.md and its companion docs (Conventions.md, Threading.md, RecordingIntegrity.md, Architecture-LuckyRobots.md). For API-level signatures, read the headers in Hazel/src/Hazel/… — they never go stale.

Build with LuckyEngine.

Engine repo on GitHub →