[{"content":"Embedding a scripting language in an Unreal project is rarely about C++ being slow to write. It\u0026rsquo;s usually about three concrete things: mobile hot-updates (App Store rules permit patching interpreted script, but not shipping a new binary), an iteration loop that doesn\u0026rsquo;t require a compile for designers and gameplay programmers, and the fact that Blueprint maintenance costs blow up at scale — node graphs can\u0026rsquo;t be diffed, can\u0026rsquo;t be reviewed, and merge conflicts are effectively unresolvable.\nLua is the most battle-tested answer to that problem, and UnLua is Tencent\u0026rsquo;s open-source take on it. Its pitch is \u0026ldquo;zero glue code\u0026rdquo;: no bindings to write, full access to every UCLASS / UPROPERTY / UFUNCTION from Lua, and the ability to override Blueprint events outright.\nThis article isn\u0026rsquo;t a usage guide — the official docs cover that. It\u0026rsquo;s a teardown: what that \u0026ldquo;zero glue\u0026rdquo; is actually bought with, where the costs land, and which parts bite.\nThe analysis is based on master at 3f112e8, functionally equivalent to v2.3.6 (released 2023-11-07, the last tag). Every file:line reference points at that revision. The maintenance status as of 2026 gets its own section at the end — it\u0026rsquo;s more nuanced than \u0026ldquo;abandoned.\u0026rdquo;\n1. Three roads, and it took the most expensive one There are only three engineering routes for exposing C++ objects to a script VM:\nApproach Examples Cost Benefit Hand-written bindings Classic Lua projects Glue for every class; silently rots when interfaces change Lowest call overhead, fully controlled behaviour Code generation sluaunreal, puerts Must run a generator, artifacts land in VCS, longer builds Static type info, near hand-written performance Runtime reflection UnLua Every call pays for the reflection layer; type errors only surface at runtime Zero maintenance — engine upgrades and Blueprint field changes are picked up automatically UnLua took the third road. Unreal\u0026rsquo;s reflection system (UClass / UFunction / FProperty) already is a complete runtime metadata layer built for the Blueprint VM and serialization, so UnLua effectively bolts a second consumer onto it. That single decision explains nearly everything downstream: adding a UPROPERTY requires no script-layer work at all, but every boundary crossing pays the reflection tax — and a lot of the \u0026ldquo;gotchas\u0026rdquo; are just Unreal\u0026rsquo;s reflection semantics leaking straight through.\nThe whole plugin is 148 h/cpp files excluding third-party Lua — 118 of them in the runtime UnLua module, plus an editor module and a UHT plugin:\nPlugins/UnLua/Source/ ├── UnLua/ # runtime (118 files), LoadingPhase = PreDefault │ ├── Private/ReflectionUtils/ # FClassDesc / FFunctionDesc / FPropertyDesc │ ├── Private/Registries/ # 7 registries (Class/Object/Function/Property/Enum/Container/Delegate) │ ├── Private/BaseLib/ # hand-exported: TArray/TMap/TSet/Delegate/Object/Class... │ ├── Private/MathLib/ # hand-exported: FVector/FRotator/FTransform... │ └── Private/LuaFunction.cpp # the heart of the override mechanism ├── UnLuaEditor/ # bind button, template generation, IntelliSense generation ├── UnLuaDefaultParamCollector/ # UHT plugin: harvests C++ default parameter values └── ThirdParty/Lua/ # lua-5.4.3 (default) and lua-5.4.4 2. How an Actor becomes a Lua table The entry point is an empty interface, UUnLuaInterface, with a single method: GetModuleName. Implement it in a Blueprint, return \u0026quot;Player.BP_PlayerCharacter_C\u0026quot;, and the binding exists.\nThe real work happens in FLuaEnv::TryBind (LuaEnv.cpp:336), which hooks GUObjectArray\u0026rsquo;s create/delete listener and filters every new object:\nstatic UClass* InterfaceClass = UUnLuaInterface::StaticClass(); const bool bImplUnluaInterface = Class-\u0026gt;ImplementsInterface(InterfaceClass); ... if (Class-\u0026gt;GetName().Contains(TEXT(\u0026#34;SKEL_\u0026#34;))) // skip skeleton classes return false; const auto ModuleName = ModuleLocator-\u0026gt;Locate(Object); Three details that are easy to miss:\nThe module name is read off the CDO (ULuaModuleLocator::Locate, LuaModuleLocator.cpp:18), so it is a class-level property — every instance of a class binds to the same Lua module. Per-instance variation requires dynamic binding (passing a module name at SpawnActor / NewObject time). There\u0026rsquo;s also ULuaModuleLocator_ByPackage, which derives the module path from the package path so you don\u0026rsquo;t implement the interface at all. Worth switching to on a large project — it saves clicking the bind button on hundreds of Blueprints. Objects created on the async loading thread can\u0026rsquo;t be bound immediately (Lua isn\u0026rsquo;t thread-safe). They queue into Candidates and get processed when OnAsyncLoadingFlushUpdate returns to the game thread. Step two is UUnLuaManager::BindClass, which does something rarely mentioned — the module table gets shallow-copied:\nif (!Class-\u0026gt;IsChildOf\u0026lt;UBlueprintFunctionLibrary\u0026gt;()) { // one Lua module may be bound to a UClass and its subclasses, so copy one // out to serve as the metatable for their instances lua_newtable(L); lua_pushnil(L); while (lua_next(L, -3) != 0) { ... } } UnLuaManager.cpp:283\nThe table require returned is not the one instances actually use. Mutating the original module table at runtime is invisible to already-bound instances — which is also the root reason hot reload needs extra machinery.\nStep three, FObjectRegistry::Bind (ObjectRegistry.cpp:113), wires up three layers:\nINSTANCE (one empty table per object) ├─ .Object = RAW_UOBJECT (userdata holding the UObject*) └─ metatable → MODULE (copy of the module table, __index = the Lua Index function) ├─ .Super = parent module table (set by UnLua.Class(super)) ├─ .Overridden = CLASS_METATABLE └─ metatable → CLASS_METATABLE (one per UStruct, __index = Class_Index C function) So self.Foo naturally resolves in the order: the instance\u0026rsquo;s own fields → the Lua module chain → Unreal reflection. Each layer owns one concern; the semantics are clean.\nFinally Bind calls Lua\u0026rsquo;s Initialize. At that point the object still carries RF_NeedInitialization, and FFunctionDesc::CheckObject (FunctionDesc.cpp:550) will reject any UFunction call:\nif (Object-\u0026gt;HasAnyFlags(RF_NeedInitialization)) { Error = FString::Printf(TEXT(\u0026#34;attempt to call UFunction \u0026#39;%s\u0026#39; in lua Initialize function on object \u0026#39;%s\u0026#39;.\u0026#34;), ...); Initialize is for pure Lua state only. Don\u0026rsquo;t touch the engine there.\n3. The core: how overriding actually works This is the most interesting part of the plugin — and the part where the official documentation is out of date.\n3.1 A UFunction has two call paths UObject::ProcessEvent eventually reaches UFunction::Invoke. If the function has FUNC_Native, it calls the thunk function pointer (FNativeFuncPtr) directly; otherwise it goes through ProcessInternal, where the Blueprint VM interprets the bytecode in Script.\nDocs/CN/How_To_Implement_Overriding.md describes two schemes: replacing the thunk, and registering a new opcode to inject into the bytecode. The latter really existed in 1.x. But grep the 2.3.6 source and there\u0026rsquo;s no custom opcode registration anywhere — 2.x replaced it with something slicker.\n3.2 Smuggling a pointer inside the bytecode These two lines at the top of LuaFunction.cpp are the cleverest thing in the plugin:\nstatic constexpr uint8 ScriptMagicHeader[] = {EX_StringConst, \u0026#39;L\u0026#39;, \u0026#39;U\u0026#39;, \u0026#39;A\u0026#39;, \u0026#39;\\0\u0026#39;, EX_UInt64Const}; static constexpr size_t ScriptMagicHeaderSize = sizeof ScriptMagicHeader; LuaFunction.cpp:23\nWhen overriding a Blueprint function the class already implements (ULuaFunction::SetActive, LuaFunction.cpp:226):\nScript = Function-\u0026gt;Script; // original bytecode moves onto the ULuaFunction Children = Function-\u0026gt;Children; // parameter property chain is shared, not copied ... Function-\u0026gt;FunctionFlags |= FUNC_Native; Function-\u0026gt;SetNativeFunc(\u0026amp;execScriptCallLua); Function-\u0026gt;Script.Empty(); Function-\u0026gt;Script.AddUninitialized(ScriptMagicHeaderSize + sizeof(ULuaFunction*)); const auto Data = Function-\u0026gt;Script.GetData(); FPlatformMemory::Memcpy(Data, ScriptMagicHeader, ScriptMagicHeaderSize); FPlatformMemory::WriteUnaligned\u0026lt;ULuaFunction*\u0026gt;(Data + ScriptMagicHeaderSize, this); The original function\u0026rsquo;s bytecode array is emptied and replaced by a 6-byte magic header plus a raw ULuaFunction*. The header is built from real opcodes (EX_StringConst \u0026ldquo;LUA\\0\u0026rdquo; followed by EX_UInt64Const), so the buffer still looks like \u0026ldquo;push a string constant, push a 64-bit constant\u0026rdquo; — internally consistent rather than garbage.\nRetrieval (ULuaFunction::Get, LuaFunction.cpp:52) is a magic comparison plus a pointer read:\nif (FPlatformMemory::Memcmp(Data, ScriptMagicHeader, ScriptMagicHeaderSize) != 0) return nullptr; return FPlatformMemory::ReadUnaligned\u0026lt;ULuaFunction*\u0026gt;(Data + ScriptMagicHeaderSize); Why go to that trouble? Because the association has to live on the UFunction object itself. An external map would be invalidated by Blueprint recompilation, FuncMap rebuilds, or class GC; a pointer hidden in Script lives and dies with the UFunction.\n3.3 One override, three UFunctions After an override there are three same-named things in memory:\nObject Where Contents Original UFunction still in the owning class\u0026rsquo;s FuncMap FUNC_Native + thunk = execScriptCallLua; Script holds magic + pointer ULuaFunction ULuaOverridesClass (transient package) copy of the original bytecode, shared param chain, FFunctionDesc \u0026lt;Name\u0026gt;__Overridden same full duplicate of the original implementation; this is what self.Overridden reaches ULuaOverridesClass (LuaOverridesClass.cpp:19) is a shadow UClass created in the transient package, deliberately flagged CLASS_NewerVersionExists to dodge FBlueprintActionDatabase::RefreshClassActions — otherwise ghost classes would show up in the editor\u0026rsquo;s node palette. It splices itself into the target class\u0026rsquo;s Children list so GC and TFieldIterator can see the ULuaFunctions inside.\nWhy a shadow class instead of patching in place? An override needs a container that can own UFunctions without belonging to the game class: a ULuaFunction must be a field of some UClass to be referenced and collected properly, yet must not actually join the game class\u0026rsquo;s field list and pollute serialization.\nThe other branch is overriding an inherited function — say AActor::ReceiveBeginPlay from a Blueprint class. Here Function-\u0026gt;GetOuter() != Class, so the bAddNew path runs: the parent\u0026rsquo;s UFunction is left untouched, and a new ULuaFunction whose thunk is execCallLua is simply added to the subclass\u0026rsquo;s FuncMap. This is what the vast majority of real usage hits.\n3.4 What can be overridden bool ULuaFunction::IsOverridable(const UFunction* Function) { static constexpr uint32 FlagMask = FUNC_Native | FUNC_Event | FUNC_Net; static constexpr uint32 FlagResult = FUNC_Native | FUNC_Event; return Function-\u0026gt;HasAnyFunctionFlags(FUNC_BlueprintEvent) || (Function-\u0026gt;FunctionFlags \u0026amp; FlagMask) == FlagResult; } LuaFunction.cpp:74\nIn plain terms: only Blueprint events (BlueprintImplementableEvent, BlueprintNativeEvent, and events/functions defined in a Blueprint graph) plus non-networked native events. Add the ClassReps scan in GetOverridableFunctions (i.e. RepNotify functions) and you have the complete overridable set.\nThis explains two FAQs:\nWhy you write ReceiveBeginPlay, not BeginPlay — the Blueprint editor shows the DisplayName; the real UFUNCTION is ReceiveBeginPlay. Why an ordinary BlueprintCallable C++ function can\u0026rsquo;t be overridden — it has no FUNC_Event. To replace a chunk of Blueprint logic, the official answer is to Collapse To Function first, then override that function. The override list itself is an intersection of two sets (UnLuaManager.cpp:309):\n// replace the corresponding UFunctions on the class with every function in the Lua table for (const auto\u0026amp; LuaFuncName : BindInfo.LuaFunctions) { UFunction** Func = BindInfo.UEFunctions.Find(LuaFuncName); if (Func) ULuaFunction::Override(Function, Class, LuaFuncName); } Function names in the Lua table ∩ overridable UFunction names. A name that matches nothing is just an ordinary Lua method, and nothing is reported — which is exactly why a typo\u0026rsquo;d function name costs so much debugging time.\nInput events and AnimNotify take a different route: they have no corresponding UFunction, so UnLua duplicates a template function off UUnLuaManager (InputAction / InputAxis / TriggerAnimNotify…), renames it to the target name, attaches it to the game class, and binds the FInputActionBinding delegate to that name (UnLuaManager.cpp:367 onwards). So writing function M:Fire_Pressed() in Lua works by conjuring a UFunction out of thin air.\n3.5 Finding the Lua function at runtime Once the thunk fires, control reaches FFunctionRegistry::Invoke (FunctionRegistry.cpp:22). On the first call it walks the Super chain looking for a Lua function of the same name, then caches the result as a registry ref:\ndo { lua_pushstring(L, FuncDesc-\u0026gt;GetLuaFunctionName()); lua_rawget(L, -2); if (lua_isfunction(L, -1)) { ...; FuncRef = luaL_ref(L, LUA_REGISTRYINDEX); break; } lua_pop(L, 1); lua_pushstring(L, \u0026#34;Super\u0026#34;); lua_rawget(L, -2); lua_remove(L, -2); } while (lua_istable(L, -1)); If nothing is found it doesn\u0026rsquo;t crash — it falls back to the original implementation:\n// the lua module may have failed to load, so forward to the original function const auto Overridden = Function-\u0026gt;GetOverridden(); if (Overridden \u0026amp;\u0026amp; Stack.Code) Overridden-\u0026gt;Invoke(Context, Stack, RESULT_PARAM); A pragmatic design: if the script is broken, the game degrades to its original Blueprint behaviour instead of exploding.\n3.6 The cost: FuncMap is process-global An override mutates UClass::FuncMap, and there is exactly one UClass per process. Which means:\nan override applies to all instances, all Worlds, all PIE sessions simultaneously; bind once in the editor and the state survives into the next PIE run unless it\u0026rsquo;s restored; recompiling a Blueprint wipes FuncMap and silently kills the override. The fix for the last one is refreshingly blunt — stuff a sentinel function into FuncMap and check whether it\u0026rsquo;s still there next time (UnLuaManager.cpp:262):\n#if WITH_EDITOR // handle the case where a Blueprint recompile cleared FuncMap if (Class-\u0026gt;FindFunctionByName(\u0026#34;__UClassBindSucceeded\u0026#34;, EIncludeSuperFlag::Type::ExcludeSuper)) return true; ULuaFunction::RestoreOverrides(Class); #endif FLuaOverrides also exposes Suspend/Resume and registers a GUObjectArray delete listener so overrides are restored automatically when a class is destroyed. All of it is tax on mutating global state.\n4. Metatables are the cache self.Health looks like one table lookup. It\u0026rsquo;s worth counting the actual steps.\nThe Lua-side Index function (embedded as a C string chunk at UnLuaLib.cpp:163):\nlocal function Index(t, k) local mt = getmetatable(t) local super = mt while super do -- 1. walk the Lua module chain first local v = rawget(super, k) if v ~= nil and not rawequal(v, NotExist) then rawset(t, k, v) -- found: cache into the instance table return v end super = rawget(super, \u0026#34;Super\u0026#34;) end local p = mt[k] -- 2. fall through to CLASS_METATABLE.__index if p ~= nil then if type(p) == \u0026#34;userdata\u0026#34; then return GetUProperty(t, p) -- 3a. property: must be read every time elseif type(p) == \u0026#34;function\u0026#34; then rawset(t, k, p) -- 3b. function: cache the closure elseif rawequal(p, NotExist) then return nil end else rawset(mt, k, NotExist) -- 4. even \u0026#34;doesn\u0026#39;t exist\u0026#34; is cached end return p end On the C side, Class_Index → GetField (LuaCore.cpp:1061):\nlua_getmetatable(L, 1); lua_pushvalue(L, 2); int32 Type = lua_rawget(L, -2); if (Type == LUA_TNIL) GetFieldInternal(L); // only the first access goes through reflection After resolving an FFieldDesc, GetFieldInternal writes the result back into the metatable (LuaCore.cpp:1030): properties as a userdata wrapping a TSharedPtr\u0026lt;ITypeOps\u0026gt;, functions as a Class_CallUFunction closure. Inherited fields get cached in both the super and the derived metatable.\nSo the real cost breaks down as:\nAccess First time Afterwards Lua method self:Foo() module-chain rawget instance-table hit, no metamethod UFunction self:K2_GetActorLocation() module-chain miss → C call → reflection lookup → build FFunctionDesc instance-table hit on the closure, straight into CallUE Property self.Health as above every time: N module-chain misses → Class_Index (C call) → metatable rawget → returns descriptor → GetUProperty (second C call) → actual read Nonexistent field one reflection lookup NotExist sentinel, pure table hit Property access is the one category that can\u0026rsquo;t be cached away — it must run every time, and it crosses the C boundary twice. Type checking, on by default (ENABLE_TYPE_CHECK=1), adds an IsA walk to every property access (LowLevel.cpp:135):\nUClass* OwnerClass = Property-\u0026gt;GetOwnerClass(); if (Object-\u0026gt;IsA(OwnerClass)) return true; luaL_error(L, ... \u0026#34;Access property from invalid owner. %s should be a %s.\u0026#34;); The practical takeaway is blunt: don\u0026rsquo;t write self.A.B.C on a hot path — hoist properties and functions into locals. Worth noting too that the bundled benchmark (Content/Script/Tests/Benchmark/UnLuaBenchmarkProxy.lua) measures the raw-userdata path after local RawObject = Proxy.Object, skipping the Lua Index and the module chain. It reports a lower bound, not what your gameplay code will see.\n5. Calls in both directions Lua → UE The skeleton of FFunctionDesc::CallUE (FunctionDesc.cpp:171):\nBuffer-\u0026gt;Get() for the parameter frame; PreCall: InitializeValue per property, WriteValue_InContainer from the Lua stack, missing arguments filled from the UHT-collected defaults; Object-\u0026gt;UObject::ProcessEvent(FinalFunction, Params) — note the non-virtual call, bypassing any subclass override of ProcessEvent; PostCall: return value and out-params pushed back to Lua, then DestroyValue; Buffer-\u0026gt;Pop(Params). Frame allocation is governed by ENABLE_PERSISTENT_PARAM_BUFFER (on by default). In persistent mode each UFunction keeps a stack of buffers, with a counter to support recursion (ParamBufferAllocator.cpp:38 — that counter is exactly what 2.3.5 added to fix the recursive-overwrite bug, #563). Turn it off and every call degrades to FMemory::Malloc + Memzero + Free. In the default configuration, a warmed-up Lua→UE call performs no heap allocation — at the price of buffers that only ever grow.\nOne subtle semantic: if the UFunction Lua is calling is itself overridden by Lua, the call is redirected to the original implementation (FunctionDesc.cpp:213):\n#if ENABLE_CALL_OVERRIDDEN_FUNCTION const auto LuaFunction = ULuaFunction::Get(Function.Get()); if (LuaFunction \u0026amp;\u0026amp; LuaFunction-\u0026gt;GetOverridden()) FinalFunction = LuaFunction-\u0026gt;GetOverridden(); #endif That\u0026rsquo;s why self.Overridden.ReceiveBeginPlay(self) reaches the original Blueprint logic: Overridden is CLASS_METATABLE, so you get a reflection closure, and the closure gets redirected inside CallUE. It\u0026rsquo;s also why you must use . and never : — Content/Script/Tutorials/02_OverrideBlueprintEvents.lua calls this out explicitly. self.Overridden:SayHi(name) would pass the metatable as self.\nUE → Lua FFunctionDesc::CallLua (FunctionDesc.cpp:78) has a complication: the call may come from Blueprint bytecode, with arguments still sitting in the instruction stream. Hence the bUnpackParams branch, which manually Stack.Steps each argument into a buffer and rebuilds an FOutParmRec chain; otherwise it just uses Stack.Locals.\nThen lua_pcall, then out-params written back in order. There\u0026rsquo;s a refreshingly honest comment here (FunctionDesc.cpp:466):\n// out value // suppose out param is also pushed on stack? this is assumed done by user... so we can not trust it What if the Lua function returns nothing? With ENABLE_TYPE_CHECK on you get an error; with it off, default values are used (changed in 2.3.5). The ordering of return value vs out-params is controlled by UNLUA_LEGACY_RETURN_ORDER.\nDefault arguments and coroutines Default values in a C++ signature do not exist in reflection data. UnLua built a whole UHT plugin for this, UnLuaDefaultParamCollector, which scans every BlueprintCallable/Exec function at compile time, emits GDefaultParamCollection, and fills the gaps in PreCall. A build-time plugin so Lua can omit a few trailing arguments — that trade tells you a lot about the project\u0026rsquo;s priorities.\nLatent functions (Delay, MoveTo, …) run on coroutines. When PreCall sees a parameter named LatentInfo, it synthesizes an FLatentActionInfo whose callback target is UUnLuaManager::OnLatentActionCompleted, with the coroutine\u0026rsquo;s registry ref as the LinkID (FunctionDesc.cpp:300):\nFLatentActionInfo LatentActionInfo(ThreadRef, GetTypeHash(FGuid::NewGuid()), TEXT(\u0026#34;OnLatentActionCompleted\u0026#34;), (Env.GetManager())); When the engine\u0026rsquo;s latent action completes it calls back into Env-\u0026gt;ResumeThread(LinkID) → lua_resume. So you can write UE.UKismetSystemLibrary.Delay(self, 1.0) and just continue — as long as the code is running inside a coroutine.\n6. Value semantics: the sharpest edge This is the part I\u0026rsquo;d most want to know before starting.\nFPropertyDesc::GetValueInternal takes a bCreateCopy flag. For structs (PropertyDesc.cpp:1219):\nif (bCreateCopy) { void *Userdata = NewUserdataWithPadding(L, StructSize, StructName.Get(), UserdataPadding); StructProperty-\u0026gt;InitializeValue(Userdata); StructProperty-\u0026gt;CopySingleValue(Userdata, ValuePtr); // real copy } else { UnLua::PushPointer(L, (void*)ValuePtr, StructName.Get(), bFirstPropOfScriptStruct); // borrowed pointer } And when Class_Index reads a property it passes false (LuaCore.cpp:1218):\n(*Property)-\u0026gt;ReadValue_InContainer(L, Self, false); So local t = self.SomeTransform gives you a view into the UObject\u0026rsquo;s memory, not a copy. Mutating t mutates the object (plenty of code relies on this for in-place edits) — but the flip side is that once the object is destroyed, or an array reallocates, or a parameter frame is reused, that view is a dangling pointer.\nThe subtler case is override parameters. From CallLuaInternal (FunctionDesc.cpp:449):\nProperty-\u0026gt;ReadValue_InContainer(L, InParams, !UNLUA_LEGACY_ARGS_PASSING); UNLUA_LEGACY_ARGS_PASSING defaults to 1, so the negation makes bCreateCopy = false — struct arguments handed to an overriding Lua function point into the caller\u0026rsquo;s parameter frame, and that buffer is recycled after the call returns. Stashing such an argument in a member variable for use next frame is the canonical way to shoot yourself. The \u0026ldquo;argument passing\u0026rdquo; setting added in 2.3.3 exists to switch this to copy mode: safe, at the cost of a struct copy per call.\nContainers behave the same way: TArray\u0026rsquo;s Get returns a copy of an element, GetRef returns a reference — the sample code\u0026rsquo;s InterpFloats:GetRef(1) uses the latter precisely because it\u0026rsquo;s about to modify the element.\nUnLua knows this is risky and ships two mitigations:\n1. Dangling check (off by default). FDanglingCheck (LuaDanglingCheck.cpp) wraps boundary calls in a guard; on destruction the guard nulls out every struct userdata pointer lent out during the call and flags container userdata as released:\nvoid* Userdata = GetUserdataFast(L, -1, \u0026amp;TwoLevelPtr); check(TwoLevelPtr) *(void**)Userdata = nullptr; Cross-frame access then yields a Lua error instead of a random crash. Worth enabling during development.\n2. The ReleasedPtr sentinel. When a UObject is destroyed, FObjectRegistry::Unbind (ObjectRegistry.cpp:200) doesn\u0026rsquo;t just drop the mapping — it rewrites the pointer stored in the userdata:\n*((void**)Userdata) = (void*)LowLevel::ReleasedPtr; Any later access from Lua hits the IsReleasedPtr check and reports attempt to read property 'X' on released object. Far friendlier than dereferencing freed memory.\n7. Stitching two garbage collectors together Lua has a GC. Unreal has a GC. Both want to own object lifetime, and the seam between them is where bugs breed. UnLua\u0026rsquo;s answer: weak tables on the Lua side, explicit root sets on the Unreal side.\nFLuaEnv\u0026rsquo;s constructor creates a batch of weak-valued tables (LuaEnv.cpp:121, ObjectRegistry.cpp:52): UnLua_ObjectMap, StructMap, ArrayMap, ScriptContainerMap, UnLua_ManualRefProxyMap. They\u0026rsquo;re purely \u0026ldquo;same C++ address → same Lua object\u0026rdquo; caches and never prevent collection.\nOn the Unreal side there are two FObjectReferencers (LuaEnv.cpp:118):\nAutoObjectReference — rooted automatically while Lua holds the object, removed via NotifyUObjectLuaGC when Lua collects the userdata; ManualObjectReference — driven by UnLua.Ref / UnLua.Unref, with an FManualRefProxy carrying a __gc as a backstop. Bound objects get one more layer: their INSTANCE table is strongly referenced via luaL_ref in the registry (ObjectRegistry.cpp:155) until Unbind. So \u0026ldquo;can the Lua table outlive the UObject?\u0026rdquo; — yes, but only until Unbind, and Unbind is driven by NotifyUObjectDeleted.\nFLuaEnv::NotifyUObjectDeleted (LuaEnv.cpp:265) notifies every registry in a fixed order, and that order is itself scar tissue:\nPropertyRegistry-\u0026gt;NotifyUObjectDeleted(Object); FunctionRegistry-\u0026gt;NotifyUObjectDeleted(Object); if (Manager) Manager-\u0026gt;NotifyUObjectDeleted(Object); ObjectRegistry-\u0026gt;NotifyUObjectDeleted(Object); ClassRegistry-\u0026gt;NotifyUObjectDeleted(Object); EnumRegistry-\u0026gt;NotifyUObjectDeleted(Object); The Lua GC configuration is worth a look too (LuaEnv.cpp:135):\n#if 504 == LUA_VERSION_NUM lua_gc(L, LUA_GCGEN, 0, 0); // 5.4: generational GC by default #else lua_gc(L, LUA_GCSETPAUSE, 100); // 5.3: aggressive incremental settings lua_gc(L, LUA_GCSETSTEPMUL, 5000); #endif And the lua_State uses Unreal\u0026rsquo;s allocator (FLuaEnv::DefaultLuaAllocator), so the Lua heap shows up in Unreal\u0026rsquo;s memory stats — which matters a lot when profiling mobile memory.\nIncidentally, the CHANGELOG shows \u0026ldquo;intermittent crash during Lua GC\u0026rdquo; being chased from the 2.2.0 GC rework all the way to April 2026 — the newest commit on develop is literally \u0026ldquo;fix intermittent crash when a UObject is GC\u0026rsquo;d on the Lua side.\u0026rdquo; The seam between the two collectors has been this plugin\u0026rsquo;s long-running sore spot.\nRules of thumb:\nIf Lua needs to hold an object long-term, don\u0026rsquo;t rely on a Lua variable alone — either the object already has an Unreal-side reference, or call UnLua.Ref; treat struct and container references as locals you discard immediately; never keep them across frames; enable the dangling check during development so problems surface where they\u0026rsquo;re caused, not where they crash; check IsValid before touching an object that may have been destroyed. 8. Performance: what the code lets you derive The headline: a reflected Lua→UE call is in the same cost bracket as a Blueprint node calling a C++ function, because both end up in ProcessEvent. UnLua\u0026rsquo;s real win is where the logic itself runs — dense branching, loops and table work are clearly faster in Lua than in the Blueprint VM, while call-heavy boundary code isn\u0026rsquo;t cheap on either side.\nFixed costs per operation:\nOperation Paid every time Property read N module-chain rawgets → C call Class_Index → metatable rawget → GetCppInstance → IsA (with type checking on) → second C call GetUProperty → ReadValue Property write same, ending in WriteValue UFunction call cached closure hit → PreCall (per-argument InitializeValue + write) → GetFunctionCallspace → ProcessEvent → PostCall + DestroyValue Overridden function invoked thunk → IUnLuaModule::GetEnv → registry lookup for the ULuaFunction → possible Stack.Step unpacking → push arguments → lua_pcall → write back out-params Statically exported call plain C function: no FFunctionDesc, no parameter frame, no ProcessEvent FString / FName crossing encoding conversion and allocation every time (TCHAR_TO_UTF8 / FString construction) Math type arithmetic FVector and friends are hand-exported, but each operation allocates a fresh userdata for the result The knobs (UnLua.Build.cs:91):\nMacro Default Effect ENABLE_TYPE_CHECK 1 Off removes the per-property-access IsA and per-argument type validation; type errors become undefined behaviour ENABLE_PERSISTENT_PARAM_BUFFER 1 Reuses parameter frames, eliminating per-call malloc/free UNLUA_LEGACY_ARGS_PASSING 1 1 = struct arguments borrow pointers (fast, dangerous); 0 = copies (slower, safe) ENABLE_CALL_OVERRIDDEN_FUNCTION 1 Provides self.Overridden, at one extra ULuaFunction::Get per CallUE AUTO_UNLUA_STARTUP 1 Creates the lua_State at engine startup UNLUA_ENABLE_DEBUG 0 Heavy logging; diagnostics only ENABLE_UNREAL_INSIGHTS 0 lua_sethook pipes Lua calls into Insights; mutually exclusive with the dead-loop check There are only a handful of optimizations the code actually supports, but they\u0026rsquo;re all real: hoist UFunctions and hot properties into locals; take over genuine hotspots with static exports; keep per-frame Tick logic in C++/Blueprint and enter Lua only on events; prefer in-place math over chains that allocate temporaries.\nOn memory, keep three ledgers in mind: one Lua table per bound object (which grows as method closures get rawset into it), one metatable per UStruct (which doubles as the field-resolution cache), and one monotonically growing parameter-buffer pool per UFunction that Lua has ever called.\n9. Hot update and tooling The hot-update capability actually lives in the module loader. FLuaEnv inserts three searchers into package.searchers (LuaEnv.cpp:98), and the filesystem one searches in this order (LuaEnv.cpp:622):\n// prefer a standalone file in the download directory FPaths::Combine(FPaths::ProjectPersistentDownloadDir(), Pattern) // then the packaged directory FPaths::Combine(FPaths::ProjectDir(), Pattern) The download directory wins over the packaged one — that is the whole patch pipeline: drop new scripts into PersistentDownloadDir and require picks them up. The default search path is Content/Script/?.lua;Plugins/UnLua/Content/Script/?.lua; changing package.path does nothing (Unreal has its own filesystem), so you change UnLua.PackagePath. For encryption or a custom package format, register a loader via FLuaEnv::AddLoader.\nUnLua.HotReload() during development is a different thing. HotReload.lua states outright that it follows 云风 (Cloud Wu)\u0026rsquo;s well-known approach, preserving upvalues and live tables where it can. But combine two earlier facts — the module table was copied at bind time, and Lua methods get rawset into instance tables — and its limits follow: new logic always applies to new instances, and applies to existing ones only insofar as the caches were correctly replaced. The file\u0026rsquo;s own framing is honest: designed for development, replace what it can, worst case you restart.\nThe rest of the engineering surface:\nIntelliSense — UnLuaEditor walks every UClass/UStruct/UEnum and emits EmmyLua annotation stubs, so ---@type BP_PlayerCharacter_C gives you completion; a commandlet exists for CI. The downside is that stubs must be regenerated when types change. Packaging — Lua files aren\u0026rsquo;t assets, so the Script directory must be listed under \u0026ldquo;Additional Non-Asset Directories to Package.\u0026rdquo; That\u0026rsquo;s FAQ item #2 for a reason. Static export — UNLUA_EXPORT_CLASS / ADD_FUNCTION / ADD_PROPERTY / EXPORT_FUNCTION, registered during static initialization and merged into the same metatable as reflected data. The plugin\u0026rsquo;s own FVector and TArray are exported this way: its own hotspots don\u0026rsquo;t go through reflection either, which is telling. Tests — UnLuaTestSuite organizes regression cases by GitHub issue number (Content/Script/Tests/Regression/Issue###/), dozens of them, including cases like Chinese Blueprint names (Issue322). Quite disciplined for an open-source plugin. 10. Status in 2026: not abandoned, relocated This is easy to get wrong, so precisely (figures from the GitHub API, August 2026):\nThe last tag is v2.3.6, 2023-11-07 — no release in nearly three years; master has had 1 commit since 2024 (a LICENSE entity change on 2025-07-07); but develop is alive: UE5.4 support landed 2024-05 (PR #700), UE5.6 compile fixes 2025-09 through 2025-12 (PR #758), and the newest commit is 2026-04-02, \u0026ldquo;fix intermittent crash when a UObject is GC\u0026rsquo;d on the Lua side\u0026rdquo; (PR #765); those commits come mainly from community contributors (jozhn, crazytuzi, and others) rather than dedicated corporate staffing; UE5.7 is not supported; the issues are still open; 2,755 stars / 718 forks / 185 open issues+PRs. The accurate summary is: the official release line is frozen at UE5.3; the usable newer code lives on develop and is community-maintained. If you\u0026rsquo;re targeting UE5.4+ today, that means pulling from develop and accepting no tags, no release notes, and issue response at volunteer pace.\nThe neighbours over the same period:\nProject Status (2026-08) Binding approach UnLua 2,755★, releases frozen at 2023-11, develop active to 2026-04 Runtime reflection sluaunreal (Tencent) 1,974★, last push 2025-10, tag 2.1.4 (2024-06) Reflection + codegen puerts (Tencent, TS/JS) 6,161★, last push 2026-07-31, the most active of the three Codegen + reflection UnrealEnginePython Last push 2022-06, effectively abandoned Reflection Angelscript (Hazelight) Not open-sourced on GitHub; distributed via angelscript.hazelight.se Compile-time static binding On adoption, the official (Chinese) FAQ has a line worth quoting, translated:\nAround forty projects inside Tencent are known to use UnLua; external projects can\u0026rsquo;t be counted.\nAnd the perennial question: can you swap in LuaJIT? There are feature/luajit and feature/lua51 branches (both stuck at a February 2023 \u0026ldquo;add LuaJIT build script\u0026rdquo; commit), and 2.3.5 added a \u0026ldquo;custom Lua version\u0026rdquo; setting. But the code imposes a hard constraint: LuaCore.cpp directly #includes lstate.h / lobject.h, reimplements lua_index2value itself, and appends a magic struct to the tail of every userdata for tagging:\n#define USERDATA_MAGIC 0x1688 #define BIT_TWOLEVEL_PTR (1 \u0026lt;\u0026lt; 5) struct FUserdataDesc { uint16 magic; uint8 tag; uint8 padding; }; All of that depends on Lua 5.4\u0026rsquo;s TValue/Udata memory layout. Switching to LuaJIT is not a matter of flipping a macro — that layer has to be rebuilt. Evaluate it as a plugin welded to Lua 5.4.\nWhen to reach for it Good fit: projects already organizing gameplay in Blueprint, needing mobile hot-updates, with existing Lua expertise on the team, and able to live with type errors that only surface at runtime. Its greatest strength is marginal cost — adding a UPROPERTY requires touching no binding code at all, and that dividend keeps paying out over a long, iteration-heavy project life.\nPoor fit: chasing peak boundary-call performance (static binding, or just write C++); needing strong typing and large-scale refactoring support (consider puerts\u0026rsquo; TypeScript route, or Angelscript); tracking the newest engine versions (budget for the develop-branch maintenance); or a team where nobody wants to read the override machinery when something breaks.\nThat last one is the real filter. UnLua\u0026rsquo;s code is good, but what it does — hiding pointers inside bytecode, hanging shadow classes off UClass, mutating a process-global FuncMap, lending raw pointers across two garbage collectors — is inherently not a black box. Using it means someone has to be able to read it. I hope this article makes that someone a little easier to find.\n","permalink":"https://hsiang0117.github.io/en/posts/unlua-source-analysis/","summary":"A source-level teardown of Tencent\u0026rsquo;s UnLua: how the override mechanism smuggles Lua functions into UFunctions, how metatables double as the reflection cache, why value semantics are the sharpest edge, how the two garbage collectors are stitched together, and what the project\u0026rsquo;s real maintenance status is in 2026.","title":"Inside UnLua: How Lua Grafts Itself onto Unreal's Reflection System"},{"content":"3DGS-Volume-Cloud GitHub\nA research project that replaces ray-marched volumetric clouds in game engines with physically-parameterized 3D Gaussian Splatting, targeting real-time rendering with dynamic relighting (arbitrary sun direction at inference time).\nBuilt on the 3DGS (Kerbl et al., 2023) codebase, with the representation, shading, rasterizer, and training pipeline reworked for participating media.\nWhy this exists Volumetric clouds in game engines have long relied on ray-marching: dozens of steps along the view ray per pixel, plus a light march at every sample, with cost climbing as resolution and cloud depth grow. 3D Gaussian Splatting replaces the per-pixel stepping with rasterization, but the cost lands on the representation — a vanilla Gaussian carries SH color plus a heuristic opacity, which describes \u0026ldquo;what color this looks like from a given direction,\u0026rdquo; not \u0026ldquo;how thick this medium is, how much it scatters, and in which direction.\u0026rdquo; Fitting a cloud with that bakes in one lighting condition: move the sun and the reconstruction no longer holds.\nSo the problem here is not \u0026ldquo;can a cloud be fitted convincingly\u0026rdquo; but whether the representation itself can be made physical while keeping rasterization speed — giving every Gaussian participating-medium quantities like extinction coefficient, scattering albedo and phase eccentricity, so that sun direction becomes an input at inference time rather than a constant baked in during training. That decision propagates all the way down: optical depth has to be integrated analytically instead of alpha-blended heuristically, self-shadowing has to be differentiable so shadow gradients can flow back, and every opacity-based heuristic in densification and pruning has to be replaced by something that still holds under the physical parameterization.\nDemo Your browser does not support the video tag. Two-stage design Stage 1 — train a physically-stable Gaussian point set on a sun-only, black-background dataset. Stage 2 — freeze Stage 1\u0026rsquo;s geometry and physical parameters and train only a global environment-lighting network, layering sky-atmosphere shading on top of the sun term for full relighting from arbitrary sun directions. Physical Gaussian parameters Each Gaussian carries medium quantities instead of SH color + heuristic opacity:\nParameter Meaning Activation σ_t peak extinction coefficient (1/m) softplus, clamp 5 ω scattering albedo (RGB) sigmoid g Henyey-Greenstein phase eccentricity 0.8·tanh, forward-scattering w_n 6-octave multi-scattering energy weights softplus The parameterization is a coherent physical design, not a patch: σ_t is intensive, so densification clones inherit it as-is without halving; opacity is the analytic quantity 1−exp(−τ) rather than a learnable parameter, so the stock opacity-reset heuristic is replaced by per-point \u0026ldquo;contribution resurrection.\u0026rdquo;\nKey differences from vanilla 3DGS 📐 Analytic optical-depth rasterization — the forked CUDA rasterizer accumulates each Gaussian\u0026rsquo;s analytic line integral of optical depth τ per pixel, giving physically correct Beer-Lambert extinction (α = 1 − exp(−τ)) instead of heuristic alpha blending. ☀️ Light-space self-shadowing (T_light) — a light-space raster pass records each Gaussian\u0026rsquo;s sun-ward transmittance (energy-weighted over the whole sunlit footprint, not center-sampled), with a natively differentiable backward that propagates shadow gradients to all occluders (full geometric gradients through scale/rotation). 💡 Physical shading \u0026amp; relighting — per-Gaussian radiance combines the HG phase function, six multi-scattering octaves, and self-shadow transmittance; the sun direction is a per-frame input, so trained clouds can be relit from arbitrary directions. 🪡 Needle surgery (structural anisotropy cap) — measured aniso tails are ~95% thin disks, not needles, so the pass fattens the thin axis ×2 (ratio halves) instead of splitting the long axis, emitting two children offset by ±σ_major/2 along the major axis with σ_t/3.2 extinction-mass conservation (volume ×2 × overlapping children → exact /4 over-cuts; /3.2 is mass-neutral in practice). It acts as a hard cap without fighting photometric gradients, converging in log2 passes. 🌱 Physics-aware densification \u0026amp; maintenance — contribution-based pruning (per-Gaussian Σ(α·T) CUDA channel), σ_t resurrection, adaptive densify threshold; new splits/clones get a 500-iteration prune_grace period so they are not pruned immediately, and the maintenance loop only runs during the densify window to prevent net destruction during settle. Environment lighting (Stage 2) L = T_sun ⊙ [Stage 1 sun term] + ω · Σ_lm E_lm · V_lm:\nT_sun — sun atmospheric transmittance, a 3-parameter analytic form exp(−m(θ)·τ_RGB) (fixed-geometry Kasten-Young air mass); low-sun darkening + reddening (τ_B\u0026gt;τ_R) falls out structurally — a purely additive term cannot express \u0026ldquo;darkening.\u0026rdquo; E_lm — low-order SH of the sky radiance field (small global MLP), additive in-scattering fill. V_lm — per-Gaussian sky-visibility SH transfer vectors, achromatic and purely geometric, stored as a code-level buffer rather than a learnable parameter — only the global T_sun/E_lm are new learnables, so the model cannot regress to vanilla 3DGS. The env network and V_lm persist as sidecars next to the PLY; the viewer/eval load them automatically. Dataset UE5-rendered volumetric cloud (WDAS cloud VDB): 60 Fibonacci-uniform hemisphere sun directions × rotating cameras = 1458 frames (train 1306 / test 152) in NeRF-synthetic format with per-frame sun_direction, including 4 fully held-out sun directions (96 frames) as relighting generalization tests. Uniform directional coverage is a precondition for healthy geometric shadow gradients. Acquisition, coordinate conversion, and test splitting are fully scripted (tools/).\nInteractive viewer A viser-based viewer: drag the sun direction live for relighting, inspect diagnostic channels (RGB / T_light / σ_t / depth), and optionally composite HDR sky backdrops.\nTech stack Python · CUDA · C++ · PyTorch — custom differentiable rasterizer forked from diff-gaussian-rasterization (analytic-tau / record_front_tau / lightpass-backward passes), three-stage data pipeline (UE capture → coordinate conversion → dataset split).\n","permalink":"https://hsiang0117.github.io/en/projects/3dgs-volume-cloud/","summary":"Physically-parameterized 3D Gaussian Splatting for real-time volumetric cloud rendering with arbitrary-sun relighting.","title":"3DGS-Volume-Cloud"},{"content":"WaterModifier GitHub\nAn AI-assisted desktop tool for editing water masks in geospatial terrain datasets. Browse a satellite tile map, click a few foreground / background points, let Segment Anything extract the water region, and write the result directly back into Cesium quantized-mesh .terrain tiles — automatically synchronized across LOD levels. Validated on a real GIS dataset (Yaohu Airport, maxzoom 18).\nBuilt as two cooperating processes: an Unreal Engine 5.4 frontend (map browsing, visualization, interaction) and a Python backend (SAM inference + terrain file surgery), talking over a TCP socket with a length-prefixed protocol.\nWhy this exists In a Cesium quantized-mesh dataset the water mask is not a layer you can open and save on its own — it lives as an extension record at the tail of every individual .terrain tile file. Its offset is not even fixed: you have to skip the 88-byte header, then step over the vertex data, the triangle indices (index width switching between 2 and 4 bytes depending on triangle count) and the west / south / east / north edge-index lists, each by its own length prefix, before you can scan the extension records and find out whether type 2 is there at all. The same body of water is also duplicated down the LOD pyramid — edit one level and every level below has four times as many tiles needing the same change.\nSo a human-scale editing intent — \u0026ldquo;this reservoir should be water\u0026rdquo; — lands on the data as byte-level rewrites of thousands of tile files. WaterModifier exists to connect those two ends: a tile pyramid laid out on screen at true geographic coordinates and a few mouse clicks on one side, bulk binary rewriting and level-by-level propagation on the other.\nFrontend (UE 5.4, C++) 🗺️ TMS tile-map browser — top-down orthographic camera with drag / scroll-zoom; the tile manager diffs the viewport\u0026rsquo;s tile range each frame and streams in only newly visible tiles 🌐 Live geo-coordinates — real-time lat/lon readout derived from the tileset\u0026rsquo;s units-per-pixel metadata; LOD switching re-anchors the camera to the same geographic position 💧 Water-mask visualization — parses the quantized-mesh .terrain binary format directly in C++ (vertex / index blocks, edge indices, then the extension records — extension type 2 is the water mask) and overlays existing water in blue Backend (Python + PyTorch) 🤖 SAM segmentation — the current viewport is exported as EXR, tone-mapped, and fed to Segment Anything (ViT-B, CUDA); left-clicks are foreground prompts, right-clicks background — iterate points until the mask is right ✍️ Manual mode — Photoshop-pen-style polygon selection, rasterized with ray-casting point-in-polygon tests 📝 In-place terrain editing — per-tile mask merge supporting both overwrite and additive modes, handling uniform 1-byte masks and full 256×256 masks, and appending the water-mask extension to tiles that never had one 🧹 Morphological cleanup — opening / closing passes remove segmentation noise, followed by smoothing convolutions so edited water edges blend naturally 🔁 LOD synchronization — edits recursively propagate to higher zoom levels by quadrant-splitting the parent mask and nearest-neighbor upsampling, all the way down to the dataset\u0026rsquo;s max LOD 📊 Live progress — a timer thread streams the modified-tile count back to the UI every 0.5 s during bulk writes Key engineering points Dual-end binary format consistency — Cesium quantized-mesh-1.0 is parsed twice, once per end: Python implements the writer (manually walking header / vertex blocks / index blocks / edge blocks / extension records, with 2/4-byte index width chosen by triangle count), UE C++ implements the reader (visualization) — both ends must agree byte-for-byte on the format Camera ↔ tile coordinate chain — a C++ function library implements the full pipeline from camera view → tile numbers → geo-coordinates (including tilemapresource.xml parsing and units-per-pixel conversion) Design decisions and trade-offs 🔌 Commands over the socket, pixels over the filesystem — the protocol carries only command keywords plus string arguments (click coordinates, LOD, terrain root, bottom-left tile indices, ortho width, tile size), with short replies like SegmentDone / ModifyDone. The two images are handed over as files in the working directory instead: UE exports the current viewport\u0026rsquo;s render target as EXR for Python to segment, and Python writes a mask PNG that UE reloads as a runtime texture. The rejected alternative was pushing pixel buffers through the socket; the cost is that both processes are pinned to one shared working directory. 🧭 Walking segment by segment rather than seeking to the mask — the mask offset depends on the vertex count, triangle count and index width, so a fixed offset is not an option and the code steps through using each segment\u0026rsquo;s own length prefix. The cost is that the same traversal is implemented twice — Python writes, UE C++ reads — and both sides\u0026rsquo; understanding of field offsets and skip rules has to stay in step permanently. 📦 The writer always emits the full grid — a mask payload is either a 1-byte uniform all-water / all-land flag or a full 256×256 = 65,536-byte grid. A 1-byte payload is promoted outright: rewrite the length field, drop that single byte, append the full grid; a tile with no mask extension at all gets one appended at the end of the file. Keeping the compact 1-byte form would save space but would require two branches at every downstream site; the cost as built is a whole-file read-modify-rewrite rather than an in-place byte patch. 📄 Three filenames as the dataset contract — the tile-map root must hold tilemapresource.xml (supplying the per-LOD units-per-pixel list) and meta.json (supplying maxzoom), and the terrain root must hold layer.json. maxzoom comes from a file rather than being inferred from the pyramid on disk: the benefit is predictable behaviour with no directory scan, the cost is that a dataset has to provide that file by convention. ⏱️ Progress reported by a self-re-arming timer — during a bulk edit a 0.5 s timer writes the completed tile count back to the UI, incrementing once per .terrain file written, including recursively synced children. It is simple and stays out of the write loop; the cost is a stream that bypasses the length-prefixed protocol and has to be recognized separately on the receiving end. Tech stack UE 5.4 / C++ · Python / PyTorch — segment-anything (ViT-B), NumPy / SciPy / OpenCV, TcpSocketPlugin; operates on TMS tile pyramids and Cesium quantized-mesh-1.0 terrain. Segmentation uses SAM ViT-B (segment_anything pinned to a specific upstream commit, torch 2.4.1 + cu124); the UI offers five ortho-width settings — 256 / 512 / 768 / 1024 / 2048 — with 1024 as the default.\n","permalink":"https://hsiang0117.github.io/en/projects/water-modifier/","summary":"An AI-assisted desktop tool for editing water masks in Cesium quantized-mesh terrain datasets — click a few points on the map, Segment Anything extracts the water, and the edit propagates through every LOD level.","title":"WaterModifier"},{"content":"TinyOpenGLRenderer GitHub\nA mini real-time renderer and scene editor built on OpenGL 4.5 + C++17, made for learning and experimenting with modern rendering techniques. Component-driven ECS-lite architecture, a RenderPass pipeline, and a Dear ImGui (docking) editor UI.\nWhy this exists Individual real-time rendering techniques are all well covered by tutorials online, but those tutorials tend to stand alone: one demo for shadows, another for volumetric clouds, a third for skeletal animation, with parameters hard-coded and a recompile needed to change a value. This project wanted the opposite — those techniques living in one scene, sharing a single lighting setup and a single render pipeline, with every parameter editable at runtime and the result visible immediately.\nA sizeable share of the work therefore went into places that produce no pixels directly: splitting rendering into an explicit RenderPass sequence (passes exchange state only through a RenderContext), turning renderable things into components attached to a GameObject, and giving every component type its own Inspector panel. The payoff is that the volumetric cloud\u0026rsquo;s dozen-plus parameters can be dragged and watched live, and several clouds with different parameters can coexist in one scene; the cost is a layer of indirection over a purpose-built demo.\nScreenshots Rendering 🎨 Forward HDR pipeline — RGBA16F render targets, Bloom (ping-pong Gaussian blur), exposure tone mapping 💡 Multiple lights — directional / point / spot, light data uploaded via SSBO 🌑 Shadows — 2D shadow map for the directional light (front-face culling against peter-panning), cube map array for omnidirectional point-light shadows ✂️ Real frustum culling — Gribb-Hartmann 6-plane extraction + AABB tests; off-screen objects (including point lights) are never submitted for drawing ☁️ Volumetric clouds (Horizon-style raymarch) — Perlin-Worley noise (low-freq FBM shaping + high-freq Worley edge erosion, domain warping, weather-map coverage), dual-lobe HG phase, 4-octave multi-scattering approximation, powder effect, self-shadowing light march; half-resolution rendering with depth-aware bilateral upsampling, empty-space skipping, and world-scale adaptive step counts. Noise textures are generated in parallel on worker threads with a versioned self-describing disk cache (auto-invalidated on parameter changes). All 15 cloud parameters (type, coverage, noise scale, lighting, wind…) are component-driven and editable live in the Inspector — multiple different clouds per scene 🦴 Skeletal animation — Assimp import, GPU skinning, bone debug overlay 🌅 Skybox and async model loading (PLY + common formats) Editor 🧩 Dear ImGui docking — scene tree / Inspector / render settings / assets / render-to-texture viewport 🔧 Component Inspector — dedicated panels for Transform, lights, materials, volumetric clouds 📌 Selection gizmo — XYZ axis arrows at the selected object\u0026rsquo;s origin (rotates with the object, always on top, constant screen size) 📜 In-app log console — GL debug output and engine logs inside the editor, no system console window ➕ Data-driven Add menu — one-click creation of lights / meshes / skybox / volumetric clouds Architecture ECS-lite — GameObject + components, per-type cached scene views RenderPass pipeline — FrameSetup → Shadows → LightUpload → ForwardHdr → Volume/Skeleton → Gizmo → Bloom → Composite, passes share state only through a RenderContext Dependency injection — Engine as composition root, no singletons (except the logger) RAII GL resources — move-only wrappers for textures / FBOs / buffers Driver-detail handling — static meshes bind a dummy texture on the bone-texture unit (8) so sampler completeness passes driver validation even in scenes without animation Tech stack C++17 · OpenGL 4.5 — GLFW, glad, glm, Assimp, Dear ImGui (docking), stb_image. Builds with Visual Studio 2022 (x64), all dependencies bundled in-repo. Dependencies are bundled in-repo (include/ + lib/); open the .sln, pick Debug/Release x64 and build. On launch it loads a default scene with a skybox, a ground plane and a directional light.\n","permalink":"https://hsiang0117.github.io/en/projects/tiny-opengl-renderer/","summary":"A mini real-time renderer / scene editor built on OpenGL 4.5 + C++17 — forward HDR pipeline, shadows, skeletal animation, raymarched volumetric clouds, and a Dear ImGui docking editor.","title":"TinyOpenGLRenderer"},{"content":"Many people learning OpenGL start with the LearnOpenGL tutorial. There are plenty of guides online for setting up an OpenGL environment, but most use Visual Studio as the IDE. CLion has a significant advantage over VS: you can configure multiple main functions via CMakeLists, meaning you don\u0026rsquo;t have to delete your previous code when moving on to later chapters — just create a new .cpp file with a new main function.\nSince the model-importing library Assimp used in the LearnOpenGL tutorial does not have a precompiled MinGW version (and compiling it yourself with MinGW tends to throw errors), we will switch CLion to the Visual Studio toolchain to use Assimp without issues.\n1. Switching CLion to the VS Toolchain Before starting, make sure the Visual Studio C++ development environment is already installed.\nAfter creating a C++ project in CLion, go to Settings → Build, Execution, Deployment → Toolchains. Click Add to create a new toolchain. Set Toolset to your VS installation directory — the rest will be auto-detected. Click Apply.\nThen go to Settings → Build, Execution, Deployment → CMake. Click Add to create a new profile — name it something like Debug-vs2022. You can also rename the default MinGW profile to Debug-Mingw for clarity. Set Build type to Debug and Toolchain to the VS toolchain you just added. Click Apply then OK.\nTwo CMake profiles will now appear in the top-right dropdown, and two corresponding build directories will show up in the project tree on the left. Run a sample program with the VS profile to verify everything works.\nNext, create three directories under the project root: include, lib, and src.\n2. Configuring GLFW Go to the GLFW website — An OpenGL library | GLFW — download the source package and extract it. You should get the following directory structure:\nCopy the entire include/GLFW folder into your project\u0026rsquo;s include directory, and copy lib-vc2022/glfw3.lib into the lib directory.\n3. Configuring GLAD Go to the GLAD website — glad.dav1d.de — select the version you need, download and extract it. You should get something like this:\nCopy both the include/glad and include/KHR folders into your project\u0026rsquo;s include directory, and copy src/glad.c into the project\u0026rsquo;s src directory.\n4. Configuring Assimp You can either pull the source from GitHub and compile it yourself, or download a pre-built version from kimkulling.itch.io/the-asset-importer-lib. The official site provides an .exe installer — download and install it to any directory.\nIn the installation directory, copy the entire include/assimp folder into your project\u0026rsquo;s include directory. Copy lib/x64/assimp-vc143-mt.lib into the project\u0026rsquo;s lib directory, and copy bin/x64/assimp-vc143-mt.dll into the cmake-build-debug-vs2022 directory (the build directory corresponding to your VS CMake profile). After this, you can uninstall the Assimp installer — it\u0026rsquo;s no longer needed.\n5. Other Configurations GLM: Clone the GLM repository from GitHub and copy the glm folder into your project\u0026rsquo;s include directory.\nstb_image: Place stb_image.h into the include directory.\nNo need to elaborate further here.\n6. Configuring CMakeLists Create two directories under the project root: demos and headers. demos will hold the .cpp source files you want to run, and headers will contain your own header files such as camera.h, shader.h, etc. You could also put your headers directly under include instead of creating a separate headers folder, but keeping them separate is cleaner.\nNow write the following in CMakeLists.txt:\ncmake_minimum_required(VERSION 3.30) project(LearnOpengl) # Replace with your project name set(CMAKE_CXX_STANDARD 20) include_directories(${PROJECT_SOURCE_DIR}/include ${PROJECT_SOURCE_DIR}/headers) link_directories(${PROJECT_SOURCE_DIR}/lib) file(GLOB files demos/*.cpp) foreach (file ${files}) string(REGEX REPLACE \u0026#34;.+/(.+)\\\\..*\u0026#34; \u0026#34;\\\\1\u0026#34; file_name ${file}) add_executable(${file_name} src/glad.c ${file}) target_link_libraries(${file_name} ${PROJECT_SOURCE_DIR}/lib/glfw3.lib) target_link_libraries(${file_name} ${PROJECT_SOURCE_DIR}/lib/assimp-vc143-mtd.lib) endforeach () Right-click CMakeLists.txt and select Reload CMake Project. CLion will automatically generate a runnable configuration for every .cpp file inside the demos directory. Run the following code to verify that GLFW and GLAD are set up correctly:\n#include \u0026lt;glad/glad.h\u0026gt; #include \u0026lt;GLFW/glfw3.h\u0026gt; #include \u0026lt;iostream\u0026gt; void framebuffer_size_callback(GLFWwindow* window, int width, int height); void processInput(GLFWwindow *window); // settings const unsigned int SCR_WIDTH = 800; const unsigned int SCR_HEIGHT = 600; const char *vertexShaderSource = \u0026#34;#version 330 core\\n\u0026#34; \u0026#34;layout (location = 0) in vec3 aPos;\\n\u0026#34; \u0026#34;void main()\\n\u0026#34; \u0026#34;{\\n\u0026#34; \u0026#34; gl_Position = vec4(aPos.x, aPos.y, aPos.z, 1.0);\\n\u0026#34; \u0026#34;}\\0\u0026#34;; const char *fragmentShaderSource = \u0026#34;#version 330 core\\n\u0026#34; \u0026#34;out vec4 FragColor;\\n\u0026#34; \u0026#34;void main()\\n\u0026#34; \u0026#34;{\\n\u0026#34; \u0026#34; FragColor = vec4(1.0f, 0.5f, 0.2f, 1.0f);\\n\u0026#34; \u0026#34;}\\n\\0\u0026#34;; int main() { // glfw: initialize and configure // ------------------------------ glfwInit(); glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3); glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3); glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE); #ifdef __APPLE__ glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE); #endif // glfw window creation // -------------------- GLFWwindow* window = glfwCreateWindow(SCR_WIDTH, SCR_HEIGHT, \u0026#34;LearnOpenGL\u0026#34;, NULL, NULL); if (window == NULL) { std::cout \u0026lt;\u0026lt; \u0026#34;Failed to create GLFW window\u0026#34; \u0026lt;\u0026lt; std::endl; glfwTerminate(); return -1; } glfwMakeContextCurrent(window); glfwSetFramebufferSizeCallback(window, framebuffer_size_callback); // glad: load all OpenGL function pointers // --------------------------------------- if (!gladLoadGLLoader((GLADloadproc)glfwGetProcAddress)) { std::cout \u0026lt;\u0026lt; \u0026#34;Failed to initialize GLAD\u0026#34; \u0026lt;\u0026lt; std::endl; return -1; } // build and compile our shader program // ------------------------------------ // vertex shader unsigned int vertexShader = glCreateShader(GL_VERTEX_SHADER); glShaderSource(vertexShader, 1, \u0026amp;vertexShaderSource, NULL); glCompileShader(vertexShader); // check for shader compile errors int success; char infoLog[512]; glGetShaderiv(vertexShader, GL_COMPILE_STATUS, \u0026amp;success); if (!success) { glGetShaderInfoLog(vertexShader, 512, NULL, infoLog); std::cout \u0026lt;\u0026lt; \u0026#34;ERROR::SHADER::VERTEX::COMPILATION_FAILED\\n\u0026#34; \u0026lt;\u0026lt; infoLog \u0026lt;\u0026lt; std::endl; } // fragment shader unsigned int fragmentShader = glCreateShader(GL_FRAGMENT_SHADER); glShaderSource(fragmentShader, 1, \u0026amp;fragmentShaderSource, NULL); glCompileShader(fragmentShader); // check for shader compile errors glGetShaderiv(fragmentShader, GL_COMPILE_STATUS, \u0026amp;success); if (!success) { glGetShaderInfoLog(fragmentShader, 512, NULL, infoLog); std::cout \u0026lt;\u0026lt; \u0026#34;ERROR::SHADER::FRAGMENT::COMPILATION_FAILED\\n\u0026#34; \u0026lt;\u0026lt; infoLog \u0026lt;\u0026lt; std::endl; } // link shaders unsigned int shaderProgram = glCreateProgram(); glAttachShader(shaderProgram, vertexShader); glAttachShader(shaderProgram, fragmentShader); glLinkProgram(shaderProgram); // check for linking errors glGetProgramiv(shaderProgram, GL_LINK_STATUS, \u0026amp;success); if (!success) { glGetProgramInfoLog(shaderProgram, 512, NULL, infoLog); std::cout \u0026lt;\u0026lt; \u0026#34;ERROR::SHADER::PROGRAM::LINKING_FAILED\\n\u0026#34; \u0026lt;\u0026lt; infoLog \u0026lt;\u0026lt; std::endl; } glDeleteShader(vertexShader); glDeleteShader(fragmentShader); // set up vertex data (and buffer(s)) and configure vertex attributes // ------------------------------------------------------------------ float vertices[] = { -0.5f, -0.5f, 0.0f, // left 0.5f, -0.5f, 0.0f, // right 0.0f, 0.5f, 0.0f // top }; unsigned int VBO, VAO; glGenVertexArrays(1, \u0026amp;VAO); glGenBuffers(1, \u0026amp;VBO); // bind the Vertex Array Object first, then bind and set vertex buffer(s), and then configure vertex attributes(s). glBindVertexArray(VAO); glBindBuffer(GL_ARRAY_BUFFER, VBO); glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW); glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(float), (void*)0); glEnableVertexAttribArray(0); // note that this is allowed, the call to glVertexAttribPointer registered VBO as the vertex attribute\u0026#39;s bound vertex buffer object so afterwards we can safely unbind glBindBuffer(GL_ARRAY_BUFFER, 0); // You can unbind the VAO afterwards so other VAO calls won\u0026#39;t accidentally modify this VAO, but this rarely happens. Modifying other // VAOs requires a call to glBindVertexArray anyways so we generally don\u0026#39;t unbind VAOs (nor VBOs) when it\u0026#39;s not directly necessary. glBindVertexArray(0); // uncomment this call to draw in wireframe polygons. //glPolygonMode(GL_FRONT_AND_BACK, GL_LINE); // render loop // ----------- while (!glfwWindowShouldClose(window)) { // input // ----- processInput(window); // render // ------ glClearColor(0.2f, 0.3f, 0.3f, 1.0f); glClear(GL_COLOR_BUFFER_BIT); // draw our first triangle glUseProgram(shaderProgram); glBindVertexArray(VAO); // seeing as we only have a single VAO there\u0026#39;s no need to bind it every time, but we\u0026#39;ll do so to keep things a bit more organized glDrawArrays(GL_TRIANGLES, 0, 3); // glBindVertexArray(0); // no need to unbind it every time // glfw: swap buffers and poll IO events (keys pressed/released, mouse moved etc.) // ------------------------------------------------------------------------------- glfwSwapBuffers(window); glfwPollEvents(); } // optional: de-allocate all resources once they\u0026#39;ve outlived their purpose: // ------------------------------------------------------------------------ glDeleteVertexArrays(1, \u0026amp;VAO); glDeleteBuffers(1, \u0026amp;VBO); glDeleteProgram(shaderProgram); // glfw: terminate, clearing all previously allocated GLFW resources. // ------------------------------------------------------------------ glfwTerminate(); return 0; } // process all input: query GLFW whether relevant keys are pressed/released this frame and react accordingly // --------------------------------------------------------------------------------------------------------- void processInput(GLFWwindow *window) { if (glfwGetKey(window, GLFW_KEY_ESCAPE) == GLFW_PRESS) glfwSetWindowShouldClose(window, true); } // glfw: whenever the window size changed (by OS or user resize) this callback function executes // --------------------------------------------------------------------------------------------- void framebuffer_size_callback(GLFWwindow* window, int width, int height) { // make sure the viewport matches the new window dimensions; note that width and // height will be significantly larger than specified on retina displays. glViewport(0, 0, width, height); } You can also grab any model file and run the following code to verify that Assimp is set up correctly:\n#include \u0026lt;assimp/Importer.hpp\u0026gt; #include \u0026lt;assimp/scene.h\u0026gt; #include \u0026lt;assimp/postprocess.h\u0026gt; #include \u0026lt;iostream\u0026gt; int main() { Assimp::Importer importer; const aiScene* scene = importer.ReadFile(\u0026#34;path_to_your_model.obj\u0026#34;, aiProcess_Triangulate); if (!scene) { std::cerr \u0026lt;\u0026lt; \u0026#34;Error: \u0026#34; \u0026lt;\u0026lt; importer.GetErrorString() \u0026lt;\u0026lt; std::endl; return -1; } std::cout \u0026lt;\u0026lt; \u0026#34;Assimp OK, load model succeed.\u0026#34; \u0026lt;\u0026lt; std::endl; return 0; } As you learn new content, simply create a new .cpp file inside the demos directory and reload the CMake project. No need to clear out your previous work — you can always revisit what you\u0026rsquo;ve learned. Put your own header files inside the headers directory.\nHappy learning!\n","permalink":"https://hsiang0117.github.io/en/posts/clion_setup/","summary":"A complete guide to configuring an OpenGL development environment in CLion, covering GLFW, GLAD, and Assimp library setup, with step-by-step instructions for switching to the Visual Studio toolchain.","title":"Setting Up OpenGL Dev Environment with CLion — GLFW + GLAD + Assimp"}]