ffi_accel

LuaJIT FFI-based accelerator for map generators

This mod is an accelerator for Minetest map generators written in Lua. It accelerates the memory-intensive functions in LuaVoxelManip and PerlinNoiseMap, by adding an emulation layer on top of them that allows these functions to run faster.

It additionally prevents the usage of the extra LuaJIT memory needed to hold the data, leaving it in the C++ memory area, making it less likely that LuaJIT crashes due to memory problems.

There are cons, too. First, it works by implementing a sort of emulation layer, making the mod believe it's writing to a Lua table when it's actually writing to the C++ memory directly. In some circumstances, this could not work in the way it's designed to work, potentially leading to mod misbehaviour or even crashes (e.g. if the table is written to after removing the PerlinNoiseMap or VoxelManip object). Fortunately, these cases are expected to be rare. But be careful to only use mods that you trust, since that kind of access could be exploitable by a rogue mod.

Second, while it accelerates the functions that read and write the data back (by replacing these operations with just a simple setup of a metatable), the reading and writing of the table data will be slower, because it's redirected through a function that performs the access to the C++ memory after doing bounds checks.

Finally, for some mods like WorldEdit there may be no acceleration at all.

Installation

Before this mod can do its job, you need to:

• Make sure you are using LuaJIT.
• Patch the Minetest engine with the ffi_accel.patch provided, and compile it.
• Add ffi_accel to secure.trusted_mods within minetest.conf (see minetest.conf.example for an explanation of secure.trusted_mods).

The mod can then be installed as usual. Enabling the ffi_accel mod will then suffice to get acceleration. If something goes wrong, you will get diagnostic information in the log. If it is successfully activated, it will log the message "Mod ffi_accel activated!".

For developers

If you are a mapgen mod developer, you may be interested in this. It is possible to write a mod that directly accesses the C++ memory, by requiring the engine patch but not the ffi_accel mod, and implementing FFI access yourself. This will eliminate the overhead that ffi_accel adds on every table access. It can even offer dual mode, for cases where either LuaJIT/FFI or trusted mode are not available. The file FFI-HOWTO.md explains how. (TODO: THIS IS NOT WRITTEN YET!)

Credits

This mod is written from scratch, based on an idea by EvidenceBKidsCode (see https://github.com/EvidenceBKidscode/ffiopt to see the mod where the idea comes from).

License

This mod is licensed under very simple and permissive terms. See the file init.lua for details.

The new code in the patch is donated to the public domain. For the patch to be functional, it necessarily includes context code that belongs to the Minetest sources, written by Perttu Ahola and the Minetest contributors and licensed under the LGPL v2.1.