feature: added the official LuaJIT documentation from LuaJIT 2.1 to our restydoc indexes.

doc/LuaJIT-2.1/ext_ffi_tutorial.pod

@@ -0,0 +1,640 @@
+=pod
+
+LuaJIT
+
+=head1 FFI Tutorial
+
+=over
+
+=item * LuaJIT
+
+=over
+
+=item * Download E<rchevron>
+
+=item * Installation
+
+=item * Running
+
+=back
+
+=item * Extensions
+
+=over
+
+=item * FFI Library
+
+=over
+
+=item * FFI Tutorial
+
+=item * ffi.* API
+
+=item * FFI Semantics
+
+=back
+
+=item * jit.* Library
+
+=item * Lua/C API
+
+=item * Profiler
+
+=back
+
+=item * Status
+
+=over
+
+=item * Changes
+
+=back
+
+=item * FAQ
+
+=item * Performance E<rchevron>
+
+=item * Wiki E<rchevron>
+
+=item * Mailing List E<rchevron>
+
+=back
+
+This page is intended to give you an overview of the features of the
+FFI library by presenting a few use cases and guidelines.
+
+This page makes no attempt to explain all of the FFI library, though.
+You'll want to have a look at the ffi.* API function reference and the
+FFI semantics to learn more.
+
+=head2 Loading the FFI Library
+
+The FFI library is built into LuaJIT by default, but it's not loaded
+and initialized by default. The suggested way to use the FFI library is
+to add the following to the start of every Lua file that needs one of
+its functions:
+
+ local ffi = require("ffi")
+
+Please note this doesn't define an C<ffi> variable in the table of
+globals E<mdash> you really need to use the local variable. The
+C<require> function ensures the library is only loaded once.
+
+Note: If you want to experiment with the FFI from the interactive
+prompt of the command line executable, omit the C<local>, as it doesn't
+preserve local variables across lines.
+
+=head2 Accessing Standard System Functions
+
+The following code explains how to access standard system functions. We
+slowly print two lines of dots by sleeping for 10 milliseconds after
+each dot:
+
+  
+ ①
+ 
+ 
+ 
+ 
+ 
+ ②
+ ③
+ ④
+ 
+ 
+ 
+ ⑤
+ 
+ 
+ 
+ 
+ 
+ ⑥local ffi = require("ffi")
+ ffi.cdef[[
+ void Sleep(int ms);
+ int poll(struct pollfd *fds, unsigned long nfds, int timeout);
+ ]]
+ 
+ local sleep
+ if ffi.os == "Windows" then
+   function sleep(s)
+     ffi.C.Sleep(s*1000)
+   end
+ else
+   function sleep(s)
+     ffi.C.poll(nil, 0, s*1000)
+   end
+ end
+ 
+ for i=1,160 do
+   io.write("."); io.flush()
+   sleep(0.01)
+ end
+ io.write("\n")
+
+Here's the step-by-step explanation:
+
+This defines the C library functions we're going to use. The part
+inside the double-brackets (in green) is just standard C syntax. You
+can usually get this info from the C header files or the documentation
+provided by each C library or C compiler.
+
+The difficulty we're facing here, is that there are different standards
+to choose from. Windows has a simple C<Sleep()> function. On other
+systems there are a variety of functions available to achieve
+sub-second sleeps, but with no clear consensus. Thankfully C<poll()>
+can be used for this task, too, and it's present on most non-Windows
+systems. The check for C<ffi.os> makes sure we use the Windows-specific
+function only on Windows systems.
+
+Here we're wrapping the call to the C function in a Lua function. This
+isn't strictly necessary, but it's helpful to deal with system-specific
+issues only in one part of the code. The way we're wrapping it ensures
+the check for the OS is only done during initialization and not for
+every call.
+
+A more subtle point is that we defined our C<sleep()> function (for the
+sake of this example) as taking the number of seconds, but accepting
+fractional seconds. Multiplying this by 1000 gets us milliseconds, but
+that still leaves it a Lua number, which is a floating-point value.
+Alas, the C<Sleep()> function only accepts an integer value. Luckily
+for us, the FFI library automatically performs the conversion when
+calling the function (truncating the FP value towards zero, like in C).
+
+Some readers will notice that C<Sleep()> is part of C<KERNEL32.DLL> and
+is also a C<stdcall> function. So how can this possibly work? The FFI
+library provides the C<ffi.C> default C library namespace, which allows
+calling functions from the default set of libraries, like a C compiler
+would. Also, the FFI library automatically detects C<stdcall>
+functions, so you don't need to declare them as such.
+
+The C<poll()> function takes a couple more arguments we're not going to
+use. You can simply use C<nil> to pass a C<NULL> pointer and C<0> for
+the C<nfds> parameter. Please note that the number C<0> I<does not
+convert to a pointer value>, unlike in C++. You really have to pass
+pointers to pointer arguments and numbers to number arguments.
+
+The page on FFI semantics has all of the gory details about conversions
+between Lua objects and C types. For the most part you don't have to
+deal with this, as it's performed automatically and it's carefully
+designed to bridge the semantic differences between Lua and C.
+
+Now that we have defined our own C<sleep()> function, we can just call
+it from plain Lua code. That wasn't so bad, huh? Turning these boring
+animated dots into a fascinating best-selling game is left as an
+exercise for the reader. :-)
+
+=head2 Accessing the zlib Compression Library
+
+The following code shows how to access the zlib compression library
+from Lua code. We'll define two convenience wrapper functions that take
+a string and compress or uncompress it to another string:
+
+  
+ ①
+ 
+ 
+ 
+ 
+ 
+ 
+ ②
+ 
+ 
+ ③
+ 
+ ④
+ 
+ 
+ ⑤
+ 
+ 
+ ⑥
+ 
+ 
+ 
+ 
+ 
+ 
+ 
+ ⑦local ffi = require("ffi")
+ ffi.cdef[[
+ unsigned long compressBound(unsigned long sourceLen);
+ int compress2(uint8_t *dest, unsigned long *destLen,
+ 	      const uint8_t *source, unsigned long sourceLen, int level);
+ int uncompress(uint8_t *dest, unsigned long *destLen,
+ 	       const uint8_t *source, unsigned long sourceLen);
+ ]]
+ local zlib = ffi.load(ffi.os == "Windows" and "zlib1" or "z")
+ 
+ local function compress(txt)
+   local n = zlib.compressBound(#txt)
+   local buf = ffi.new("uint8_t[?]", n)
+   local buflen = ffi.new("unsigned long[1]", n)
+   local res = zlib.compress2(buf, buflen, txt, #txt, 9)
+   assert(res == 0)
+   return ffi.string(buf, buflen[0])
+ end
+ 
+ local function uncompress(comp, n)
+   local buf = ffi.new("uint8_t[?]", n)
+   local buflen = ffi.new("unsigned long[1]", n)
+   local res = zlib.uncompress(buf, buflen, comp, #comp)
+   assert(res == 0)
+   return ffi.string(buf, buflen[0])
+ end
+ 
+ -- Simple test code.
+ local txt = string.rep("abcd", 1000)
+ print("Uncompressed size: ", #txt)
+ local c = compress(txt)
+ print("Compressed size: ", #c)
+ local txt2 = uncompress(c, #txt)
+ assert(txt2 == txt)
+
+Here's the step-by-step explanation:
+
+This defines some of the C functions provided by zlib. For the sake of
+this example, some type indirections have been reduced and it uses the
+pre-defined fixed-size integer types, while still adhering to the zlib
+API/ABI.
+
+This loads the zlib shared library. On POSIX systems it's named
+C<libz.so> and usually comes pre-installed. Since C<ffi.load()>
+automatically adds any missing standard prefixes/suffixes, we can
+simply load the C<"z"> library. On Windows it's named C<zlib1.dll> and
+you'll have to download it first from the E<rchevron> zlib site. The
+check for C<ffi.os> makes sure we pass the right name to C<ffi.load()>.
+
+First, the maximum size of the compression buffer is obtained by
+calling the C<zlib.compressBound> function with the length of the
+uncompressed string. The next line allocates a byte buffer of this
+size. The C<[?]> in the type specification indicates a variable-length
+array (VLA). The actual number of elements of this array is given as
+the 2nd argument to C<ffi.new()>.
+
+This may look strange at first, but have a look at the declaration of
+the C<compress2> function from zlib: the destination length is defined
+as a pointer! This is because you pass in the maximum buffer size and
+get back the actual length that was used.
+
+In C you'd pass in the address of a local variable (C<&buflen>). But
+since there's no address-of operator in Lua, we'll just pass in a
+one-element array. Conveniently it can be initialized with the maximum
+buffer size in one step. Calling the actual C<zlib.compress2> function
+is then straightforward.
+
+We want to return the compressed data as a Lua string, so we'll use
+C<ffi.string()>. It needs a pointer to the start of the data and the
+actual length. The length has been returned in the C<buflen> array, so
+we'll just get it from there.
+
+Note that since the function returns now, the C<buf> and C<buflen>
+variables will eventually be garbage collected. This is fine, because
+C<ffi.string()> has copied the contents to a newly created (interned)
+Lua string. If you plan to call this function lots of times, consider
+reusing the buffers and/or handing back the results in buffers instead
+of strings. This will reduce the overhead for garbage collection and
+string interning.
+
+The C<uncompress> functions does the exact opposite of the C<compress>
+function. The compressed data doesn't include the size of the original
+string, so this needs to be passed in. Otherwise no surprises here.
+
+The code, that makes use of the functions we just defined, is just
+plain Lua code. It doesn't need to know anything about the LuaJIT FFI
+E<mdash> the convenience wrapper functions completely hide it.
+
+One major advantage of the LuaJIT FFI is that you are now able to write
+those wrappers I<in Lua>. And at a fraction of the time it would cost
+you to create an extra C module using the Lua/C API. Many of the
+simpler C functions can probably be used directly from your Lua code,
+without any wrappers.
+
+Side note: the zlib API uses the C<long> type for passing lengths and
+sizes around. But all those zlib functions actually only deal with 32
+bit values. This is an unfortunate choice for a public API, but may be
+explained by zlib's history E<mdash> we'll just have to deal with it.
+
+First, you should know that a C<long> is a 64 bit type e.g. on
+POSIX/x64 systems, but a 32 bit type on Windows/x64 and on 32 bit
+systems. Thus a C<long> result can be either a plain Lua number or a
+boxed 64 bit integer cdata object, depending on the target system.
+
+Ok, so the C<ffi.*> functions generally accept cdata objects wherever
+you'd want to use a number. That's why we get a away with passing C<n>
+to C<ffi.string()> above. But other Lua library functions or modules
+don't know how to deal with this. So for maximum portability one needs
+to use C<tonumber()> on returned C<long> results before passing them
+on. Otherwise the application might work on some systems, but would
+fail in a POSIX/x64 environment.
+
+=head2 Defining Metamethods for a C Type
+
+The following code explains how to define metamethods for a C type. We
+define a simple point type and add some operations to it:
+
+  
+ ①
+ 
+ 
+ 
+ ②
+ 
+ ③
+ 
+ ④
+ 
+ 
+ 
+ ⑤
+ 
+ ⑥local ffi = require("ffi")
+ ffi.cdef[[
+ typedef struct { double x, y; } point_t;
+ ]]
+ 
+ local point
+ local mt = {
+   __add = function(a, b) return point(a.x+b.x, a.y+b.y) end,
+   __len = function(a) return math.sqrt(a.x*a.x + a.y*a.y) end,
+   __index = {
+     area = function(a) return a.x*a.x + a.y*a.y end,
+   },
+ }
+ point = ffi.metatype("point_t", mt)
+ 
+ local a = point(3, 4)
+ print(a.x, a.y)  --> 3  4
+ print(#a)        --> 5
+ print(a:area())  --> 25
+ local b = a + point(0.5, 8)
+ print(#b)        --> 12.5
+
+Here's the step-by-step explanation:
+
+This defines the C type for a two-dimensional point object.
+
+We have to declare the variable holding the point constructor first,
+because it's used inside of a metamethod.
+
+Let's define an C<__add> metamethod which adds the coordinates of two
+points and creates a new point object. For simplicity, this function
+assumes that both arguments are points. But it could be any mix of
+objects, if at least one operand is of the required type (e.g. adding a
+point plus a number or vice versa). Our C<__len> metamethod returns the
+distance of a point to the origin.
+
+If we run out of operators, we can define named methods, too. Here the
+C<__index> table defines an C<area> function. For custom indexing
+needs, one might want to define C<__index> and C<__newindex>
+I<functions> instead.
+
+This associates the metamethods with our C type. This only needs to be
+done once. For convenience, a constructor is returned by
+C<ffi.metatype()>. We're not required to use it, though. The original C
+type can still be used e.g. to create an array of points. The
+metamethods automatically apply to any and all uses of this type.
+
+Please note that the association with a metatable is permanent and
+B<the metatable must not be modified afterwards!> Ditto for the
+C<__index> table.
+
+Here are some simple usage examples for the point type and their
+expected results. The pre-defined operations (such as C<a.x>) can be
+freely mixed with the newly defined metamethods. Note that C<area> is a
+method and must be called with the Lua syntax for methods: C<a:area()>,
+not C<a.area()>.
+
+The C type metamethod mechanism is most useful when used in conjunction
+with C libraries that are written in an object-oriented style. Creators
+return a pointer to a new instance and methods take an instance pointer
+as the first argument. Sometimes you can just point C<__index> to the
+library namespace and C<__gc> to the destructor and you're done. But
+often enough you'll want to add convenience wrappers, e.g. to return
+actual Lua strings or when returning multiple values.
+
+Some C libraries only declare instance pointers as an opaque C<void *>
+type. In this case you can use a fake type for all declarations, e.g. a
+pointer to a named (incomplete) struct will do: C<typedef struct
+foo_type *foo_handle>. The C side doesn't know what you declare with
+the LuaJIT FFI, but as long as the underlying types are compatible,
+everything still works.
+
+=head2 Translating C Idioms
+
+Here's a list of common C idioms and their translation to the LuaJIT
+FFI:
+
+Idiom
+
+C code
+
+Lua code
+
+Pointer dereference
+
+C<int *p;>
+
+x = *p;
+
+*p = y;
+
+x = B<p[0]>
+
+B<p[0]> = y
+
+Pointer indexing
+
+C<int i, *p;>
+
+x = p[i];
+
+p[i+1] = y;
+
+x = p[i]
+
+p[i+1] = y
+
+Array indexing
+
+C<int i, a[];>
+
+x = a[i];
+
+a[i+1] = y;
+
+x = a[i]
+
+a[i+1] = y
+
+C<struct>/C<union> dereference
+
+C<struct foo s;>
+
+x = s.field;
+
+s.field = y;
+
+x = s.field
+
+s.field = y
+
+C<struct>/C<union> pointer deref.
+
+C<struct foo *sp;>
+
+x = sp-E<gt>field;
+
+sp-E<gt>field = y;
+
+x = B<s.field>
+
+B<s.field> = y
+
+Pointer arithmetic
+
+C<int i, *p;>
+
+x = p + i;
+
+y = p - i;
+
+x = p + i
+
+y = p - i
+
+Pointer difference
+
+C<int *p1, *p2;>
+
+C<x = p1 - p2;>
+
+C<x = p1 - p2>
+
+Array element pointer
+
+C<int i, a[];>
+
+C<x = &a[i];>
+
+C<x = B<a+i>>
+
+Cast pointer to address
+
+C<int *p;>
+
+C<x = (intptr_t)p;>
+
+x = tonumber(
+
+ffi.cast("intptr_t",
+
+p))
+
+Functions with outargs
+
+C<void foo(int *inoutlen);>
+
+int len = x;
+
+foo(&len);
+
+y = len;
+
+local len =
+
+ffi.new("int[1]", x)
+
+foo(len)
+
+y = len[0]
+
+Vararg conversions
+
+C<int printf(char *fmt, ...);>
+
+printf("%g", 1.0);
+
+printf("%d", 1);
+
+printf("%g", 1);
+
+printf("%d",
+
+B<ffi.new("int", 1)>)
+
+=head2 To Cache or Not to Cache
+
+It's a common Lua idiom to cache library functions in local variables
+or upvalues, e.g.:
+
+ local byte, char = string.byte, string.char
+ local function foo(x)
+   return char(byte(x)+1)
+ end
+
+This replaces several hash-table lookups with a (faster) direct use of
+a local or an upvalue. This is less important with LuaJIT, since the
+JIT compiler optimizes hash-table lookups a lot and is even able to
+hoist most of them out of the inner loops. It can't eliminate I<all> of
+them, though, and it saves some typing for often-used functions. So
+there's still a place for this, even with LuaJIT.
+
+The situation is a bit different with C function calls via the FFI
+library. The JIT compiler has special logic to eliminate I<all of the
+lookup overhead> for functions resolved from a C library namespace!
+Thus it's not helpful and actually counter-productive to cache
+individual C functions like this:
+
+ local funca, funcb = ffi.C.funca, ffi.C.funcb -- Not helpful!
+ local function foo(x, n)
+   for i=1,n do funcb(funca(x, i), 1) end
+ end
+
+This turns them into indirect calls and generates bigger and slower
+machine code. Instead you'll want to cache the namespace itself and
+rely on the JIT compiler to eliminate the lookups:
+
+ local C = ffi.C          -- Instead use this!
+ local function foo(x, n)
+   for i=1,n do C.funcb(C.funca(x, i), 1) end
+ end
+
+This generates both shorter and faster code. So B<don't cache C
+functions>, but B<do> cache namespaces! Most often the namespace is
+already in a local variable at an outer scope, e.g. from C<local lib =
+ffi.load(...)>. Note that copying it to a local variable in the
+function scope is unnecessary.
+
+----
+
+Copyright E<copy> 2005-2016 Mike Pall E<middot> Contact
+
+=cut
+
+#Pod::HTML2Pod conversion notes:
+#From file ext_ffi_tutorial.html
+# 22557 bytes of input
+#Wed Jun 29 13:18:15 2016 agentzh
+# No a_name switch not specified, so will not try to render <a name='...'>
+# No a_href switch not specified, so will not try to render <a href='...'>
+# Deleting phrasal "code" element (`tt_100) because it has super-phrasal elements (`br_33, `br_34) as children.
+# Deleting phrasal "code" element (`tt_99) because it has super-phrasal elements (`br_31, `br_32) as children.
+# Deleting phrasal "b" element (`b_8) because it has super-phrasal elements (`br_27, `br_28, `br_29) as children.
+# Deleting phrasal "code" element (`tt_97) because it has super-phrasal elements (`br_27, `br_28, `br_29) as children.
+# Deleting phrasal "code" element (`tt_96) because it has super-phrasal elements (`br_25, `br_26) as children.
+# Deleting phrasal "b" element (`b_7) because it has super-phrasal elements (`br_22, `br_23) as children.
+# Deleting phrasal "code" element (`tt_94) because it has super-phrasal elements (`br_22, `br_23) as children.
+# Deleting phrasal "code" element (`tt_85) because it has super-phrasal elements (`br_18) as children.
+# Deleting phrasal "code" element (`tt_84) because it has super-phrasal elements (`br_17) as children.
+# Deleting phrasal "code" element (`tt_82) because it has super-phrasal elements (`br_15) as children.
+# Deleting phrasal "code" element (`tt_81) because it has super-phrasal elements (`br_14) as children.
+# Deleting phrasal "code" element (`tt_77) because it has super-phrasal elements (`br_12) as children.
+# Deleting phrasal "code" element (`tt_76) because it has super-phrasal elements (`br_11) as children.
+# Deleting phrasal "code" element (`tt_72) because it has super-phrasal elements (`br_9) as children.
+# Deleting phrasal "code" element (`tt_71) because it has super-phrasal elements (`br_8) as children.
+# Deleting phrasal "code" element (`tt_69) because it has super-phrasal elements (`br_6) as children.
+# Deleting phrasal "code" element (`tt_68) because it has super-phrasal elements (`br_5) as children.
+# Deleting phrasal "code" element (`tt_66) because it has super-phrasal elements (`br_3) as children.
+# Deleting phrasal "code" element (`tt_65) because it has super-phrasal elements (`br_2) as children.